2026-02-21T22:02:09.190 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T22:02:09.208 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T22:02:09.292 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058 branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle description: rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{bitmap} base mem$/{low} onode-segment$/{512K-onoff} write$/{random/{compr$/{yes$/{zstd}} random}}}} rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/snaps-few-objects} email: null first_in_suite: false flavor: default job_id: '63058' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: mon ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 30 debug ms: 20 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore debug onode segmentation random: true bluestore fsck on mount: true bluestore onode segment size: 512K bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd memory target: 1610612736 osd objectstore: bluestore osd op num shards: 1 osd op num threads per shard: 2 osd op queue: debug_random osd op queue cut off: debug_random osd_mclock_skip_benchmark: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - \(MON_NETSPLIT) - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: 7c8cf961761b37084e637a305d42f77c52437802 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: 7c8cf961761b37084e637a305d42f77c52437802 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 53f8441172c46e34523029ab88f9db7a45514e84 owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 1998 sha1: 7c8cf961761b37084e637a305d42f77c52437802 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa suite_relpath: qa suite_repo: https://github.com/NitzanMordhai/ceph.git suite_sha1: 53f8441172c46e34523029ab88f9db7a45514e84 targets: trial102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOXave8YfKZ3yfEKhO4KtEEL6Zm3dIpp31jkVw/Ay1f/vmGPVrpDc3cd3VzpniI7i753dQICxHsRHSSJ841vX8= trial178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPw7rKHKMaXRfTPWFRma0ND1/z2SVZuyD6LRPYnFdpzvXH2KYesDg4i5/rTpqPlw05zWyWs5H0jQNpRHLGtFYFM= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 - rados: clients: - client.0 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_21:55:34 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T22:02:09.292 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa; will attempt to use it 2026-02-21T22:02:09.293 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa/tasks 2026-02-21T22:02:09.293 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T22:02:09.294 INFO:teuthology.task.internal:Checking packages... 2026-02-21T22:02:09.294 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7c8cf961761b37084e637a305d42f77c52437802' 2026-02-21T22:02:09.294 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T22:02:09.294 INFO:teuthology.packaging:ref: None 2026-02-21T22:02:09.294 INFO:teuthology.packaging:tag: None 2026-02-21T22:02:09.294 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T22:02:09.294 INFO:teuthology.packaging:sha1: 7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:02:09.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T22:02:09.418 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:02:09.419 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T22:02:09.432 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T22:02:09.432 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T22:02:09.447 INFO:teuthology.task.internal:Saving configuration 2026-02-21T22:02:09.454 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T22:02:09.467 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T22:02:09.523 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial102.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 21:56:36.604202', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILDo9OH9ChpM/BOeAKR6AxAf8qejFbensBPEy3fdVyyx'} 2026-02-21T22:02:09.585 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial178.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 21:56:36.605137', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPTTDmncvAOv98A3BheTx76qOifLyx3xleHn5cKAtBJX'} 2026-02-21T22:02:09.585 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T22:02:09.599 INFO:teuthology.task.internal:roles: ubuntu@trial102.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2026-02-21T22:02:09.599 INFO:teuthology.task.internal:roles: ubuntu@trial178.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2026-02-21T22:02:09.599 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T22:02:09.747 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x77f548e749a0>, signals=[15]) 2026-02-21T22:02:09.747 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T22:02:09.760 INFO:teuthology.task.internal:Opening connections... 2026-02-21T22:02:09.760 DEBUG:teuthology.task.internal:connecting to ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:02:09.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:02:09.829 DEBUG:teuthology.task.internal:connecting to ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:02:09.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:02:09.901 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T22:02:09.912 DEBUG:teuthology.orchestra.run.trial102:> uname -m 2026-02-21T22:02:09.916 INFO:teuthology.orchestra.run.trial102.stdout:x86_64 2026-02-21T22:02:09.916 DEBUG:teuthology.orchestra.run.trial102:> cat /etc/os-release 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:NAME="Ubuntu" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:VERSION_ID="22.04" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:VERSION_CODENAME=jammy 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:ID=ubuntu 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:ID_LIKE=debian 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-21T22:02:09.965 INFO:teuthology.orchestra.run.trial102.stdout:UBUNTU_CODENAME=jammy 2026-02-21T22:02:09.966 INFO:teuthology.lock.ops:Updating trial102.front.sepia.ceph.com on lock server 2026-02-21T22:02:10.034 DEBUG:teuthology.orchestra.run.trial178:> uname -m 2026-02-21T22:02:10.038 INFO:teuthology.orchestra.run.trial178.stdout:x86_64 2026-02-21T22:02:10.038 DEBUG:teuthology.orchestra.run.trial178:> cat /etc/os-release 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:NAME="Ubuntu" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:VERSION_ID="22.04" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:VERSION_CODENAME=jammy 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:ID=ubuntu 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:ID_LIKE=debian 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-21T22:02:10.084 INFO:teuthology.orchestra.run.trial178.stdout:UBUNTU_CODENAME=jammy 2026-02-21T22:02:10.084 INFO:teuthology.lock.ops:Updating trial178.front.sepia.ceph.com on lock server 2026-02-21T22:02:10.149 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T22:02:10.162 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T22:02:10.175 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T22:02:10.176 DEBUG:teuthology.orchestra.run.trial102:> test '!' -e /home/ubuntu/cephtest 2026-02-21T22:02:10.177 DEBUG:teuthology.orchestra.run.trial178:> test '!' -e /home/ubuntu/cephtest 2026-02-21T22:02:10.180 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T22:02:10.192 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T22:02:10.192 DEBUG:teuthology.orchestra.run.trial102:> test -z $(ls -A /var/lib/ceph) 2026-02-21T22:02:10.223 DEBUG:teuthology.orchestra.run.trial178:> test -z $(ls -A /var/lib/ceph) 2026-02-21T22:02:10.225 INFO:teuthology.orchestra.run.trial102.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-21T22:02:10.227 INFO:teuthology.orchestra.run.trial178.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-21T22:02:10.228 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T22:02:10.359 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T22:02:10.380 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-21T22:02:10.380 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-21T22:02:10.380 DEBUG:teuthology.orchestra.run.trial102:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T22:02:10.380 DEBUG:teuthology.orchestra.run.trial178:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T22:02:10.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:02:10.384 DEBUG:teuthology.orchestra.run.trial102:> uname -r 2026-02-21T22:02:10.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:02:10.384 DEBUG:teuthology.orchestra.run.trial178:> uname -r 2026-02-21T22:02:10.428 INFO:teuthology.orchestra.run.trial178.stdout:5.15.0-170-generic 2026-02-21T22:02:10.428 INFO:teuthology.task.kernel:Running kernel on trial178: 5.15.0-170-generic 2026-02-21T22:02:10.428 DEBUG:teuthology.orchestra.run.trial178:> sudo apt-get clean 2026-02-21T22:02:10.430 INFO:teuthology.orchestra.run.trial102.stdout:5.15.0-170-generic 2026-02-21T22:02:10.430 INFO:teuthology.task.kernel:Running kernel on trial102: 5.15.0-170-generic 2026-02-21T22:02:10.430 DEBUG:teuthology.orchestra.run.trial102:> sudo apt-get clean 2026-02-21T22:02:10.486 DEBUG:teuthology.orchestra.run.trial178:> sudo apt-get update 2026-02-21T22:02:10.492 DEBUG:teuthology.orchestra.run.trial102:> sudo apt-get update 2026-02-21T22:02:10.916 INFO:teuthology.orchestra.run.trial178.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T22:02:10.985 INFO:teuthology.orchestra.run.trial102.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T22:02:10.990 INFO:teuthology.orchestra.run.trial178.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-21T22:02:11.075 INFO:teuthology.orchestra.run.trial102.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-21T22:02:11.271 INFO:teuthology.orchestra.run.trial178.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-21T22:02:11.394 INFO:teuthology.orchestra.run.trial178.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-21T22:02:11.411 INFO:teuthology.orchestra.run.trial102.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-21T22:02:11.509 INFO:teuthology.orchestra.run.trial178.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-21T22:02:11.560 INFO:teuthology.orchestra.run.trial102.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-21T22:02:11.700 INFO:teuthology.orchestra.run.trial102.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-21T22:02:11.837 INFO:teuthology.orchestra.run.trial178.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-21T22:02:11.847 INFO:teuthology.orchestra.run.trial178.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-21T22:02:11.847 INFO:teuthology.orchestra.run.trial178.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-21T22:02:11.876 INFO:teuthology.orchestra.run.trial178.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-21T22:02:11.882 INFO:teuthology.orchestra.run.trial178.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-21T22:02:11.882 INFO:teuthology.orchestra.run.trial178.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-21T22:02:12.011 INFO:teuthology.orchestra.run.trial178.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-21T22:02:12.029 INFO:teuthology.orchestra.run.trial178.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-21T22:02:12.100 INFO:teuthology.orchestra.run.trial102.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-21T22:02:12.100 INFO:teuthology.orchestra.run.trial178.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-21T22:02:12.108 INFO:teuthology.orchestra.run.trial178.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-21T22:02:12.108 INFO:teuthology.orchestra.run.trial178.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-21T22:02:12.113 INFO:teuthology.orchestra.run.trial102.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-21T22:02:12.113 INFO:teuthology.orchestra.run.trial102.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-21T22:02:12.137 INFO:teuthology.orchestra.run.trial178.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-21T22:02:12.137 INFO:teuthology.orchestra.run.trial178.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-21T22:02:12.148 INFO:teuthology.orchestra.run.trial102.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-21T22:02:12.155 INFO:teuthology.orchestra.run.trial102.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-21T22:02:12.156 INFO:teuthology.orchestra.run.trial102.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-21T22:02:12.257 INFO:teuthology.orchestra.run.trial178.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-21T22:02:12.313 INFO:teuthology.orchestra.run.trial102.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-21T22:02:12.335 INFO:teuthology.orchestra.run.trial102.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-21T22:02:12.423 INFO:teuthology.orchestra.run.trial102.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-21T22:02:12.431 INFO:teuthology.orchestra.run.trial102.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-21T22:02:12.432 INFO:teuthology.orchestra.run.trial102.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-21T22:02:12.466 INFO:teuthology.orchestra.run.trial102.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-21T22:02:12.466 INFO:teuthology.orchestra.run.trial102.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-21T22:02:12.609 INFO:teuthology.orchestra.run.trial102.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-21T22:02:15.916 INFO:teuthology.orchestra.run.trial178.stdout:Fetched 22.6 MB in 3s (8419 kB/s) 2026-02-21T22:02:16.094 INFO:teuthology.orchestra.run.trial102.stdout:Fetched 22.6 MB in 3s (7812 kB/s) 2026-02-21T22:02:16.451 INFO:teuthology.orchestra.run.trial178.stdout:Reading package lists... 2026-02-21T22:02:16.464 DEBUG:teuthology.orchestra.run.trial178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-21T22:02:16.497 INFO:teuthology.orchestra.run.trial178.stdout:Reading package lists... 2026-02-21T22:02:16.608 INFO:teuthology.orchestra.run.trial178.stdout:Building dependency tree... 2026-02-21T22:02:16.608 INFO:teuthology.orchestra.run.trial178.stdout:Reading state information... 2026-02-21T22:02:16.631 INFO:teuthology.orchestra.run.trial102.stdout:Reading package lists... 2026-02-21T22:02:16.644 DEBUG:teuthology.orchestra.run.trial102:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-21T22:02:16.677 INFO:teuthology.orchestra.run.trial102.stdout:Reading package lists... 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout:linux-image-generic set to manually installed. 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout: sg3-utils-udev 2026-02-21T22:02:16.697 INFO:teuthology.orchestra.run.trial178.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T22:02:16.717 INFO:teuthology.orchestra.run.trial178.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T22:02:16.719 DEBUG:teuthology.orchestra.run.trial178:> dpkg -s linux-image-generic 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Package: linux-image-generic 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Status: install ok installed 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Priority: optional 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Section: kernel 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Installed-Size: 21 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Maintainer: Ubuntu Kernel Team 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Architecture: amd64 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Source: linux-meta 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Version: 5.15.0.170.159 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Recommends: thermald 2026-02-21T22:02:16.770 INFO:teuthology.orchestra.run.trial178.stdout:Description: Generic Linux kernel image 2026-02-21T22:02:16.771 INFO:teuthology.orchestra.run.trial178.stdout: This package will always depend on the latest generic kernel image 2026-02-21T22:02:16.771 INFO:teuthology.orchestra.run.trial178.stdout: available. 2026-02-21T22:02:16.771 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T22:02:16.771 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-21T22:02:16.771 DEBUG:teuthology.orchestra.run.trial178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T22:02:16.790 INFO:teuthology.orchestra.run.trial102.stdout:Building dependency tree... 2026-02-21T22:02:16.790 INFO:teuthology.orchestra.run.trial102.stdout:Reading state information... 2026-02-21T22:02:16.821 INFO:teuthology.orchestra.run.trial178.stdout:ttyS1 2026-02-21T22:02:16.854 DEBUG:teuthology.parallel:result is None 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout:linux-image-generic set to manually installed. 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout: sg3-utils-udev 2026-02-21T22:02:16.877 INFO:teuthology.orchestra.run.trial102.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T22:02:16.897 INFO:teuthology.orchestra.run.trial102.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T22:02:16.899 DEBUG:teuthology.orchestra.run.trial102:> dpkg -s linux-image-generic 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Package: linux-image-generic 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Status: install ok installed 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Priority: optional 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Section: kernel 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Installed-Size: 21 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Maintainer: Ubuntu Kernel Team 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Architecture: amd64 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Source: linux-meta 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Version: 5.15.0.170.159 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Recommends: thermald 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout:Description: Generic Linux kernel image 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout: This package will always depend on the latest generic kernel image 2026-02-21T22:02:16.951 INFO:teuthology.orchestra.run.trial102.stdout: available. 2026-02-21T22:02:16.951 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T22:02:16.951 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T22:02:16.951 DEBUG:teuthology.orchestra.run.trial102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T22:02:17.002 INFO:teuthology.orchestra.run.trial102.stdout:ttyS1 2026-02-21T22:02:17.027 DEBUG:teuthology.parallel:result is None 2026-02-21T22:02:17.027 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T22:02:17.039 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T22:02:17.039 DEBUG:teuthology.orchestra.run.trial102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T22:02:17.075 DEBUG:teuthology.orchestra.run.trial178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T22:02:17.079 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T22:02:17.090 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T22:02:17.107 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T22:02:17.108 DEBUG:teuthology.orchestra.run.trial102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T22:02:17.119 DEBUG:teuthology.orchestra.run.trial178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T22:02:17.126 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T22:02:17.141 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T22:02:17.141 DEBUG:teuthology.orchestra.run.trial102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T22:02:17.163 DEBUG:teuthology.orchestra.run.trial178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T22:02:17.170 INFO:teuthology.orchestra.run.trial102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:02:17.173 INFO:teuthology.orchestra.run.trial178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:02:17.176 INFO:teuthology.orchestra.run.trial102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:02:17.179 INFO:teuthology.orchestra.run.trial178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T22:02:17.180 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T22:02:17.196 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T22:02:17.197 DEBUG:teuthology.orchestra.run.trial102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T22:02:17.219 DEBUG:teuthology.orchestra.run.trial178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T22:02:17.228 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T22:02:17.244 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T22:02:17.244 DEBUG:teuthology.orchestra.run.trial102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T22:02:17.267 DEBUG:teuthology.orchestra.run.trial178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T22:02:17.272 DEBUG:teuthology.orchestra.run.trial102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:02:17.313 DEBUG:teuthology.orchestra.run.trial102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:02:17.357 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:02:17.357 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T22:02:17.405 DEBUG:teuthology.orchestra.run.trial178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:02:17.409 DEBUG:teuthology.orchestra.run.trial178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:02:17.456 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:02:17.456 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T22:02:17.506 DEBUG:teuthology.orchestra.run.trial102:> sudo service rsyslog restart 2026-02-21T22:02:17.507 DEBUG:teuthology.orchestra.run.trial178:> sudo service rsyslog restart 2026-02-21T22:02:17.566 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T22:02:17.577 INFO:teuthology.task.internal:Starting timer... 2026-02-21T22:02:17.577 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T22:02:17.593 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T22:02:17.607 DEBUG:teuthology.task.selinux:Excluding trial102: OS 'ubuntu' does not support SELinux 2026-02-21T22:02:17.607 DEBUG:teuthology.task.selinux:Excluding trial178: OS 'ubuntu' does not support SELinux 2026-02-21T22:02:17.607 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T22:02:17.607 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-21T22:02:17.607 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T22:02:17.607 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T22:02:17.621 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T22:02:17.627 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T22:02:17.627 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial102.front.sepia.ceph.com,trial178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T22:04:25.074 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial102.front.sepia.ceph.com'), Remote(name='ubuntu@trial178.front.sepia.ceph.com')] 2026-02-21T22:04:25.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial102.front.sepia.ceph.com' 2026-02-21T22:04:25.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:04:25.188 DEBUG:teuthology.orchestra.run.trial102:> true 2026-02-21T22:04:25.439 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial102.front.sepia.ceph.com' 2026-02-21T22:04:25.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial178.front.sepia.ceph.com' 2026-02-21T22:04:25.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T22:04:25.528 DEBUG:teuthology.orchestra.run.trial178:> true 2026-02-21T22:04:25.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial178.front.sepia.ceph.com' 2026-02-21T22:04:25.780 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T22:04:25.798 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T22:04:25.799 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T22:04:25.799 DEBUG:teuthology.orchestra.run.trial102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:04:25.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T22:04:25.802 DEBUG:teuthology.orchestra.run.trial178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Command line: ntpd -gq 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: ---------------------------------------------------- 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: ntp-4 is maintained by Network Time Foundation, 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: corporation. Support and training for ntp-4 are 2026-02-21T22:04:25.819 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: available at https://www.nwtime.org/support 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: ---------------------------------------------------- 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: proto: precision = 0.020 usec (-25) 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: basedate set to 2022-02-04 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: gps base set to 2022-02-06 (week 2196) 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: restrict ::: KOD does nothing without LIMITED. 2026-02-21T22:04:25.820 INFO:teuthology.orchestra.run.trial102.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T22:04:25.821 INFO:teuthology.orchestra.run.trial102.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-21T22:04:25.821 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen and drop on 0 v6wildcard [::]:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen normally on 2 lo 127.0.0.1:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen normally on 3 enp1s0f0 10.20.193.102:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen normally on 4 lo [::1]:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:6230%2]:123 2026-02-21T22:04:25.822 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:25 ntpd[9414]: Listening on routing socket on fd #22 for interface updates 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Command line: ntpd -gq 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: ---------------------------------------------------- 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: ntp-4 is maintained by Network Time Foundation, 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: corporation. Support and training for ntp-4 are 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: available at https://www.nwtime.org/support 2026-02-21T22:04:25.843 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: ---------------------------------------------------- 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: proto: precision = 0.020 usec (-25) 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: basedate set to 2022-02-04 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: gps base set to 2022-02-06 (week 2196) 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T22:04:25.844 INFO:teuthology.orchestra.run.trial178.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-21T22:04:25.845 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: restrict ::: KOD does nothing without LIMITED. 2026-02-21T22:04:25.845 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen and drop on 0 v6wildcard [::]:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen normally on 2 lo 127.0.0.1:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen normally on 3 enp1s0f0 10.20.193.178:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen normally on 4 lo [::1]:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:62b0%2]:123 2026-02-21T22:04:25.846 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:25 ntpd[9426]: Listening on routing socket on fd #22 for interface updates 2026-02-21T22:04:26.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:26.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:26.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:26.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:26.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:26.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:26.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:26.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:26.980 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:26.980 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:26.980 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:26.981 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:26 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:27.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:27.840 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:27.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.850 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:27.850 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:27.894 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:27.894 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:27.894 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:27.894 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:28.056 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:28.056 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:28.056 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:28.056 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:27 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:28.842 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:28.842 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:28.842 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:28.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:28.845 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:28.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:28.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:28.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:28.979 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:28.979 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:28.980 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:28.980 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:28 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:29.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:29.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:29.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:29.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:29.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:29.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:29.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:29.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:30.056 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:30.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:29 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.844 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:30.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:30.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:30.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:30.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:30.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:30.985 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:30.985 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:30.985 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:30.985 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:30 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:31.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:31.839 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:31.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:31.849 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:31.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:31.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:31.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:31.893 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:32.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:32.057 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:31 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:32.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.843 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.844 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.844 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.844 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.844 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:32.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:32.846 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:32.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:32.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:32.847 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:32.847 INFO:teuthology.orchestra.run.trial178.stdout:21 Feb 22:04:32 ntpd[9426]: ntpd: time slew +0.012079 s 2026-02-21T22:04:32.847 INFO:teuthology.orchestra.run.trial178.stdout:ntpd: time slew +0.012079s 2026-02-21T22:04:32.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-21T22:04:32.848 INFO:teuthology.orchestra.run.trial178.stderr:21 Feb 22:04:32 ntpd[9426]: can't open /var/log/ntpstats/loopstats.20260221: Permission denied 2026-02-21T22:04:32.907 INFO:teuthology.orchestra.run.trial178.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T22:04:32.908 INFO:teuthology.orchestra.run.trial178.stdout:============================================================================== 2026-02-21T22:04:32.908 INFO:teuthology.orchestra.run.trial178.stdout: 172-233-155-39. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T22:04:32.908 INFO:teuthology.orchestra.run.trial178.stdout: time1.lshiy.com .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T22:04:32.908 INFO:teuthology.orchestra.run.trial178.stdout: LAX.CALTICK.NET .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T22:04:32.908 INFO:teuthology.orchestra.run.trial178.stdout: 69.172.133.130 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T22:04:33.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:33.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:33.819 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:33.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:33.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:33.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:33.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:33.820 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stdout:21 Feb 22:04:33 ntpd[9414]: ntpd: time set +0.135229 s 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stdout:ntpd: time set +0.135229s 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-21T22:04:33.838 INFO:teuthology.orchestra.run.trial102.stderr:21 Feb 22:04:33 ntpd[9414]: can't open /var/log/ntpstats/loopstats.20260221: Permission denied 2026-02-21T22:04:33.874 INFO:teuthology.orchestra.run.trial102.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T22:04:33.874 INFO:teuthology.orchestra.run.trial102.stdout:============================================================================== 2026-02-21T22:04:33.874 INFO:teuthology.orchestra.run.trial102.stdout: 198.23.133.146 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T22:04:33.875 INFO:teuthology.run_tasks:Running task install... 2026-02-21T22:04:33.886 DEBUG:teuthology.task.install:project ceph 2026-02-21T22:04:33.886 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}} 2026-02-21T22:04:33.886 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'} 2026-02-21T22:04:33.886 INFO:teuthology.task.install:Using flavor: default 2026-02-21T22:04:33.889 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T22:04:33.889 INFO:teuthology.task.install:extra packages: [] 2026-02-21T22:04:33.889 DEBUG:teuthology.orchestra.run.trial102:> sudo apt-key list | grep Ceph 2026-02-21T22:04:33.889 DEBUG:teuthology.orchestra.run.trial178:> sudo apt-key list | grep Ceph 2026-02-21T22:04:33.942 INFO:teuthology.orchestra.run.trial178.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-21T22:04:33.967 INFO:teuthology.orchestra.run.trial102.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-21T22:04:33.971 INFO:teuthology.orchestra.run.trial178.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-21T22:04:33.972 INFO:teuthology.orchestra.run.trial178.stdout:uid [ unknown] Ceph.com (release key) 2026-02-21T22:04:33.972 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-21T22:04:33.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:04:34.077 INFO:teuthology.orchestra.run.trial102.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-21T22:04:34.077 INFO:teuthology.orchestra.run.trial102.stdout:uid [ unknown] Ceph.com (release key) 2026-02-21T22:04:34.077 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-21T22:04:34.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:04:34.170 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default/ 2026-02-21T22:04:34.170 INFO:teuthology.task.install.deb:Package version is 20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:04:34.199 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default/ 2026-02-21T22:04:34.199 INFO:teuthology.task.install.deb:Package version is 20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:04:34.524 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:04:34.525 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-21T22:04:34.535 DEBUG:teuthology.orchestra.run.trial178:> sudo apt-get update 2026-02-21T22:04:34.571 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:04:34.571 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-21T22:04:34.582 DEBUG:teuthology.orchestra.run.trial102:> sudo apt-get update 2026-02-21T22:04:34.914 INFO:teuthology.orchestra.run.trial178.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy InRelease 2026-02-21T22:04:34.962 INFO:teuthology.orchestra.run.trial102.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy InRelease 2026-02-21T22:04:34.978 INFO:teuthology.orchestra.run.trial178.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-21T22:04:35.030 INFO:teuthology.orchestra.run.trial102.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-21T22:04:35.035 INFO:teuthology.orchestra.run.trial178.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T22:04:35.047 INFO:teuthology.orchestra.run.trial178.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-21T22:04:35.080 INFO:teuthology.orchestra.run.trial102.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T22:04:35.102 INFO:teuthology.orchestra.run.trial102.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-21T22:04:35.117 INFO:teuthology.orchestra.run.trial178.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-21T22:04:35.118 INFO:teuthology.orchestra.run.trial178.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-21T22:04:35.162 INFO:teuthology.orchestra.run.trial102.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-21T22:04:35.173 INFO:teuthology.orchestra.run.trial102.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-21T22:04:35.205 INFO:teuthology.orchestra.run.trial178.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-21T22:04:35.250 INFO:teuthology.orchestra.run.trial102.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-21T22:04:35.293 INFO:teuthology.orchestra.run.trial178.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-21T22:04:35.338 INFO:teuthology.orchestra.run.trial102.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-21T22:04:35.434 INFO:teuthology.orchestra.run.trial178.stdout:Fetched 26.9 kB in 1s (36.4 kB/s) 2026-02-21T22:04:35.481 INFO:teuthology.orchestra.run.trial102.stdout:Fetched 26.9 kB in 1s (36.7 kB/s) 2026-02-21T22:04:35.968 INFO:teuthology.orchestra.run.trial178.stdout:Reading package lists... 2026-02-21T22:04:35.981 DEBUG:teuthology.orchestra.run.trial178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5425-g7c8cf961-1jammy cephadm=20.3.0-5425-g7c8cf961-1jammy ceph-mds=20.3.0-5425-g7c8cf961-1jammy ceph-mgr=20.3.0-5425-g7c8cf961-1jammy ceph-osd=20.3.0-5425-g7c8cf961-1jammy ceph-osd-classic=20.3.0-5425-g7c8cf961-1jammy ceph-common=20.3.0-5425-g7c8cf961-1jammy ceph-fuse=20.3.0-5425-g7c8cf961-1jammy ceph-test=20.3.0-5425-g7c8cf961-1jammy ceph-volume=20.3.0-5425-g7c8cf961-1jammy radosgw=20.3.0-5425-g7c8cf961-1jammy python3-rados=20.3.0-5425-g7c8cf961-1jammy python3-rgw=20.3.0-5425-g7c8cf961-1jammy python3-cephfs=20.3.0-5425-g7c8cf961-1jammy python3-rbd=20.3.0-5425-g7c8cf961-1jammy libcephfs2=20.3.0-5425-g7c8cf961-1jammy libcephfs-dev=20.3.0-5425-g7c8cf961-1jammy librados2=20.3.0-5425-g7c8cf961-1jammy librbd1=20.3.0-5425-g7c8cf961-1jammy rbd-fuse=20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:04:36.006 INFO:teuthology.orchestra.run.trial102.stdout:Reading package lists... 2026-02-21T22:04:36.015 INFO:teuthology.orchestra.run.trial178.stdout:Reading package lists... 2026-02-21T22:04:36.020 DEBUG:teuthology.orchestra.run.trial102:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5425-g7c8cf961-1jammy cephadm=20.3.0-5425-g7c8cf961-1jammy ceph-mds=20.3.0-5425-g7c8cf961-1jammy ceph-mgr=20.3.0-5425-g7c8cf961-1jammy ceph-osd=20.3.0-5425-g7c8cf961-1jammy ceph-osd-classic=20.3.0-5425-g7c8cf961-1jammy ceph-common=20.3.0-5425-g7c8cf961-1jammy ceph-fuse=20.3.0-5425-g7c8cf961-1jammy ceph-test=20.3.0-5425-g7c8cf961-1jammy ceph-volume=20.3.0-5425-g7c8cf961-1jammy radosgw=20.3.0-5425-g7c8cf961-1jammy python3-rados=20.3.0-5425-g7c8cf961-1jammy python3-rgw=20.3.0-5425-g7c8cf961-1jammy python3-cephfs=20.3.0-5425-g7c8cf961-1jammy python3-rbd=20.3.0-5425-g7c8cf961-1jammy libcephfs2=20.3.0-5425-g7c8cf961-1jammy libcephfs-dev=20.3.0-5425-g7c8cf961-1jammy librados2=20.3.0-5425-g7c8cf961-1jammy librbd1=20.3.0-5425-g7c8cf961-1jammy rbd-fuse=20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:04:36.053 INFO:teuthology.orchestra.run.trial102.stdout:Reading package lists... 2026-02-21T22:04:36.126 INFO:teuthology.orchestra.run.trial178.stdout:Building dependency tree... 2026-02-21T22:04:36.127 INFO:teuthology.orchestra.run.trial178.stdout:Reading state information... 2026-02-21T22:04:36.162 INFO:teuthology.orchestra.run.trial102.stdout:Building dependency tree... 2026-02-21T22:04:36.162 INFO:teuthology.orchestra.run.trial102.stdout:Reading state information... 2026-02-21T22:04:36.219 INFO:teuthology.orchestra.run.trial178.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T22:04:36.219 INFO:teuthology.orchestra.run.trial178.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T22:04:36.219 INFO:teuthology.orchestra.run.trial178.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T22:04:36.219 INFO:teuthology.orchestra.run.trial178.stdout: sg3-utils-udev 2026-02-21T22:04:36.219 INFO:teuthology.orchestra.run.trial178.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout:The following additional packages will be installed: 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-21T22:04:36.220 INFO:teuthology.orchestra.run.trial178.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout:Suggested packages: 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout:Recommended packages: 2026-02-21T22:04:36.221 INFO:teuthology.orchestra.run.trial178.stdout: btrfs-tools 2026-02-21T22:04:36.253 INFO:teuthology.orchestra.run.trial178.stdout:The following NEW packages will be installed: 2026-02-21T22:04:36.253 INFO:teuthology.orchestra.run.trial178.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-21T22:04:36.254 INFO:teuthology.orchestra.run.trial178.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-21T22:04:36.255 INFO:teuthology.orchestra.run.trial178.stdout: socat xmlstarlet 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: sg3-utils-udev 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout:The following additional packages will be installed: 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-21T22:04:36.257 INFO:teuthology.orchestra.run.trial102.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-21T22:04:36.258 INFO:teuthology.orchestra.run.trial102.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout:Suggested packages: 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout:Recommended packages: 2026-02-21T22:04:36.259 INFO:teuthology.orchestra.run.trial102.stdout: btrfs-tools 2026-02-21T22:04:36.288 INFO:teuthology.orchestra.run.trial102.stdout:The following NEW packages will be installed: 2026-02-21T22:04:36.288 INFO:teuthology.orchestra.run.trial102.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-21T22:04:36.289 INFO:teuthology.orchestra.run.trial102.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-21T22:04:36.290 INFO:teuthology.orchestra.run.trial102.stdout: socat xmlstarlet 2026-02-21T22:04:36.355 INFO:teuthology.orchestra.run.trial178.stdout:0 upgraded, 87 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T22:04:36.355 INFO:teuthology.orchestra.run.trial178.stdout:Need to get 702 MB of archives. 2026-02-21T22:04:36.355 INFO:teuthology.orchestra.run.trial178.stdout:After this operation, 2670 MB of additional disk space will be used. 2026-02-21T22:04:36.355 INFO:teuthology.orchestra.run.trial178.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-21T22:04:36.400 INFO:teuthology.orchestra.run.trial178.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-21T22:04:36.401 INFO:teuthology.orchestra.run.trial178.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-21T22:04:36.411 INFO:teuthology.orchestra.run.trial178.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-21T22:04:36.440 INFO:teuthology.orchestra.run.trial178.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-21T22:04:36.441 INFO:teuthology.orchestra.run.trial178.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-21T22:04:36.446 INFO:teuthology.orchestra.run.trial178.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-21T22:04:36.448 INFO:teuthology.orchestra.run.trial178.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-21T22:04:36.449 INFO:teuthology.orchestra.run.trial178.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-21T22:04:36.449 INFO:teuthology.orchestra.run.trial178.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-21T22:04:36.450 INFO:teuthology.orchestra.run.trial178.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-21T22:04:36.454 INFO:teuthology.orchestra.run.trial178.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-21T22:04:36.454 INFO:teuthology.orchestra.run.trial178.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-21T22:04:36.455 INFO:teuthology.orchestra.run.trial178.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-21T22:04:36.461 INFO:teuthology.orchestra.run.trial178.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-21T22:04:36.461 INFO:teuthology.orchestra.run.trial178.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-21T22:04:36.461 INFO:teuthology.orchestra.run.trial178.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-21T22:04:36.461 INFO:teuthology.orchestra.run.trial178.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-21T22:04:36.461 INFO:teuthology.orchestra.run.trial178.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-21T22:04:36.470 INFO:teuthology.orchestra.run.trial178.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-21T22:04:36.470 INFO:teuthology.orchestra.run.trial178.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-21T22:04:36.472 INFO:teuthology.orchestra.run.trial178.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-21T22:04:36.472 INFO:teuthology.orchestra.run.trial178.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-21T22:04:36.473 INFO:teuthology.orchestra.run.trial178.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-21T22:04:36.479 INFO:teuthology.orchestra.run.trial178.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-21T22:04:36.480 INFO:teuthology.orchestra.run.trial178.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-21T22:04:36.480 INFO:teuthology.orchestra.run.trial178.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-21T22:04:36.494 INFO:teuthology.orchestra.run.trial178.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-21T22:04:36.496 INFO:teuthology.orchestra.run.trial178.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-21T22:04:36.496 INFO:teuthology.orchestra.run.trial178.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-21T22:04:36.506 INFO:teuthology.orchestra.run.trial178.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-21T22:04:36.506 INFO:teuthology.orchestra.run.trial178.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-21T22:04:36.506 INFO:teuthology.orchestra.run.trial178.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-21T22:04:36.506 INFO:teuthology.orchestra.run.trial178.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-21T22:04:36.506 INFO:teuthology.orchestra.run.trial178.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-21T22:04:36.507 INFO:teuthology.orchestra.run.trial178.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-21T22:04:36.508 INFO:teuthology.orchestra.run.trial178.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-21T22:04:36.509 INFO:teuthology.orchestra.run.trial178.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-21T22:04:36.510 INFO:teuthology.orchestra.run.trial178.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-21T22:04:36.517 INFO:teuthology.orchestra.run.trial178.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-21T22:04:36.519 INFO:teuthology.orchestra.run.trial178.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-21T22:04:36.520 INFO:teuthology.orchestra.run.trial178.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-21T22:04:36.522 INFO:teuthology.orchestra.run.trial178.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-21T22:04:36.524 INFO:teuthology.orchestra.run.trial178.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-21T22:04:36.524 INFO:teuthology.orchestra.run.trial178.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-21T22:04:36.527 INFO:teuthology.orchestra.run.trial178.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-21T22:04:36.528 INFO:teuthology.orchestra.run.trial178.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-21T22:04:36.528 INFO:teuthology.orchestra.run.trial178.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-21T22:04:36.531 INFO:teuthology.orchestra.run.trial178.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-21T22:04:36.536 INFO:teuthology.orchestra.run.trial178.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-21T22:04:36.537 INFO:teuthology.orchestra.run.trial178.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-21T22:04:36.538 INFO:teuthology.orchestra.run.trial178.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-21T22:04:36.538 INFO:teuthology.orchestra.run.trial178.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-21T22:04:36.628 INFO:teuthology.orchestra.run.trial178.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5425-g7c8cf961-1jammy [3789 kB] 2026-02-21T22:04:36.642 INFO:teuthology.orchestra.run.trial102.stdout:0 upgraded, 87 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T22:04:36.642 INFO:teuthology.orchestra.run.trial102.stdout:Need to get 702 MB of archives. 2026-02-21T22:04:36.642 INFO:teuthology.orchestra.run.trial102.stdout:After this operation, 2670 MB of additional disk space will be used. 2026-02-21T22:04:36.642 INFO:teuthology.orchestra.run.trial102.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5425-g7c8cf961-1jammy [3789 kB] 2026-02-21T22:04:36.703 INFO:teuthology.orchestra.run.trial102.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-21T22:04:37.116 INFO:teuthology.orchestra.run.trial102.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-21T22:04:37.129 INFO:teuthology.orchestra.run.trial102.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-21T22:04:37.213 INFO:teuthology.orchestra.run.trial102.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-21T22:04:37.459 INFO:teuthology.orchestra.run.trial102.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-21T22:04:37.471 INFO:teuthology.orchestra.run.trial102.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-21T22:04:37.504 INFO:teuthology.orchestra.run.trial102.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-21T22:04:37.514 INFO:teuthology.orchestra.run.trial102.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-21T22:04:37.516 INFO:teuthology.orchestra.run.trial102.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-21T22:04:37.517 INFO:teuthology.orchestra.run.trial102.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-21T22:04:37.518 INFO:teuthology.orchestra.run.trial102.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-21T22:04:37.539 INFO:teuthology.orchestra.run.trial102.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-21T22:04:37.540 INFO:teuthology.orchestra.run.trial102.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-21T22:04:37.547 INFO:teuthology.orchestra.run.trial102.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-21T22:04:37.628 INFO:teuthology.orchestra.run.trial102.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-21T22:04:37.629 INFO:teuthology.orchestra.run.trial102.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-21T22:04:37.629 INFO:teuthology.orchestra.run.trial102.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-21T22:04:37.630 INFO:teuthology.orchestra.run.trial102.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-21T22:04:37.630 INFO:teuthology.orchestra.run.trial102.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-21T22:04:37.630 INFO:teuthology.orchestra.run.trial102.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-21T22:04:37.630 INFO:teuthology.orchestra.run.trial102.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-21T22:04:37.635 INFO:teuthology.orchestra.run.trial102.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-21T22:04:37.642 INFO:teuthology.orchestra.run.trial102.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-21T22:04:37.720 INFO:teuthology.orchestra.run.trial102.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-21T22:04:37.723 INFO:teuthology.orchestra.run.trial102.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-21T22:04:37.723 INFO:teuthology.orchestra.run.trial102.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-21T22:04:37.724 INFO:teuthology.orchestra.run.trial102.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-21T22:04:37.754 INFO:teuthology.orchestra.run.trial102.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-21T22:04:37.757 INFO:teuthology.orchestra.run.trial102.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-21T22:04:37.757 INFO:teuthology.orchestra.run.trial102.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-21T22:04:37.828 INFO:teuthology.orchestra.run.trial102.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-21T22:04:37.828 INFO:teuthology.orchestra.run.trial102.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-21T22:04:37.828 INFO:teuthology.orchestra.run.trial102.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-21T22:04:37.829 INFO:teuthology.orchestra.run.trial102.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-21T22:04:37.829 INFO:teuthology.orchestra.run.trial102.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-21T22:04:37.830 INFO:teuthology.orchestra.run.trial102.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-21T22:04:37.902 INFO:teuthology.orchestra.run.trial102.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-21T22:04:37.903 INFO:teuthology.orchestra.run.trial102.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-21T22:04:37.905 INFO:teuthology.orchestra.run.trial102.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-21T22:04:37.986 INFO:teuthology.orchestra.run.trial178.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5425-g7c8cf961-1jammy [2914 kB] 2026-02-21T22:04:37.990 INFO:teuthology.orchestra.run.trial102.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-21T22:04:37.993 INFO:teuthology.orchestra.run.trial102.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-21T22:04:37.996 INFO:teuthology.orchestra.run.trial102.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-21T22:04:38.000 INFO:teuthology.orchestra.run.trial102.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-21T22:04:38.003 INFO:teuthology.orchestra.run.trial102.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-21T22:04:38.004 INFO:teuthology.orchestra.run.trial102.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-21T22:04:38.004 INFO:teuthology.orchestra.run.trial102.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-21T22:04:38.078 INFO:teuthology.orchestra.run.trial102.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-21T22:04:38.079 INFO:teuthology.orchestra.run.trial102.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-21T22:04:38.086 INFO:teuthology.orchestra.run.trial102.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-21T22:04:38.167 INFO:teuthology.orchestra.run.trial102.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-21T22:04:38.169 INFO:teuthology.orchestra.run.trial102.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-21T22:04:38.169 INFO:teuthology.orchestra.run.trial102.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-21T22:04:38.170 INFO:teuthology.orchestra.run.trial102.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-21T22:04:38.463 INFO:teuthology.orchestra.run.trial102.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5425-g7c8cf961-1jammy [2914 kB] 2026-02-21T22:04:39.130 INFO:teuthology.orchestra.run.trial178.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5425-g7c8cf961-1jammy [882 kB] 2026-02-21T22:04:39.529 INFO:teuthology.orchestra.run.trial178.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5425-g7c8cf961-1jammy [359 kB] 2026-02-21T22:04:39.668 INFO:teuthology.orchestra.run.trial178.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5425-g7c8cf961-1jammy [32.8 kB] 2026-02-21T22:04:39.669 INFO:teuthology.orchestra.run.trial178.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5425-g7c8cf961-1jammy [225 kB] 2026-02-21T22:04:39.740 INFO:teuthology.orchestra.run.trial102.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5425-g7c8cf961-1jammy [882 kB] 2026-02-21T22:04:39.806 INFO:teuthology.orchestra.run.trial178.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5425-g7c8cf961-1jammy [88.5 kB] 2026-02-21T22:04:39.808 INFO:teuthology.orchestra.run.trial178.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5425-g7c8cf961-1jammy [339 kB] 2026-02-21T22:04:40.015 INFO:teuthology.orchestra.run.trial178.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5425-g7c8cf961-1jammy [9086 kB] 2026-02-21T22:04:40.157 INFO:teuthology.orchestra.run.trial102.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5425-g7c8cf961-1jammy [359 kB] 2026-02-21T22:04:40.349 INFO:teuthology.orchestra.run.trial102.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5425-g7c8cf961-1jammy [32.8 kB] 2026-02-21T22:04:40.350 INFO:teuthology.orchestra.run.trial102.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5425-g7c8cf961-1jammy [225 kB] 2026-02-21T22:04:40.481 INFO:teuthology.orchestra.run.trial102.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5425-g7c8cf961-1jammy [88.5 kB] 2026-02-21T22:04:40.545 INFO:teuthology.orchestra.run.trial102.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5425-g7c8cf961-1jammy [339 kB] 2026-02-21T22:04:40.745 INFO:teuthology.orchestra.run.trial102.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5425-g7c8cf961-1jammy [9086 kB] 2026-02-21T22:04:43.946 INFO:teuthology.orchestra.run.trial178.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5425-g7c8cf961-1jammy [112 kB] 2026-02-21T22:04:44.040 INFO:teuthology.orchestra.run.trial178.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5425-g7c8cf961-1jammy [261 kB] 2026-02-21T22:04:44.202 INFO:teuthology.orchestra.run.trial178.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5425-g7c8cf961-1jammy [31.5 MB] 2026-02-21T22:04:45.998 INFO:teuthology.orchestra.run.trial102.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5425-g7c8cf961-1jammy [112 kB] 2026-02-21T22:04:46.088 INFO:teuthology.orchestra.run.trial102.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5425-g7c8cf961-1jammy [261 kB] 2026-02-21T22:04:46.260 INFO:teuthology.orchestra.run.trial102.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5425-g7c8cf961-1jammy [31.5 MB] 2026-02-21T22:05:18.073 INFO:teuthology.orchestra.run.trial178.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5425-g7c8cf961-1jammy [4983 kB] 2026-02-21T22:05:21.692 INFO:teuthology.orchestra.run.trial102.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5425-g7c8cf961-1jammy [4983 kB] 2026-02-21T22:05:26.925 INFO:teuthology.orchestra.run.trial178.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5425-g7c8cf961-1jammy [294 kB] 2026-02-21T22:05:27.291 INFO:teuthology.orchestra.run.trial178.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5425-g7c8cf961-1jammy [127 kB] 2026-02-21T22:05:27.424 INFO:teuthology.orchestra.run.trial178.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5425-g7c8cf961-1jammy [926 kB] 2026-02-21T22:05:28.372 INFO:teuthology.orchestra.run.trial178.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5425-g7c8cf961-1jammy [7541 kB] 2026-02-21T22:05:30.543 INFO:teuthology.orchestra.run.trial102.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5425-g7c8cf961-1jammy [294 kB] 2026-02-21T22:05:30.961 INFO:teuthology.orchestra.run.trial102.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5425-g7c8cf961-1jammy [127 kB] 2026-02-21T22:05:31.140 INFO:teuthology.orchestra.run.trial102.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5425-g7c8cf961-1jammy [926 kB] 2026-02-21T22:05:32.367 INFO:teuthology.orchestra.run.trial102.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5425-g7c8cf961-1jammy [7541 kB] 2026-02-21T22:05:37.574 INFO:teuthology.orchestra.run.trial178.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5425-g7c8cf961-1jammy [468 MB] 2026-02-21T22:05:40.596 INFO:teuthology.orchestra.run.trial102.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5425-g7c8cf961-1jammy [468 MB] 2026-02-21T22:12:49.917 INFO:teuthology.orchestra.run.trial102.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5425-g7c8cf961-1jammy [180 kB] 2026-02-21T22:12:50.014 INFO:teuthology.orchestra.run.trial102.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5425-g7c8cf961-1jammy [14.1 kB] 2026-02-21T22:12:50.014 INFO:teuthology.orchestra.run.trial102.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5425-g7c8cf961-1jammy [1030 kB] 2026-02-21T22:12:50.537 INFO:teuthology.orchestra.run.trial102.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5425-g7c8cf961-1jammy [2437 kB] 2026-02-21T22:12:51.899 INFO:teuthology.orchestra.run.trial102.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5425-g7c8cf961-1jammy [1069 kB] 2026-02-21T22:12:52.578 INFO:teuthology.orchestra.run.trial102.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5425-g7c8cf961-1jammy [190 kB] 2026-02-21T22:12:52.711 INFO:teuthology.orchestra.run.trial102.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5425-g7c8cf961-1jammy [54.8 MB] 2026-02-21T22:13:00.608 INFO:teuthology.orchestra.run.trial178.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5425-g7c8cf961-1jammy [180 kB] 2026-02-21T22:13:00.741 INFO:teuthology.orchestra.run.trial178.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5425-g7c8cf961-1jammy [14.1 kB] 2026-02-21T22:13:00.744 INFO:teuthology.orchestra.run.trial178.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5425-g7c8cf961-1jammy [1030 kB] 2026-02-21T22:13:01.462 INFO:teuthology.orchestra.run.trial178.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5425-g7c8cf961-1jammy [2437 kB] 2026-02-21T22:13:03.041 INFO:teuthology.orchestra.run.trial178.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5425-g7c8cf961-1jammy [1069 kB] 2026-02-21T22:13:03.771 INFO:teuthology.orchestra.run.trial178.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5425-g7c8cf961-1jammy [190 kB] 2026-02-21T22:13:03.887 INFO:teuthology.orchestra.run.trial178.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5425-g7c8cf961-1jammy [54.8 MB] 2026-02-21T22:13:36.585 INFO:teuthology.orchestra.run.trial178.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5425-g7c8cf961-1jammy [8626 kB] 2026-02-21T22:13:40.037 INFO:teuthology.orchestra.run.trial102.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5425-g7c8cf961-1jammy [8626 kB] 2026-02-21T22:13:41.114 INFO:teuthology.orchestra.run.trial178.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5425-g7c8cf961-1jammy [14.2 kB] 2026-02-21T22:13:41.116 INFO:teuthology.orchestra.run.trial178.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5425-g7c8cf961-1jammy [68.6 MB] 2026-02-21T22:13:43.877 INFO:teuthology.orchestra.run.trial102.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5425-g7c8cf961-1jammy [14.2 kB] 2026-02-21T22:13:43.877 INFO:teuthology.orchestra.run.trial102.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5425-g7c8cf961-1jammy [68.6 MB] 2026-02-21T22:14:14.841 INFO:teuthology.orchestra.run.trial102.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5425-g7c8cf961-1jammy [136 kB] 2026-02-21T22:14:14.908 INFO:teuthology.orchestra.run.trial102.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5425-g7c8cf961-1jammy [45.8 kB] 2026-02-21T22:14:14.936 INFO:teuthology.orchestra.run.trial102.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5425-g7c8cf961-1jammy [32.4 kB] 2026-02-21T22:14:14.946 INFO:teuthology.orchestra.run.trial102.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5425-g7c8cf961-1jammy [44.2 kB] 2026-02-21T22:14:14.954 INFO:teuthology.orchestra.run.trial102.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5425-g7c8cf961-1jammy [18.3 MB] 2026-02-21T22:14:24.700 INFO:teuthology.orchestra.run.trial102.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5425-g7c8cf961-1jammy [93.7 kB] 2026-02-21T22:14:25.056 INFO:teuthology.orchestra.run.trial102.stdout:Fetched 702 MB in 9min 48s (1193 kB/s) 2026-02-21T22:14:25.076 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-21T22:14:25.237 INFO:teuthology.orchestra.run.trial102.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-21T22:14:25.239 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-21T22:14:25.240 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T22:14:25.262 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-21T22:14:25.274 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-21T22:14:25.275 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T22:14:25.290 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-21T22:14:25.302 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-21T22:14:25.303 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T22:14:25.323 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-21T22:14:25.331 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:25.336 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:25.367 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-21T22:14:25.375 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:25.376 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:25.392 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-21T22:14:25.400 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:25.401 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:25.422 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-21T22:14:25.430 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-21T22:14:25.431 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T22:14:25.447 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package librados2. 2026-02-21T22:14:25.455 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../07-librados2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.455 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking librados2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.498 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libnbd0. 2026-02-21T22:14:25.506 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-21T22:14:25.507 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-21T22:14:25.522 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package librbd1. 2026-02-21T22:14:25.530 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../09-librbd1_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.531 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking librbd1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.569 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libcephfs2. 2026-02-21T22:14:25.577 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.578 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libcephfs2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.598 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-rados. 2026-02-21T22:14:25.606 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.606 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-rados (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.623 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-21T22:14:25.632 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:25.632 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-ceph-argparse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.646 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-cephfs. 2026-02-21T22:14:25.654 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.654 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-cephfs (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.671 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-21T22:14:25.679 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:25.679 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.697 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-21T22:14:25.705 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-21T22:14:25.706 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T22:14:25.722 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-prettytable. 2026-02-21T22:14:25.730 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-21T22:14:25.730 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-21T22:14:25.745 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-rbd. 2026-02-21T22:14:25.753 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.754 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-rbd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.772 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-21T22:14:25.780 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-21T22:14:25.780 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T22:14:25.798 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package librgw2. 2026-02-21T22:14:25.806 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../19-librgw2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.806 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking librgw2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.905 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-rgw. 2026-02-21T22:14:25.914 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.914 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-rgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.929 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-21T22:14:25.938 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-21T22:14:25.938 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T22:14:25.953 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libradosstriper1. 2026-02-21T22:14:25.961 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.962 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libradosstriper1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:25.978 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-common. 2026-02-21T22:14:25.986 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:25.987 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.320 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-base. 2026-02-21T22:14:26.328 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:26.333 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-base (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.391 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-21T22:14:26.399 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-21T22:14:26.399 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-21T22:14:26.414 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-cheroot. 2026-02-21T22:14:26.422 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-21T22:14:26.422 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T22:14:26.439 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-21T22:14:26.448 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-21T22:14:26.448 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-21T22:14:26.463 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-21T22:14:26.471 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-21T22:14:26.471 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-21T22:14:26.486 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-21T22:14:26.495 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-21T22:14:26.495 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-21T22:14:26.509 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-tempora. 2026-02-21T22:14:26.517 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-21T22:14:26.518 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-21T22:14:26.533 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-portend. 2026-02-21T22:14:26.541 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-21T22:14:26.541 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-21T22:14:26.556 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-21T22:14:26.564 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-21T22:14:26.565 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-21T22:14:26.579 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-21T22:14:26.588 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-21T22:14:26.588 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-21T22:14:26.612 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-natsort. 2026-02-21T22:14:26.620 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-21T22:14:26.620 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-21T22:14:26.637 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-21T22:14:26.645 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:26.646 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.678 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-21T22:14:26.685 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:26.686 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.701 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr. 2026-02-21T22:14:26.709 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:26.709 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.731 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mon. 2026-02-21T22:14:26.740 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:26.740 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:26.806 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-21T22:14:26.815 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-21T22:14:26.815 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T22:14:26.832 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-21T22:14:26.840 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:26.840 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-osd-classic (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.688 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-osd. 2026-02-21T22:14:30.701 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:30.701 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-osd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.721 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph. 2026-02-21T22:14:30.734 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../42-ceph_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:30.735 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.750 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-fuse. 2026-02-21T22:14:30.759 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:30.759 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.781 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mds. 2026-02-21T22:14:30.791 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:30.791 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mds (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.823 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package cephadm. 2026-02-21T22:14:30.832 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../45-cephadm_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:30.832 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.848 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-21T22:14:30.857 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T22:14:30.857 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:30.878 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-21T22:14:30.886 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:30.887 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:30.908 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-21T22:14:30.917 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-21T22:14:30.917 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-21T22:14:30.933 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-routes. 2026-02-21T22:14:30.941 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-21T22:14:30.941 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T22:14:30.962 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-21T22:14:30.970 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:30.971 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:31.651 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-21T22:14:31.661 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-21T22:14:31.661 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T22:14:31.700 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-joblib. 2026-02-21T22:14:31.709 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-21T22:14:31.709 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T22:14:31.737 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-21T22:14:31.745 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-21T22:14:31.746 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-21T22:14:31.762 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-sklearn. 2026-02-21T22:14:31.771 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-21T22:14:31.771 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T22:14:31.856 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-21T22:14:31.865 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:31.865 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:32.023 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-cachetools. 2026-02-21T22:14:32.031 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-21T22:14:32.032 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-21T22:14:32.047 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-rsa. 2026-02-21T22:14:32.055 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-21T22:14:32.056 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-21T22:14:32.073 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-google-auth. 2026-02-21T22:14:32.082 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-21T22:14:32.082 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-21T22:14:32.100 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-21T22:14:32.108 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-21T22:14:32.109 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T22:14:32.126 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-websocket. 2026-02-21T22:14:32.135 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-21T22:14:32.135 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-21T22:14:32.137 INFO:teuthology.orchestra.run.trial178.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5425-g7c8cf961-1jammy [136 kB] 2026-02-21T22:14:32.152 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-21T22:14:32.160 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-21T22:14:32.161 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T22:14:32.222 INFO:teuthology.orchestra.run.trial178.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5425-g7c8cf961-1jammy [45.8 kB] 2026-02-21T22:14:32.254 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-21T22:14:32.262 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:32.263 INFO:teuthology.orchestra.run.trial178.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5425-g7c8cf961-1jammy [32.4 kB] 2026-02-21T22:14:32.263 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:32.277 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-21T22:14:32.285 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-21T22:14:32.286 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T22:14:32.287 INFO:teuthology.orchestra.run.trial178.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5425-g7c8cf961-1jammy [44.2 kB] 2026-02-21T22:14:32.302 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-21T22:14:32.310 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T22:14:32.311 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:32.316 INFO:teuthology.orchestra.run.trial178.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5425-g7c8cf961-1jammy [18.3 MB] 2026-02-21T22:14:32.326 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package jq. 2026-02-21T22:14:32.334 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T22:14:32.334 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:32.349 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package socat. 2026-02-21T22:14:32.358 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-21T22:14:32.358 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-21T22:14:32.379 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package xmlstarlet. 2026-02-21T22:14:32.387 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-21T22:14:32.387 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-21T22:14:32.423 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-test. 2026-02-21T22:14:32.431 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:32.432 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-test (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.092 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package ceph-volume. 2026-02-21T22:14:33.101 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:33.101 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking ceph-volume (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.125 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-21T22:14:33.134 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:33.134 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libcephfs-daemon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.149 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-21T22:14:33.158 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:33.158 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libcephfs-proxy2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.174 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-21T22:14:33.182 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:33.182 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking libcephfs-dev (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.200 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package nvme-cli. 2026-02-21T22:14:33.208 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-21T22:14:33.209 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T22:14:33.242 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-21T22:14:33.251 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T22:14:33.251 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:33.281 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-21T22:14:33.290 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-21T22:14:33.291 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-21T22:14:33.305 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-pluggy. 2026-02-21T22:14:33.314 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-21T22:14:33.314 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-21T22:14:33.330 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-psutil. 2026-02-21T22:14:33.339 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-21T22:14:33.339 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-21T22:14:33.358 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-py. 2026-02-21T22:14:33.366 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-21T22:14:33.367 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-21T22:14:33.388 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-pygments. 2026-02-21T22:14:33.397 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-21T22:14:33.397 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T22:14:33.439 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-toml. 2026-02-21T22:14:33.448 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-21T22:14:33.448 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-21T22:14:33.464 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-pytest. 2026-02-21T22:14:33.473 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-21T22:14:33.473 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T22:14:33.497 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-simplejson. 2026-02-21T22:14:33.505 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-21T22:14:33.506 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-21T22:14:33.532 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package python3-webob. 2026-02-21T22:14:33.540 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-21T22:14:33.541 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T22:14:33.558 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-21T22:14:33.566 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-21T22:14:33.567 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-21T22:14:33.635 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package radosgw. 2026-02-21T22:14:33.643 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../85-radosgw_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:33.644 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking radosgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.815 INFO:teuthology.orchestra.run.trial102.stdout:Selecting previously unselected package rbd-fuse. 2026-02-21T22:14:33.824 INFO:teuthology.orchestra.run.trial102.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:33.824 INFO:teuthology.orchestra.run.trial102.stdout:Unpacking rbd-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:33.851 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-21T22:14:33.930 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T22:14:33.931 INFO:teuthology.orchestra.run.trial102.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T22:14:34.011 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T22:14:34.292 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-21T22:14:34.670 INFO:teuthology.orchestra.run.trial102.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-21T22:14:34.680 INFO:teuthology.orchestra.run.trial102.stdout:Setting up cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:34.737 INFO:teuthology.orchestra.run.trial102.stdout:Adding system user cephadm....done 2026-02-21T22:14:34.743 INFO:teuthology.orchestra.run.trial102.stdout:usermod: no changes 2026-02-21T22:14:34.748 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-21T22:14:34.821 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:34.822 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-21T22:14:34.893 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-21T22:14:34.971 INFO:teuthology.orchestra.run.trial102.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T22:14:34.972 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-21T22:14:35.068 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T22:14:35.192 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-21T22:14:35.268 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-21T22:14:35.343 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-ceph-argparse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:35.411 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T22:14:35.412 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-21T22:14:35.413 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T22:14:35.414 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T22:14:35.415 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-21T22:14:35.537 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-21T22:14:35.619 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libcephfs-proxy2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:35.620 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T22:14:35.700 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-21T22:14:35.792 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T22:14:36.041 INFO:teuthology.orchestra.run.trial102.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-21T22:14:36.042 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T22:14:36.136 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:36.263 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T22:14:36.343 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-21T22:14:36.403 INFO:teuthology.orchestra.run.trial102.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-21T22:14:36.404 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:36.495 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T22:14:36.984 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:36.988 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-21T22:14:37.065 INFO:teuthology.orchestra.run.trial102.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T22:14:37.066 INFO:teuthology.orchestra.run.trial102.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-21T22:14:37.067 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-21T22:14:37.144 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-21T22:14:37.217 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:37.218 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-21T22:14:37.303 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-21T22:14:37.375 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-21T22:14:37.456 INFO:teuthology.orchestra.run.trial102.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T22:14:37.457 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-21T22:14:37.540 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T22:14:37.541 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T22:14:37.617 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T22:14:37.700 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-21T22:14:37.775 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:37.776 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T22:14:37.912 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-21T22:14:37.985 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:37.986 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-21T22:14:38.071 INFO:teuthology.orchestra.run.trial102.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:38.072 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-21T22:14:38.208 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T22:14:38.209 INFO:teuthology.orchestra.run.trial102.stdout:Setting up librados2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.210 INFO:teuthology.orchestra.run.trial102.stdout:Setting up librgw2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.211 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.212 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T22:14:38.725 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libcephfs2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.726 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libradosstriper1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.727 INFO:teuthology.orchestra.run.trial102.stdout:Setting up librbd1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.728 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr-modules-core (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.729 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:38.801 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T22:14:38.802 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T22:14:39.199 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libcephfs-dev (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.200 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-rados (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.201 INFO:teuthology.orchestra.run.trial102.stdout:Setting up libcephfs-daemon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.201 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-rbd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.202 INFO:teuthology.orchestra.run.trial102.stdout:Setting up rbd-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.203 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-rgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.204 INFO:teuthology.orchestra.run.trial102.stdout:Setting up python3-cephfs (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.205 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.234 INFO:teuthology.orchestra.run.trial102.stdout:Adding group ceph....done 2026-02-21T22:14:39.269 INFO:teuthology.orchestra.run.trial102.stdout:Adding system user ceph....done 2026-02-21T22:14:39.279 INFO:teuthology.orchestra.run.trial102.stdout:Setting system user ceph properties....done 2026-02-21T22:14:39.284 INFO:teuthology.orchestra.run.trial102.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-21T22:14:39.371 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-21T22:14:39.637 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-21T22:14:39.998 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-test (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:39.999 INFO:teuthology.orchestra.run.trial102.stdout:Setting up radosgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:40.248 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:14:40.248 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:14:40.590 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-base (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:40.709 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-21T22:14:41.085 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mds (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:41.144 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T22:14:41.144 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T22:14:41.482 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:41.558 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T22:14:41.558 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T22:14:41.907 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:41.908 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-osd-classic (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:41.925 INFO:teuthology.orchestra.run.trial102.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-21T22:14:41.935 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:41.945 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.015 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T22:14:42.015 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T22:14:42.376 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr-cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.388 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-mgr-dashboard (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.399 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-osd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.487 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T22:14:42.487 INFO:teuthology.orchestra.run.trial102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T22:14:42.497 INFO:teuthology.orchestra.run.trial178.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5425-g7c8cf961-1jammy [93.7 kB] 2026-02-21T22:14:42.812 INFO:teuthology.orchestra.run.trial178.stdout:Fetched 702 MB in 10min 6s (1158 kB/s) 2026-02-21T22:14:42.832 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-21T22:14:42.847 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.848 INFO:teuthology.orchestra.run.trial102.stdout:Setting up ceph-volume (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:42.957 INFO:teuthology.orchestra.run.trial102.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-21T22:14:42.987 INFO:teuthology.orchestra.run.trial178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-21T22:14:42.989 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-21T22:14:42.991 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T22:14:43.013 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-21T22:14:43.025 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-21T22:14:43.025 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T22:14:43.040 INFO:teuthology.orchestra.run.trial102.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-21T22:14:43.042 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-21T22:14:43.054 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-21T22:14:43.054 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T22:14:43.073 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-21T22:14:43.082 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:43.087 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:43.118 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-21T22:14:43.126 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:43.127 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:43.142 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-21T22:14:43.151 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T22:14:43.151 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:43.171 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-21T22:14:43.179 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-21T22:14:43.179 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T22:14:43.195 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package librados2. 2026-02-21T22:14:43.203 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../07-librados2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.204 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking librados2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.245 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libnbd0. 2026-02-21T22:14:43.254 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-21T22:14:43.254 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-21T22:14:43.268 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package librbd1. 2026-02-21T22:14:43.277 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../09-librbd1_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.277 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking librbd1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.314 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libcephfs2. 2026-02-21T22:14:43.322 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.322 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libcephfs2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.342 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-rados. 2026-02-21T22:14:43.350 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.350 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-rados (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.367 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-21T22:14:43.375 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:43.376 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-ceph-argparse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.390 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-cephfs. 2026-02-21T22:14:43.398 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.398 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-cephfs (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.415 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-21T22:14:43.423 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:43.424 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.442 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-21T22:14:43.450 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-21T22:14:43.450 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T22:14:43.466 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-prettytable. 2026-02-21T22:14:43.474 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-21T22:14:43.474 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-21T22:14:43.490 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-rbd. 2026-02-21T22:14:43.498 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.498 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-rbd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.515 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-21T22:14:43.523 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-21T22:14:43.523 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T22:14:43.542 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package librgw2. 2026-02-21T22:14:43.550 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../19-librgw2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.551 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking librgw2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.644 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-rgw. 2026-02-21T22:14:43.652 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.653 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-rgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.667 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-21T22:14:43.675 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-21T22:14:43.676 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout:Running kernel seems to be up-to-date. 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout:The processor microcode seems to be up-to-date. 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout:No services need to be restarted. 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.688 INFO:teuthology.orchestra.run.trial102.stdout:No containers need to be restarted. 2026-02-21T22:14:43.689 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.689 INFO:teuthology.orchestra.run.trial102.stdout:No user sessions are running outdated binaries. 2026-02-21T22:14:43.689 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:14:43.689 INFO:teuthology.orchestra.run.trial102.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-21T22:14:43.690 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libradosstriper1. 2026-02-21T22:14:43.699 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.699 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libradosstriper1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.715 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-common. 2026-02-21T22:14:43.723 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:43.724 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:43.987 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-base. 2026-02-21T22:14:43.995 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:44.000 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-base (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:44.058 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-21T22:14:44.066 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-21T22:14:44.067 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-21T22:14:44.081 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-cheroot. 2026-02-21T22:14:44.089 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-21T22:14:44.090 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T22:14:44.106 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-21T22:14:44.115 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-21T22:14:44.115 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-21T22:14:44.129 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-21T22:14:44.137 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-21T22:14:44.137 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-21T22:14:44.152 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-21T22:14:44.159 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-21T22:14:44.160 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-21T22:14:44.174 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-tempora. 2026-02-21T22:14:44.182 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-21T22:14:44.183 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-21T22:14:44.198 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-portend. 2026-02-21T22:14:44.205 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-21T22:14:44.206 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-21T22:14:44.220 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-21T22:14:44.228 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-21T22:14:44.228 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-21T22:14:44.243 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-21T22:14:44.251 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-21T22:14:44.251 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-21T22:14:44.275 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-natsort. 2026-02-21T22:14:44.284 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-21T22:14:44.284 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-21T22:14:44.299 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-21T22:14:44.307 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:44.307 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:44.338 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-21T22:14:44.346 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:44.347 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:44.362 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr. 2026-02-21T22:14:44.370 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:44.371 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:44.392 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mon. 2026-02-21T22:14:44.401 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:44.401 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:44.463 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-21T22:14:44.471 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-21T22:14:44.472 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T22:14:44.479 INFO:teuthology.orchestra.run.trial102.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-21T22:14:44.486 DEBUG:teuthology.parallel:result is None 2026-02-21T22:14:44.487 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-21T22:14:44.495 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:44.496 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-osd-classic (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.679 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-osd. 2026-02-21T22:14:48.691 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:48.691 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-osd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.711 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph. 2026-02-21T22:14:48.723 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../42-ceph_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:48.724 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.738 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-fuse. 2026-02-21T22:14:48.750 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:48.750 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.776 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mds. 2026-02-21T22:14:48.787 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:48.787 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mds (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.820 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package cephadm. 2026-02-21T22:14:48.828 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../45-cephadm_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:48.829 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.844 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-21T22:14:48.852 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T22:14:48.852 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:48.874 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-21T22:14:48.883 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:48.883 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:48.904 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-21T22:14:48.912 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-21T22:14:48.913 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-21T22:14:48.927 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-routes. 2026-02-21T22:14:48.935 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-21T22:14:48.935 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T22:14:48.956 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-21T22:14:48.965 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:48.965 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:49.642 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-21T22:14:49.651 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-21T22:14:49.652 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T22:14:49.690 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-joblib. 2026-02-21T22:14:49.699 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-21T22:14:49.699 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T22:14:49.725 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-21T22:14:49.734 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-21T22:14:49.734 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-21T22:14:49.749 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-sklearn. 2026-02-21T22:14:49.758 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-21T22:14:49.758 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T22:14:49.838 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-21T22:14:49.847 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:49.847 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:50.006 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-cachetools. 2026-02-21T22:14:50.014 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-21T22:14:50.015 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-21T22:14:50.029 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-rsa. 2026-02-21T22:14:50.038 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-21T22:14:50.038 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-21T22:14:50.055 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-google-auth. 2026-02-21T22:14:50.064 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-21T22:14:50.064 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-21T22:14:50.081 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-21T22:14:50.090 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-21T22:14:50.090 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T22:14:50.107 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-websocket. 2026-02-21T22:14:50.115 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-21T22:14:50.116 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-21T22:14:50.133 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-21T22:14:50.141 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-21T22:14:50.141 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T22:14:50.239 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-21T22:14:50.248 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:50.248 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:50.262 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-21T22:14:50.271 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-21T22:14:50.271 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T22:14:50.287 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-21T22:14:50.295 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T22:14:50.296 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:50.310 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package jq. 2026-02-21T22:14:50.319 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T22:14:50.319 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:50.333 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package socat. 2026-02-21T22:14:50.342 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-21T22:14:50.343 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-21T22:14:50.362 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package xmlstarlet. 2026-02-21T22:14:50.370 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-21T22:14:50.370 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-21T22:14:50.406 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-test. 2026-02-21T22:14:50.414 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:50.415 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-test (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.040 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package ceph-volume. 2026-02-21T22:14:51.049 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5425-g7c8cf961-1jammy_all.deb ... 2026-02-21T22:14:51.050 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking ceph-volume (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.073 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-21T22:14:51.082 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:51.082 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libcephfs-daemon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.098 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-21T22:14:51.106 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:51.107 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libcephfs-proxy2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.120 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-21T22:14:51.128 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:51.129 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking libcephfs-dev (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.144 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package nvme-cli. 2026-02-21T22:14:51.152 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-21T22:14:51.153 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T22:14:51.184 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-21T22:14:51.193 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T22:14:51.194 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:51.224 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-21T22:14:51.232 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-21T22:14:51.232 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-21T22:14:51.246 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-pluggy. 2026-02-21T22:14:51.255 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-21T22:14:51.255 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-21T22:14:51.270 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-psutil. 2026-02-21T22:14:51.279 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-21T22:14:51.279 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-21T22:14:51.298 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-py. 2026-02-21T22:14:51.307 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-21T22:14:51.307 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-21T22:14:51.328 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-pygments. 2026-02-21T22:14:51.336 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-21T22:14:51.337 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T22:14:51.379 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-toml. 2026-02-21T22:14:51.387 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-21T22:14:51.388 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-21T22:14:51.402 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-pytest. 2026-02-21T22:14:51.411 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-21T22:14:51.411 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T22:14:51.432 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-simplejson. 2026-02-21T22:14:51.441 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-21T22:14:51.442 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-21T22:14:51.467 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package python3-webob. 2026-02-21T22:14:51.475 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-21T22:14:51.476 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T22:14:51.491 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-21T22:14:51.500 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-21T22:14:51.500 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-21T22:14:51.569 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package radosgw. 2026-02-21T22:14:51.578 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../85-radosgw_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:51.578 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking radosgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.738 INFO:teuthology.orchestra.run.trial178.stdout:Selecting previously unselected package rbd-fuse. 2026-02-21T22:14:51.747 INFO:teuthology.orchestra.run.trial178.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5425-g7c8cf961-1jammy_amd64.deb ... 2026-02-21T22:14:51.748 INFO:teuthology.orchestra.run.trial178.stdout:Unpacking rbd-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:51.775 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-21T22:14:51.841 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T22:14:51.841 INFO:teuthology.orchestra.run.trial178.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T22:14:51.913 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T22:14:52.193 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-21T22:14:52.554 INFO:teuthology.orchestra.run.trial178.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-21T22:14:52.564 INFO:teuthology.orchestra.run.trial178.stdout:Setting up cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:52.608 INFO:teuthology.orchestra.run.trial178.stdout:Adding system user cephadm....done 2026-02-21T22:14:52.614 INFO:teuthology.orchestra.run.trial178.stdout:usermod: no changes 2026-02-21T22:14:52.619 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-21T22:14:52.685 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:52.685 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-21T22:14:52.758 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-21T22:14:52.832 INFO:teuthology.orchestra.run.trial178.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T22:14:52.834 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-21T22:14:52.929 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T22:14:53.055 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-21T22:14:53.128 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-21T22:14:53.204 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-ceph-argparse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:53.272 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T22:14:53.273 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-21T22:14:53.274 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T22:14:53.275 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T22:14:53.276 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-21T22:14:53.400 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-21T22:14:53.479 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libcephfs-proxy2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:53.480 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T22:14:53.559 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-21T22:14:53.646 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T22:14:53.899 INFO:teuthology.orchestra.run.trial178.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-21T22:14:53.900 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T22:14:53.996 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T22:14:54.124 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T22:14:54.206 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-21T22:14:54.278 INFO:teuthology.orchestra.run.trial178.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-21T22:14:54.279 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:54.373 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T22:14:54.885 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:54.889 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-21T22:14:54.969 INFO:teuthology.orchestra.run.trial178.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T22:14:54.970 INFO:teuthology.orchestra.run.trial178.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-21T22:14:54.971 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-21T22:14:55.044 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-21T22:14:55.107 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:55.108 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-21T22:14:55.184 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-21T22:14:55.257 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-21T22:14:55.336 INFO:teuthology.orchestra.run.trial178.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T22:14:55.337 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-21T22:14:55.421 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T22:14:55.422 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T22:14:55.500 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T22:14:55.585 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-21T22:14:55.658 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:55.659 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T22:14:55.797 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-21T22:14:55.870 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T22:14:55.871 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-21T22:14:55.959 INFO:teuthology.orchestra.run.trial178.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-21T22:14:55.959 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-21T22:14:56.097 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T22:14:56.098 INFO:teuthology.orchestra.run.trial178.stdout:Setting up librados2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.098 INFO:teuthology.orchestra.run.trial178.stdout:Setting up librgw2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.099 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.100 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T22:14:56.621 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libcephfs2 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.622 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libradosstriper1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.623 INFO:teuthology.orchestra.run.trial178.stdout:Setting up librbd1 (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.624 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr-modules-core (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.625 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:56.701 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T22:14:56.701 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T22:14:57.088 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libcephfs-dev (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.089 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-rados (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.090 INFO:teuthology.orchestra.run.trial178.stdout:Setting up libcephfs-daemon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.090 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-rbd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.091 INFO:teuthology.orchestra.run.trial178.stdout:Setting up rbd-fuse (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.092 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-rgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.093 INFO:teuthology.orchestra.run.trial178.stdout:Setting up python3-cephfs (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.094 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-common (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.122 INFO:teuthology.orchestra.run.trial178.stdout:Adding group ceph....done 2026-02-21T22:14:57.157 INFO:teuthology.orchestra.run.trial178.stdout:Adding system user ceph....done 2026-02-21T22:14:57.198 INFO:teuthology.orchestra.run.trial178.stdout:Setting system user ceph properties....done 2026-02-21T22:14:57.198 INFO:teuthology.orchestra.run.trial178.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-21T22:14:57.261 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-21T22:14:57.517 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-21T22:14:57.886 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-test (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:57.887 INFO:teuthology.orchestra.run.trial178.stdout:Setting up radosgw (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:58.127 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:14:58.128 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T22:14:58.476 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-base (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:58.559 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-21T22:14:58.898 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mds (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:58.962 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T22:14:58.962 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T22:14:59.311 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:59.389 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T22:14:59.390 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T22:14:59.740 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:59.741 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-osd-classic (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:59.759 INFO:teuthology.orchestra.run.trial178.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-21T22:14:59.769 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:59.780 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mon (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:14:59.854 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T22:14:59.854 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T22:15:00.206 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr-cephadm (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:15:00.217 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-mgr-dashboard (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:15:00.228 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-osd (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:15:00.310 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T22:15:00.310 INFO:teuthology.orchestra.run.trial178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T22:15:00.684 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:15:00.685 INFO:teuthology.orchestra.run.trial178.stdout:Setting up ceph-volume (20.3.0-5425-g7c8cf961-1jammy) ... 2026-02-21T22:15:00.796 INFO:teuthology.orchestra.run.trial178.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-21T22:15:00.879 INFO:teuthology.orchestra.run.trial178.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-21T22:15:01.534 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout:Running kernel seems to be up-to-date. 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout:The processor microcode seems to be up-to-date. 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout:No services need to be restarted. 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout:No containers need to be restarted. 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.535 INFO:teuthology.orchestra.run.trial178.stdout:No user sessions are running outdated binaries. 2026-02-21T22:15:01.536 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:01.536 INFO:teuthology.orchestra.run.trial178.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-21T22:15:02.349 INFO:teuthology.orchestra.run.trial178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-21T22:15:02.355 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:02.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:15:02.522 DEBUG:teuthology.orchestra.run.trial102:> dpkg-query -W -f '${Version}' ceph 2026-02-21T22:15:02.545 INFO:teuthology.orchestra.run.trial102.stdout:20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:15:02.546 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:15:02.546 INFO:teuthology.task.install:The correct ceph version 20.3.0-5425-g7c8cf961-1jammy is installed. 2026-02-21T22:15:02.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T22:15:02.706 DEBUG:teuthology.orchestra.run.trial178:> dpkg-query -W -f '${Version}' ceph 2026-02-21T22:15:02.716 INFO:teuthology.orchestra.run.trial178.stdout:20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:15:02.717 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5425-g7c8cf961-1jammy 2026-02-21T22:15:02.717 INFO:teuthology.task.install:The correct ceph version 20.3.0-5425-g7c8cf961-1jammy is installed. 2026-02-21T22:15:02.720 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T22:15:02.720 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:02.720 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T22:15:02.732 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:02.732 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T22:15:02.769 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T22:15:02.770 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:02.770 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T22:15:02.780 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T22:15:02.828 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:02.828 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T22:15:02.837 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T22:15:02.888 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T22:15:02.889 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:02.889 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T22:15:02.898 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T22:15:02.948 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:02.948 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T22:15:02.956 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T22:15:03.004 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T22:15:03.005 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:03.005 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T22:15:03.014 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T22:15:03.064 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:03.064 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T22:15:03.073 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T22:15:03.124 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T22:15:03.236 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T22:15:03.236 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod 777 /var/log/ceph 2026-02-21T22:15:03.238 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod 777 /var/log/ceph 2026-02-21T22:15:03.246 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T22:15:03.246 DEBUG:teuthology.orchestra.run.trial102:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T22:15:03.289 DEBUG:teuthology.orchestra.run.trial178:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T22:15:03.296 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T22:15:03.296 DEBUG:teuthology.orchestra.run.trial102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T22:15:03.337 DEBUG:teuthology.orchestra.run.trial178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T22:15:03.359 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T22:15:03.360 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 20, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore debug onode segmentation random': True, 'bluestore fsck on mount': True, 'bluestore onode segment size': '512K', 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd_mclock_skip_benchmark': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', '\\(MON_NETSPLIT)', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2026-02-21T22:15:03.360 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058', 'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle', 'description': 'rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{bitmap} base mem$/{low} onode-segment$/{512K-onoff} write$/{random/{compr$/{yes$/{zstd}} random}}}} rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/snaps-few-objects}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '63058', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'trial', 'name': 'nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 20, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore debug onode segmentation random': True, 'bluestore fsck on mount': True, 'bluestore onode segment size': '512K', 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd_mclock_skip_benchmark': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', '\\(MON_NETSPLIT)', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'distro-suffix': 'rockylinux-10'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '53f8441172c46e34523029ab88f9db7a45514e84'}}, 'owner': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 1998, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/NitzanMordhai/ceph.git', 'suite_sha1': '53f8441172c46e34523029ab88f9db7a45514e84', 'targets': {'trial102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOXave8YfKZ3yfEKhO4KtEEL6Zm3dIpp31jkVw/Ay1f/vmGPVrpDc3cd3VzpniI7i753dQICxHsRHSSJ841vX8=', 'trial178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPw7rKHKMaXRfTPWFRma0ND1/z2SVZuyD6LRPYnFdpzvXH2KYesDg4i5/rTpqPlw05zWyWs5H0jQNpRHLGtFYFM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-21_21:55:34', 'tube': 'trial', 'user': 'nmordech', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T22:15:03.361 DEBUG:teuthology.orchestra.run.trial102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T22:15:03.390 DEBUG:teuthology.orchestra.run.trial178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T22:15:03.394 DEBUG:teuthology.orchestra.run.trial102:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T22:15:03.434 DEBUG:teuthology.orchestra.run.trial178:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T22:15:03.444 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:03.444 DEBUG:teuthology.orchestra.run.trial102:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T22:15:03.485 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T22:15:03.485 DEBUG:teuthology.orchestra.run.trial102:> stat /dev/vg_nvme/lv_1 2026-02-21T22:15:03.532 INFO:teuthology.orchestra.run.trial102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T22:15:03.532 INFO:teuthology.orchestra.run.trial102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout:Device: 5h/5d Inode: 903 Links: 1 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout:Access: 2026-02-21 22:04:12.763623786 +0000 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout:Modify: 2026-02-21 22:04:12.759623854 +0000 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout:Change: 2026-02-21 22:04:12.759623854 +0000 2026-02-21T22:15:03.533 INFO:teuthology.orchestra.run.trial102.stdout: Birth: - 2026-02-21T22:15:03.533 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T22:15:03.583 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records in 2026-02-21T22:15:03.584 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records out 2026-02-21T22:15:03.584 INFO:teuthology.orchestra.run.trial102.stderr:512 bytes copied, 0.000175586 s, 2.9 MB/s 2026-02-21T22:15:03.585 DEBUG:teuthology.orchestra.run.trial102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T22:15:03.634 DEBUG:teuthology.orchestra.run.trial102:> stat /dev/vg_nvme/lv_2 2026-02-21T22:15:03.680 INFO:teuthology.orchestra.run.trial102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T22:15:03.680 INFO:teuthology.orchestra.run.trial102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:03.680 INFO:teuthology.orchestra.run.trial102.stdout:Device: 5h/5d Inode: 914 Links: 1 2026-02-21T22:15:03.681 INFO:teuthology.orchestra.run.trial102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:03.681 INFO:teuthology.orchestra.run.trial102.stdout:Access: 2026-02-21 22:04:13.235615747 +0000 2026-02-21T22:15:03.681 INFO:teuthology.orchestra.run.trial102.stdout:Modify: 2026-02-21 22:04:13.075618472 +0000 2026-02-21T22:15:03.681 INFO:teuthology.orchestra.run.trial102.stdout:Change: 2026-02-21 22:04:13.075618472 +0000 2026-02-21T22:15:03.681 INFO:teuthology.orchestra.run.trial102.stdout: Birth: - 2026-02-21T22:15:03.681 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T22:15:03.728 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records in 2026-02-21T22:15:03.728 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records out 2026-02-21T22:15:03.728 INFO:teuthology.orchestra.run.trial102.stderr:512 bytes copied, 0.000163834 s, 3.1 MB/s 2026-02-21T22:15:03.729 DEBUG:teuthology.orchestra.run.trial102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T22:15:03.778 DEBUG:teuthology.orchestra.run.trial102:> stat /dev/vg_nvme/lv_3 2026-02-21T22:15:03.824 INFO:teuthology.orchestra.run.trial102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T22:15:03.824 INFO:teuthology.orchestra.run.trial102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout:Device: 5h/5d Inode: 938 Links: 1 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout:Access: 2026-02-21 22:04:13.547610433 +0000 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout:Modify: 2026-02-21 22:04:13.391613089 +0000 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout:Change: 2026-02-21 22:04:13.391613089 +0000 2026-02-21T22:15:03.825 INFO:teuthology.orchestra.run.trial102.stdout: Birth: - 2026-02-21T22:15:03.825 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T22:15:03.871 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records in 2026-02-21T22:15:03.871 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records out 2026-02-21T22:15:03.872 INFO:teuthology.orchestra.run.trial102.stderr:512 bytes copied, 0.000182599 s, 2.8 MB/s 2026-02-21T22:15:03.873 DEBUG:teuthology.orchestra.run.trial102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T22:15:03.922 DEBUG:teuthology.orchestra.run.trial102:> stat /dev/vg_nvme/lv_4 2026-02-21T22:15:03.968 INFO:teuthology.orchestra.run.trial102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T22:15:03.968 INFO:teuthology.orchestra.run.trial102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:03.968 INFO:teuthology.orchestra.run.trial102.stdout:Device: 5h/5d Inode: 962 Links: 1 2026-02-21T22:15:03.968 INFO:teuthology.orchestra.run.trial102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:03.969 INFO:teuthology.orchestra.run.trial102.stdout:Access: 2026-02-21 22:04:13.863605051 +0000 2026-02-21T22:15:03.969 INFO:teuthology.orchestra.run.trial102.stdout:Modify: 2026-02-21 22:04:13.711607640 +0000 2026-02-21T22:15:03.969 INFO:teuthology.orchestra.run.trial102.stdout:Change: 2026-02-21 22:04:13.711607640 +0000 2026-02-21T22:15:03.969 INFO:teuthology.orchestra.run.trial102.stdout: Birth: - 2026-02-21T22:15:03.969 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T22:15:04.015 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records in 2026-02-21T22:15:04.015 INFO:teuthology.orchestra.run.trial102.stderr:1+0 records out 2026-02-21T22:15:04.015 INFO:teuthology.orchestra.run.trial102.stderr:512 bytes copied, 0.000206613 s, 2.5 MB/s 2026-02-21T22:15:04.016 DEBUG:teuthology.orchestra.run.trial102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T22:15:04.066 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:04.066 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:04.066 DEBUG:teuthology.orchestra.run.trial178:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T22:15:04.071 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T22:15:04.072 DEBUG:teuthology.orchestra.run.trial178:> stat /dev/vg_nvme/lv_1 2026-02-21T22:15:04.116 INFO:teuthology.orchestra.run.trial178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T22:15:04.116 INFO:teuthology.orchestra.run.trial178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout:Device: 5h/5d Inode: 898 Links: 1 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout:Access: 2026-02-21 22:04:12.639399129 +0000 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout:Modify: 2026-02-21 22:04:12.487401713 +0000 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout:Change: 2026-02-21 22:04:12.487401713 +0000 2026-02-21T22:15:04.117 INFO:teuthology.orchestra.run.trial178.stdout: Birth: - 2026-02-21T22:15:04.117 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T22:15:04.164 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records in 2026-02-21T22:15:04.164 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records out 2026-02-21T22:15:04.164 INFO:teuthology.orchestra.run.trial178.stderr:512 bytes copied, 0.000178321 s, 2.9 MB/s 2026-02-21T22:15:04.165 DEBUG:teuthology.orchestra.run.trial178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T22:15:04.214 DEBUG:teuthology.orchestra.run.trial178:> stat /dev/vg_nvme/lv_2 2026-02-21T22:15:04.260 INFO:teuthology.orchestra.run.trial178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T22:15:04.260 INFO:teuthology.orchestra.run.trial178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout:Device: 5h/5d Inode: 922 Links: 1 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout:Access: 2026-02-21 22:04:12.971393486 +0000 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout:Modify: 2026-02-21 22:04:12.815396137 +0000 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout:Change: 2026-02-21 22:04:12.815396137 +0000 2026-02-21T22:15:04.261 INFO:teuthology.orchestra.run.trial178.stdout: Birth: - 2026-02-21T22:15:04.261 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T22:15:04.307 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records in 2026-02-21T22:15:04.307 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records out 2026-02-21T22:15:04.308 INFO:teuthology.orchestra.run.trial178.stderr:512 bytes copied, 0.000167592 s, 3.1 MB/s 2026-02-21T22:15:04.309 DEBUG:teuthology.orchestra.run.trial178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T22:15:04.358 DEBUG:teuthology.orchestra.run.trial178:> stat /dev/vg_nvme/lv_3 2026-02-21T22:15:04.404 INFO:teuthology.orchestra.run.trial178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T22:15:04.404 INFO:teuthology.orchestra.run.trial178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:04.404 INFO:teuthology.orchestra.run.trial178.stdout:Device: 5h/5d Inode: 944 Links: 1 2026-02-21T22:15:04.404 INFO:teuthology.orchestra.run.trial178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:04.405 INFO:teuthology.orchestra.run.trial178.stdout:Access: 2026-02-21 22:04:13.135390699 +0000 2026-02-21T22:15:04.405 INFO:teuthology.orchestra.run.trial178.stdout:Modify: 2026-02-21 22:04:13.131390765 +0000 2026-02-21T22:15:04.405 INFO:teuthology.orchestra.run.trial178.stdout:Change: 2026-02-21 22:04:13.131390765 +0000 2026-02-21T22:15:04.405 INFO:teuthology.orchestra.run.trial178.stdout: Birth: - 2026-02-21T22:15:04.405 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T22:15:04.451 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records in 2026-02-21T22:15:04.452 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records out 2026-02-21T22:15:04.452 INFO:teuthology.orchestra.run.trial178.stderr:512 bytes copied, 0.000185295 s, 2.8 MB/s 2026-02-21T22:15:04.453 DEBUG:teuthology.orchestra.run.trial178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T22:15:04.502 DEBUG:teuthology.orchestra.run.trial178:> stat /dev/vg_nvme/lv_4 2026-02-21T22:15:04.548 INFO:teuthology.orchestra.run.trial178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T22:15:04.548 INFO:teuthology.orchestra.run.trial178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T22:15:04.548 INFO:teuthology.orchestra.run.trial178.stdout:Device: 5h/5d Inode: 970 Links: 1 2026-02-21T22:15:04.549 INFO:teuthology.orchestra.run.trial178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T22:15:04.549 INFO:teuthology.orchestra.run.trial178.stdout:Access: 2026-02-21 22:04:13.579383151 +0000 2026-02-21T22:15:04.549 INFO:teuthology.orchestra.run.trial178.stdout:Modify: 2026-02-21 22:04:13.427385735 +0000 2026-02-21T22:15:04.549 INFO:teuthology.orchestra.run.trial178.stdout:Change: 2026-02-21 22:04:13.427385735 +0000 2026-02-21T22:15:04.549 INFO:teuthology.orchestra.run.trial178.stdout: Birth: - 2026-02-21T22:15:04.549 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T22:15:04.595 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records in 2026-02-21T22:15:04.595 INFO:teuthology.orchestra.run.trial178.stderr:1+0 records out 2026-02-21T22:15:04.596 INFO:teuthology.orchestra.run.trial178.stderr:512 bytes copied, 0.00016159 s, 3.2 MB/s 2026-02-21T22:15:04.597 DEBUG:teuthology.orchestra.run.trial178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T22:15:04.646 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:04.646 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@trial178.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2026-02-21T22:15:04.646 INFO:tasks.ceph:Generating config... 2026-02-21T22:15:04.649 INFO:tasks.ceph:[client] debug monc = 20 2026-02-21T22:15:04.649 INFO:tasks.ceph:[client] debug ms = 1 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] ms bind msgr2 = False 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] ms inject delay max = 1 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2026-02-21T22:15:04.649 INFO:tasks.ceph:[global] ms inject delay type = mon 2026-02-21T22:15:04.650 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2026-02-21T22:15:04.650 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2026-02-21T22:15:04.650 INFO:tasks.ceph:[global] ms type = async 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mgr] debug monc = 10 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] debug mon = 30 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] debug ms = 20 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-21T22:15:04.650 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] paxos min = 10 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] paxos service trim min = 5 2026-02-21T22:15:04.651 INFO:tasks.ceph:[mon] paxos trim min = 10 2026-02-21T22:15:04.651 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-21T22:15:04.651 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-21T22:15:04.651 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2026-02-21T22:15:04.651 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-21T22:15:04.651 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore debug onode segmentation random = True 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore onode segment size = 512K 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] debug bluefs = 20 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] debug bluestore = 20 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-21T22:15:04.652 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd op num shards = 1 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd op num threads per shard = 2 2026-02-21T22:15:04.653 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-21T22:15:04.654 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-21T22:15:04.654 INFO:tasks.ceph:[osd] osd_mclock_skip_benchmark = True 2026-02-21T22:15:04.654 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T22:15:04.654 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T22:15:04.672 INFO:teuthology.orchestra.run.trial102.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T22:15:04.675 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T22:15:04.736 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T22:15:04.784 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:10.20.193.102:6789]'), ('mon.c', '[v1:10.20.193.102:6790]'), ('mon.b', '[v1:10.20.193.178:6789]')] 2026-02-21T22:15:04.785 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:10.20.193.102:6789],[v1:10.20.193.102:6790],[v1:10.20.193.178:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore debug onode segmentation random': True, 'bluestore fsck on mount': True, 'bluestore onode segment size': '512K', 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd_mclock_skip_benchmark': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 20, 'debug mon': 30, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2026-02-21T22:15:04.786 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:04.787 DEBUG:teuthology.orchestra.run.trial102:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T22:15:04.829 DEBUG:teuthology.orchestra.run.trial102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:10.20.193.102:6789]' --addv c '[v1:10.20.193.102:6790]' --addv b '[v1:10.20.193.178:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:04.891 INFO:teuthology.orchestra.run.trial102.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:04.891 INFO:teuthology.orchestra.run.trial102.stdout:monmaptool: generated fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:04.891 INFO:teuthology.orchestra.run.trial102.stdout:setting min_mon_release = tentacle 2026-02-21T22:15:04.891 INFO:teuthology.orchestra.run.trial102.stdout:epoch 0 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:min_mon_release 20 (tentacle) 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:election_strategy: 3 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:04.892 INFO:teuthology.orchestra.run.trial102.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-21T22:15:04.894 DEBUG:teuthology.orchestra.run.trial102:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T22:15:04.940 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 740d24fb-83fc-48ed-963d-a0ec1185b484... 2026-02-21T22:15:04.941 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T22:15:04.982 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout:[global] 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: chdir = "" 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: auth supported = cephx 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: filestore xattr use omap = true 2026-02-21T22:15:05.000 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: mon clock drift allowed = 1.000 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: osd crush chooseleaf type = 0 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: auth debug = true 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: ms die on old message = true 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: ms die on bug = true 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: mon pg warn max object skew = 0 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.001 INFO:teuthology.orchestra.run.trial102.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: osd pool default size = 2 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon osd allow primary affinity = true 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon osd allow pg remap = true 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon warn on legacy crush tunables = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon warn on crush straw calc version zero = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon warn on no sortbitwise = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon warn on osd down out interval zero = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon warn on too few osds = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T22:15:05.002 INFO:teuthology.orchestra.run.trial102.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: mon_allow_pool_size_one = true 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: osd default data pool replay window = 5 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: mon allow pool delete = true 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: mon cluster log file level = debug 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: debug asserts on shutdown = true 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: mon health detail to clog = false 2026-02-21T22:15:05.003 INFO:teuthology.orchestra.run.trial102.stdout: mon host = "[v1:10.20.193.102:6789],[v1:10.20.193.102:6790],[v1:10.20.193.178:6789]" 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: mon client directed command retry = 5 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: mon election default strategy = 3 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms bind msgr2 = False 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms inject delay max = 1 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms inject delay probability = 0.005 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms inject delay type = mon 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms inject internal delays = 0.002 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms inject socket failures = 2500 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: ms type = async 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: fsid = 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.004 INFO:teuthology.orchestra.run.trial102.stdout:[osd] 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd journal size = 100 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd scrub load threshold = 5.0 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd scrub max interval = 600 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd mclock profile = high_recovery_ops 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd mclock skip benchmark = True 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd recover clone overlap = true 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd recovery max chunk = 1048576 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd debug shutdown = true 2026-02-21T22:15:05.005 INFO:teuthology.orchestra.run.trial102.stdout: osd debug op order = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd debug verify stray on activate = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd debug trim objects = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd open classes on start = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd debug pg log writeout = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd deep scrub update digest min age = 30 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: osd map max advance = 10 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: journal zero on create = true 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.006 INFO:teuthology.orchestra.run.trial102.stdout: filestore ondisk finisher threads = 3 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: filestore apply finisher threads = 3 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bdev debug aio = true 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: osd debug misdirected ops = true 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bdev async discard = True 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bdev enable discard = True 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore allocator = bitmap 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore block size = 96636764160 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore compression algorithm = zstd 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore compression mode = aggressive 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore debug onode segmentation random = True 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore fsck on mount = True 2026-02-21T22:15:05.007 INFO:teuthology.orchestra.run.trial102.stdout: bluestore onode segment size = 512K 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: bluestore write v2 random = True 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: bluestore zero block detection = True 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: debug bluefs = 20 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: debug bluestore = 20 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: debug ms = 1 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: debug osd = 20 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: debug rocksdb = 10 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: mon osd backfillfull_ratio = 0.85 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: mon osd full ratio = 0.9 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: mon osd nearfull ratio = 0.8 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: osd debug verify cached snaps = True 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: osd debug verify missing on start = True 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: osd failsafe full ratio = 0.95 2026-02-21T22:15:05.008 INFO:teuthology.orchestra.run.trial102.stdout: osd mclock override recovery settings = True 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd memory target = 1610612736 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd objectstore = bluestore 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd op num shards = 1 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd op num threads per shard = 2 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd op queue = debug_random 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd op queue cut off = debug_random 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: osd_mclock_skip_benchmark = True 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout:[mgr] 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: debug ms = 1 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: debug mgr = 20 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: debug mon = 20 2026-02-21T22:15:05.009 INFO:teuthology.orchestra.run.trial102.stdout: debug auth = 20 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mgr/telemetry/nag = false 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: debug monc = 10 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout:[mon] 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: debug ms = 20 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: debug mon = 30 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: debug paxos = 20 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: debug auth = 20 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mon data avail warn = 5 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mon mgr mkfs grace = 240 2026-02-21T22:15:05.010 INFO:teuthology.orchestra.run.trial102.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: mon osd reporter subtree level = osd 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: mon osd prime pg temp = true 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T22:15:05.011 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: # 1m isn't quite enough 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon_down_mkfs_grace = 2m 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon_warn_on_filestore_osds = false 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon min osdmap epochs = 25 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon osdmap full prune interval = 2 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon osdmap full prune min = 15 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon osdmap full prune txsize = 2 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon scrub inject crc mismatch = 0.01 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon scrub inject missing keys = 0.05 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: mon scrub interval = 300 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: paxos min = 10 2026-02-21T22:15:05.012 INFO:teuthology.orchestra.run.trial102.stdout: paxos service trim min = 5 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: paxos trim min = 10 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout:[client] 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: rgw cache enabled = true 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: rgw enable ops log = true 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: rgw enable usage log = true 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: debug monc = 20 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout: debug ms = 1 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout:[mon.a] 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout:[mon.c] 2026-02-21T22:15:05.013 INFO:teuthology.orchestra.run.trial102.stdout:[mon.b] 2026-02-21T22:15:05.015 INFO:teuthology.orchestra.run.trial178.stdout:[global] 2026-02-21T22:15:05.015 INFO:teuthology.orchestra.run.trial178.stdout: chdir = "" 2026-02-21T22:15:05.015 INFO:teuthology.orchestra.run.trial178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: auth supported = cephx 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: filestore xattr use omap = true 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: mon clock drift allowed = 1.000 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: osd crush chooseleaf type = 0 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: auth debug = true 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: ms die on old message = true 2026-02-21T22:15:05.016 INFO:teuthology.orchestra.run.trial178.stdout: ms die on bug = true 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: mon pg warn max object skew = 0 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: osd pool default size = 2 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: mon osd allow primary affinity = true 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: mon osd allow pg remap = true 2026-02-21T22:15:05.017 INFO:teuthology.orchestra.run.trial178.stdout: mon warn on legacy crush tunables = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon warn on crush straw calc version zero = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon warn on no sortbitwise = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon warn on osd down out interval zero = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon warn on too few osds = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: mon_allow_pool_size_one = true 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: osd default data pool replay window = 5 2026-02-21T22:15:05.018 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon allow pool delete = true 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon cluster log file level = debug 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: debug asserts on shutdown = true 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon health detail to clog = false 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon host = "[v1:10.20.193.102:6789],[v1:10.20.193.102:6790],[v1:10.20.193.178:6789]" 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon client directed command retry = 5 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: mon election default strategy = 3 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: ms bind msgr2 = False 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: ms inject delay max = 1 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: ms inject delay probability = 0.005 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: ms inject delay type = mon 2026-02-21T22:15:05.019 INFO:teuthology.orchestra.run.trial178.stdout: ms inject internal delays = 0.002 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: ms inject socket failures = 2500 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: ms type = async 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: fsid = 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout:[osd] 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd journal size = 100 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd scrub load threshold = 5.0 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd scrub max interval = 600 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd mclock profile = high_recovery_ops 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd mclock skip benchmark = True 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.020 INFO:teuthology.orchestra.run.trial178.stdout: osd recover clone overlap = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd recovery max chunk = 1048576 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd debug shutdown = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd debug op order = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd debug verify stray on activate = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd debug trim objects = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd open classes on start = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd debug pg log writeout = true 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd deep scrub update digest min age = 30 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.021 INFO:teuthology.orchestra.run.trial178.stdout: osd map max advance = 10 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: journal zero on create = true 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: filestore ondisk finisher threads = 3 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: filestore apply finisher threads = 3 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: bdev debug aio = true 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: osd debug misdirected ops = true 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: bdev async discard = True 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: bdev enable discard = True 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: bluestore allocator = bitmap 2026-02-21T22:15:05.022 INFO:teuthology.orchestra.run.trial178.stdout: bluestore block size = 96636764160 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore compression algorithm = zstd 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore compression mode = aggressive 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore debug onode segmentation random = True 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore fsck on mount = True 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore onode segment size = 512K 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore write v2 random = True 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: bluestore zero block detection = True 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: debug bluefs = 20 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: debug bluestore = 20 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: debug ms = 1 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: debug osd = 20 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: debug rocksdb = 10 2026-02-21T22:15:05.023 INFO:teuthology.orchestra.run.trial178.stdout: mon osd backfillfull_ratio = 0.85 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: mon osd full ratio = 0.9 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: mon osd nearfull ratio = 0.8 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd debug verify cached snaps = True 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd debug verify missing on start = True 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd failsafe full ratio = 0.95 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd mclock override recovery settings = True 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd memory target = 1610612736 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd objectstore = bluestore 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd op num shards = 1 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd op num threads per shard = 2 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd op queue = debug_random 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd op queue cut off = debug_random 2026-02-21T22:15:05.024 INFO:teuthology.orchestra.run.trial178.stdout: osd_mclock_skip_benchmark = True 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout:[mgr] 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug ms = 1 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug mgr = 20 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug mon = 20 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug auth = 20 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: mgr/telemetry/nag = false 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug monc = 10 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout:[mon] 2026-02-21T22:15:05.025 INFO:teuthology.orchestra.run.trial178.stdout: debug ms = 20 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: debug mon = 30 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: debug paxos = 20 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: debug auth = 20 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon data avail warn = 5 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon mgr mkfs grace = 240 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon reweight min pgs per osd = 4 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon osd reporter subtree level = osd 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon osd prime pg temp = true 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: mon reweight min bytes per osd = 10 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T22:15:05.026 INFO:teuthology.orchestra.run.trial178.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: # 1m isn't quite enough 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon_down_mkfs_grace = 2m 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon_warn_on_filestore_osds = false 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon min osdmap epochs = 25 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon osdmap full prune interval = 2 2026-02-21T22:15:05.027 INFO:teuthology.orchestra.run.trial178.stdout: mon osdmap full prune min = 15 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: mon osdmap full prune txsize = 2 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: mon scrub inject crc mismatch = 0.01 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: mon scrub inject missing keys = 0.05 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: mon scrub interval = 300 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: paxos min = 10 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: paxos service trim min = 5 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: paxos trim min = 10 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout:[client] 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: rgw cache enabled = true 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: rgw enable ops log = true 2026-02-21T22:15:05.028 INFO:teuthology.orchestra.run.trial178.stdout: rgw enable usage log = true 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout: debug monc = 20 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout: debug ms = 1 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout:[mon.a] 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout:[mon.c] 2026-02-21T22:15:05.029 INFO:teuthology.orchestra.run.trial178.stdout:[mon.b] 2026-02-21T22:15:05.029 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T22:15:05.030 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T22:15:05.078 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T22:15:05.078 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:05.079 DEBUG:teuthology.orchestra.run.trial102:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T22:15:05.125 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:05.125 DEBUG:teuthology.orchestra.run.trial102:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T22:15:05.173 INFO:tasks.ceph:Sending monmap to node ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:15:05.174 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:05.174 DEBUG:teuthology.orchestra.run.trial102:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T22:15:05.174 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T22:15:05.230 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:05.231 DEBUG:teuthology.orchestra.run.trial102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:05.277 INFO:tasks.ceph:Sending monmap to node ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:15:05.277 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:05.277 DEBUG:teuthology.orchestra.run.trial178:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T22:15:05.277 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T22:15:05.293 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:05.293 DEBUG:teuthology.orchestra.run.trial178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:05.341 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T22:15:05.341 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T22:15:05.341 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T22:15:05.407 INFO:teuthology.orchestra.run.trial178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T22:15:05.410 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T22:15:05.411 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T22:15:05.411 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T22:15:05.469 INFO:teuthology.orchestra.run.trial178.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T22:15:05.477 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T22:15:05.477 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@trial178.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2026-02-21T22:15:05.477 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T22:15:05.487 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:05.487 INFO:tasks.ceph:role: osd.0 2026-02-21T22:15:05.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:15:05.487 DEBUG:teuthology.orchestra.run.trial102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T22:15:05.537 INFO:teuthology.orchestra.run.trial102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:05.538 INFO:teuthology.orchestra.run.trial102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:06.110 INFO:teuthology.orchestra.run.trial102.stdout:Discarding blocks...Done. 2026-02-21T22:15:06.112 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial102.front.sepia.ceph.com -o noatime 2026-02-21T22:15:06.112 DEBUG:teuthology.orchestra.run.trial102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T22:15:06.170 DEBUG:teuthology.orchestra.run.trial102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T22:15:06.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:06.219 INFO:teuthology.orchestra.run.trial102.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:06.220 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T22:15:06.269 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:06.269 INFO:tasks.ceph:role: osd.1 2026-02-21T22:15:06.269 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:15:06.269 DEBUG:teuthology.orchestra.run.trial102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:06.323 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:06.324 INFO:teuthology.orchestra.run.trial102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:06.889 INFO:teuthology.orchestra.run.trial102.stdout:Discarding blocks...Done. 2026-02-21T22:15:06.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial102.front.sepia.ceph.com -o noatime 2026-02-21T22:15:06.891 DEBUG:teuthology.orchestra.run.trial102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T22:15:06.949 DEBUG:teuthology.orchestra.run.trial102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T22:15:06.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:06.999 INFO:teuthology.orchestra.run.trial102.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:06.999 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T22:15:07.049 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:07.049 INFO:tasks.ceph:role: osd.2 2026-02-21T22:15:07.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:15:07.049 DEBUG:teuthology.orchestra.run.trial102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T22:15:07.102 INFO:teuthology.orchestra.run.trial102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:07.102 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:07.102 INFO:teuthology.orchestra.run.trial102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:07.102 INFO:teuthology.orchestra.run.trial102.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:07.103 INFO:teuthology.orchestra.run.trial102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:07.662 INFO:teuthology.orchestra.run.trial102.stdout:Discarding blocks...Done. 2026-02-21T22:15:07.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial102.front.sepia.ceph.com -o noatime 2026-02-21T22:15:07.663 DEBUG:teuthology.orchestra.run.trial102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T22:15:07.717 DEBUG:teuthology.orchestra.run.trial102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T22:15:07.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:07.767 INFO:teuthology.orchestra.run.trial102.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:07.767 DEBUG:teuthology.orchestra.run.trial102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:07.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:07.826+0000 7f86f274fa40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-21T22:15:07.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:07.826+0000 7f86f274fa40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-21T22:15:07.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:07.826+0000 7f86f274fa40 -1 bdev(0x563294225800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:07.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:07.826+0000 7f86f274fa40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-21T22:15:08.706 DEBUG:teuthology.orchestra.run.trial102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T22:15:08.757 DEBUG:teuthology.orchestra.run.trial102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:08.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:08.818+0000 7f584f48fa40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-21T22:15:08.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:08.818+0000 7f584f48fa40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-21T22:15:08.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:08.818+0000 7f584f48fa40 -1 bdev(0x55b7a98a7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:08.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:08.818+0000 7f584f48fa40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-21T22:15:09.682 DEBUG:teuthology.orchestra.run.trial102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T22:15:09.733 DEBUG:teuthology.orchestra.run.trial102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:09.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:09.794+0000 7f49b1512a40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-21T22:15:09.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:09.794+0000 7f49b1512a40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-21T22:15:09.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:09.794+0000 7f49b1512a40 -1 bdev(0x55e47fd0f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:09.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:09.794+0000 7f49b1512a40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-21T22:15:10.604 DEBUG:teuthology.orchestra.run.trial102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T22:15:10.658 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-21T22:15:10.668 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:10.668 INFO:tasks.ceph:role: osd.3 2026-02-21T22:15:10.668 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:15:10.668 DEBUG:teuthology.orchestra.run.trial178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:10.722 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:10.723 INFO:teuthology.orchestra.run.trial178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:11.294 INFO:teuthology.orchestra.run.trial178.stdout:Discarding blocks...Done. 2026-02-21T22:15:11.296 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial178.front.sepia.ceph.com -o noatime 2026-02-21T22:15:11.296 DEBUG:teuthology.orchestra.run.trial178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-21T22:15:11.353 DEBUG:teuthology.orchestra.run.trial178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-21T22:15:11.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:11.403 INFO:teuthology.orchestra.run.trial178.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:11.403 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-21T22:15:11.453 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:11.453 INFO:tasks.ceph:role: osd.4 2026-02-21T22:15:11.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:15:11.453 DEBUG:teuthology.orchestra.run.trial178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:11.507 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:11.508 INFO:teuthology.orchestra.run.trial178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:12.082 INFO:teuthology.orchestra.run.trial178.stdout:Discarding blocks...Done. 2026-02-21T22:15:12.084 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial178.front.sepia.ceph.com -o noatime 2026-02-21T22:15:12.084 DEBUG:teuthology.orchestra.run.trial178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-21T22:15:12.138 DEBUG:teuthology.orchestra.run.trial178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-21T22:15:12.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:12.187 INFO:teuthology.orchestra.run.trial178.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:12.188 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-21T22:15:12.236 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2026-02-21T22:15:12.236 INFO:tasks.ceph:role: osd.5 2026-02-21T22:15:12.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:15:12.236 DEBUG:teuthology.orchestra.run.trial178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T22:15:12.286 INFO:teuthology.orchestra.run.trial178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T22:15:12.286 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T22:15:12.286 INFO:teuthology.orchestra.run.trial178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T22:15:12.286 INFO:teuthology.orchestra.run.trial178.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout: = sunit=0 swidth=0 blks 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T22:15:12.287 INFO:teuthology.orchestra.run.trial178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T22:15:12.851 INFO:teuthology.orchestra.run.trial178.stdout:Discarding blocks...Done. 2026-02-21T22:15:12.852 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial178.front.sepia.ceph.com -o noatime 2026-02-21T22:15:12.852 DEBUG:teuthology.orchestra.run.trial178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2026-02-21T22:15:12.905 DEBUG:teuthology.orchestra.run.trial178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-21T22:15:12.954 INFO:teuthology.orchestra.run.trial178.stderr:sudo: /sbin/restorecon: command not found 2026-02-21T22:15:12.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:15:12.955 DEBUG:teuthology.orchestra.run.trial178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:13.017 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:13.017+0000 7f7b84b0fa40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-21T22:15:13.018 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:13.017+0000 7f7b84b0fa40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-21T22:15:13.018 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:13.017+0000 7f7b84b0fa40 -1 bdev(0x555754877800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:13.018 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:13.017+0000 7f7b84b0fa40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-21T22:15:13.910 DEBUG:teuthology.orchestra.run.trial178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-21T22:15:13.961 DEBUG:teuthology.orchestra.run.trial178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:14.025 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:14.025+0000 7fec6392da40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-21T22:15:14.025 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:14.025+0000 7fec6392da40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-21T22:15:14.025 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:14.025+0000 7fec6392da40 -1 bdev(0x55ab7ee1f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:14.025 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:14.025+0000 7fec6392da40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-21T22:15:14.918 DEBUG:teuthology.orchestra.run.trial178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-21T22:15:14.970 DEBUG:teuthology.orchestra.run.trial178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:15.033 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:15.033+0000 7ff22a6aca40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-21T22:15:15.033 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:15.033+0000 7ff22a6aca40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-21T22:15:15.034 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:15.033+0000 7ff22a6aca40 -1 bdev(0x5643bbb35800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-21T22:15:15.034 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:15.033+0000 7ff22a6aca40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-21T22:15:15.882 DEBUG:teuthology.orchestra.run.trial178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-21T22:15:15.933 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T22:15:15.934 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:15.934 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T22:15:15.945 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:15.945 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T22:15:15.997 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:15.997 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T22:15:16.049 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:16.049 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T22:15:16.060 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:16.060 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-21T22:15:16.108 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:16.108 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-21T22:15:16.156 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:16.156 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-21T22:15:16.204 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:16.204 DEBUG:teuthology.orchestra.run.trial178:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T22:15:16.248 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T22:15:16.249 DEBUG:teuthology.orchestra.run.trial102:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T22:15:16.251 DEBUG:teuthology.orchestra.run.trial178:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T22:15:16.292 INFO:teuthology.orchestra.run.trial102.stdout:[osd.0] 2026-02-21T22:15:16.292 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBrLppp1t1/MRAAb+mn5SnqDVBzSchQuhMwEA== 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout:[osd.1] 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBsLpppC0PzMBAAR8+qd6PeMK9e7moDuXLYRg== 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout:[osd.2] 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBtLpppRsWeLxAAKkiVSlxHr7k38jzbtlszpw== 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout:[mgr.x] 2026-02-21T22:15:16.293 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBpLppp4RJqGBAAPSnBGYUtqkovdNTxkIzoZg== 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout:[osd.3] 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBxLpppC5ArARAA5ikLs0QzmbHtDiyptm1y4g== 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout:[osd.4] 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout: key = AQByLpppHBajARAAds+EuDIaHlbK08RfMWx4aw== 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout:[osd.5] 2026-02-21T22:15:16.294 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBzLpppXbQXAhAA5wWWzcDxeKEy/Ljfd1n1hg== 2026-02-21T22:15:16.295 INFO:teuthology.orchestra.run.trial102.stdout:[client.0] 2026-02-21T22:15:16.295 INFO:teuthology.orchestra.run.trial102.stdout: key = AQBpLpppLl8fHBAAm2mFMnID54FfEGrZAZMlVA== 2026-02-21T22:15:16.298 INFO:teuthology.orchestra.run.trial178.stdout:[osd.0] 2026-02-21T22:15:16.298 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBrLppp1t1/MRAAb+mn5SnqDVBzSchQuhMwEA== 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout:[osd.1] 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBsLpppC0PzMBAAR8+qd6PeMK9e7moDuXLYRg== 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout:[osd.2] 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBtLpppRsWeLxAAKkiVSlxHr7k38jzbtlszpw== 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout:[mgr.x] 2026-02-21T22:15:16.299 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBpLppp4RJqGBAAPSnBGYUtqkovdNTxkIzoZg== 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout:[osd.3] 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBxLpppC5ArARAA5ikLs0QzmbHtDiyptm1y4g== 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout:[osd.4] 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout: key = AQByLpppHBajARAAds+EuDIaHlbK08RfMWx4aw== 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout:[osd.5] 2026-02-21T22:15:16.300 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBzLpppXbQXAhAA5wWWzcDxeKEy/Ljfd1n1hg== 2026-02-21T22:15:16.301 INFO:teuthology.orchestra.run.trial178.stdout:[client.0] 2026-02-21T22:15:16.301 INFO:teuthology.orchestra.run.trial178.stdout: key = AQBpLpppLl8fHBAAm2mFMnID54FfEGrZAZMlVA== 2026-02-21T22:15:16.301 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.342 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.364 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.406 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.428 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.469 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.491 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T22:15:16.533 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T22:15:16.554 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.597 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.619 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.661 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.683 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.725 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T22:15:16.747 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T22:15:16.790 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T22:15:16.813 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T22:15:16.814 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T22:15:16.861 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:15:16.931 DEBUG:teuthology.orchestra.run.trial102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T22:15:16.980 DEBUG:teuthology.orchestra.run.trial102:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-21T22:15:17.028 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:15:17.094 DEBUG:teuthology.orchestra.run.trial102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-21T22:15:17.144 DEBUG:teuthology.orchestra.run.trial178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-21T22:15:17.154 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T22:15:17.222 DEBUG:teuthology.orchestra.run.trial178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-21T22:15:17.272 DEBUG:teuthology.orchestra.run.trial102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:17.275 DEBUG:teuthology.orchestra.run.trial178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T22:15:17.320 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T22:15:17.320 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T22:15:17.321 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T22:15:17.323 INFO:tasks.ceph.mon.a:Started 2026-02-21T22:15:17.323 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-21T22:15:17.323 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-21T22:15:17.325 INFO:tasks.ceph.mon.c:Started 2026-02-21T22:15:17.325 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:15:17.326 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:15:17.362 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:15:17.362 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T22:15:17.362 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T22:15:17.362 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T22:15:17.364 INFO:tasks.ceph.mgr.x:Started 2026-02-21T22:15:17.364 DEBUG:tasks.ceph:set 0 configs 2026-02-21T22:15:17.365 DEBUG:teuthology.orchestra.run.trial102:> sudo ceph --cluster ceph config dump 2026-02-21T22:15:17.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: get_monmap_and_config 2026-02-21T22:15:17.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: build_initial_monmap 2026-02-21T22:15:17.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: monmap: 2026-02-21T22:15:17.423 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:17.423135+0000 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:17.423135+0000 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:17.424 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:17.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 1 Processor -- start 2026-02-21T22:15:17.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 1 -- start start 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: init 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-b con 0x7f1a3c091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-a con 0x7f1a3c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-c con 0x7f1a3c1854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c057a20 con 0x7f1a3c0909a0 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c091d20 con 0x7f1a3c1854b0 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c057740 con 0x7f1a3c091240 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a42ffa640 10 monclient(hunting): authenticate will time out at 1126.659267s 2026-02-21T22:15:17.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a33fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 0x7f1a3c183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:36280/0 (socket says 10.20.193.102:36280) 2026-02-21T22:15:17.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.418+0000 7f1a33fff640 1 -- 10.20.193.102:0/3157632602 learned_addr learned my addr 10.20.193.102:0/3157632602 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:17.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.422+0000 7f1a33fff640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a3c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:17.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.422+0000 7f1a40d6f640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6790/0 conn(0x7f1a3c091240 legacy=0x7f1a3c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:17.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:17.422+0000 7f1a41570640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.178:6789/0 conn(0x7f1a3c1854b0 legacy=0x7f1a3c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:20.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): tick 2026-02-21T22:15:20.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:15:20.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 monclient(hunting): continuing hunt 2026-02-21T22:15:20.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6790/0 conn(0x7f1a3c091240 legacy=0x7f1a3c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.178:6789/0 conn(0x7f1a3c1854b0 legacy=0x7f1a3c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a3c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): picked mon.noname-b con 0x7f1a3c091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): picked mon.noname-a con 0x7f1a3c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): picked mon.noname-c con 0x7f1a3c1854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c090dc0 con 0x7f1a3c0909a0 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a20069170 con 0x7f1a3c1854b0 2026-02-21T22:15:20.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2006a410 con 0x7f1a3c091240 2026-02-21T22:15:20.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.417+0000 7f1a327fc640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:20.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.421+0000 7f1a33fff640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a20045120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:20.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.421+0000 7f1a40d6f640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6790/0 conn(0x7f1a3c091240 legacy=0x7f1a20023970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:20.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.421+0000 7f1a41570640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.178:6789/0 conn(0x7f1a3c1854b0 legacy=0x7f1a200668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:20.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:20.421+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:15:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170438840 0 0) 0x7f1a3c090dc0 con 0x7f1a3c0909a0 2026-02-21T22:15:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): my global_id is 4098 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a28003620 con 0x7f1a3c0909a0 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082940445 0 0) 0x7f1a20069170 con 0x7f1a3c1854b0 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient(hunting): my global_id is 4102 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a3c090dc0 con 0x7f1a3c1854b0 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2159327701 0 0) 0x7f1a28003620 con 0x7f1a3c0909a0 2026-02-21T22:15:22.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6790/0 conn(0x7f1a3c091240 legacy=0x7f1a20023970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.178:6789/0 conn(0x7f1a3c1854b0 legacy=0x7f1a200668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 monclient: found mon.noname-a 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2006b710 con 0x7f1a3c0909a0 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1a24003150 con 0x7f1a3c0909a0 2026-02-21T22:15:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: dump: 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.715 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 5 monclient: authenticate success, global_id 4098 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1a24003490 con 0x7f1a3c0909a0 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 1 -- 10.20.193.102:0/3157632602 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1a24004b50 con 0x7f1a3c0909a0 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a337fe640 10 monclient: dump: 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.716 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 4 set_mon_vals no callback set 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 10 monclient: shutdown 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 1 -- 10.20.193.102:0/3157632602 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a20045120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 1 -- 10.20.193.102:0/3157632602 shutdown_connections 2026-02-21T22:15:22.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 1 -- 10.20.193.102:0/3157632602 >> 10.20.193.102:0/3157632602 conn(0x7f1a3c05e660 msgr2=0x7f1a3c095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:22.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 1 -- 10.20.193.102:0/3157632602 shutdown_connections 2026-02-21T22:15:22.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.705+0000 7f1a42ffa640 1 -- 10.20.193.102:0/3157632602 wait complete. 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: build_initial_monmap 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: monmap: 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:22.722 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 Processor -- start 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- start start 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: init 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-a con 0x7f1a3c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:22.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-c con 0x7f1a3c091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient(hunting): picked mon.noname-b con 0x7f1a3c1854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c14d660 con 0x7f1a3c0909a0 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c1348b0 con 0x7f1a3c1854b0 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3c133a40 con 0x7f1a3c091240 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient(hunting): authenticate will time out at 1131.948928s 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a40d6f640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 0x7f1a3c14cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:59574/0 (socket says 10.20.193.102:59574) 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a40d6f640 1 -- 10.20.193.102:0/207319097 learned_addr learned my addr 10.20.193.102:0/207319097 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a40d6f640 1 -- 10.20.193.102:0/207319097 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a3c14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770793404 0 0) 0x7f1a3c14d660 con 0x7f1a3c0909a0 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): my global_id is 4104 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a0c003620 con 0x7f1a3c0909a0 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2215776860 0 0) 0x7f1a0c003620 con 0x7f1a3c0909a0 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1a3c14d660 con 0x7f1a3c0909a0 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1a2c003330 con 0x7f1a3c0909a0 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): dump: 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.725 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1803397998 0 0) 0x7f1a3c14d660 con 0x7f1a3c0909a0 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 >> v1:10.20.193.102:6790/0 conn(0x7f1a3c091240 legacy=0x7f1a3c174300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 >> v1:10.20.193.178:6789/0 conn(0x7f1a3c1854b0 legacy=0x7f1a3c175a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 monclient: found mon.a 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c132550 con 0x7f1a3c0909a0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 5 monclient: authenticate success, global_id 4104 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _renew_subs 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c134010 con 0x7f1a3c0909a0 2026-02-21T22:15:22.727 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _renew_subs 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1a3c133280 con 0x7f1a3c0909a0 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1a2c0039c0 con 0x7f1a3c0909a0 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1a2c003550 con 0x7f1a3c0909a0 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a32ffd640 4 set_mon_vals no callback set 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 10 monclient: dump: 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.728 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7f1a2c004250 con 0x7f1a3c0909a0 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.709+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 720+0+0 (unknown 1238907761 0 0) 0x7f1a2c006d30 con 0x7f1a3c0909a0 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a42ffa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a42ffa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a42ffa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c0008e0 con 0x7f1a3c0909a0 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7f1a2c006230 con 0x7f1a3c0909a0 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.713+0000 7f1a317fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:22.736 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.733+0000 7f13a8824280 -1 mgr[py] Loading module balancer in main interpreter 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 10 start_mon_command cmd=[{"prefix": "config dump"}] 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 10 monclient: _send_command 2 [{"prefix": "config dump"}] 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1a3c095010 con 0x7f1a3c0909a0 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a317fa640 1 -- 10.20.193.102:0/207319097 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "config dump"}]=0 v1) ==== 59+0+36 (unknown 4069917974 0 2919340917) 0x7f1a2c005910 con 0x7f1a3c0909a0 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config dump"}] 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a317fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:22.747 INFO:teuthology.orchestra.run.trial102.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 10 monclient: shutdown 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 >> v1:10.20.193.102:6789/0 conn(0x7f1a3c0909a0 legacy=0x7f1a3c14cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 shutdown_connections 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 >> 10.20.193.102:0/207319097 conn(0x7f1a3c05e660 msgr2=0x7f1a3c089a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 shutdown_connections 2026-02-21T22:15:22.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.741+0000 7f1a42ffa640 1 -- 10.20.193.102:0/207319097 wait complete. 2026-02-21T22:15:22.759 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T22:15:22.759 DEBUG:teuthology.orchestra.run.trial102:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T22:15:22.787 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.785+0000 7f13a8824280 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-21T22:15:22.789 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.789+0000 7f13a8824280 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-21T22:15:22.801 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.801+0000 7f13a8824280 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-21T22:15:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.809+0000 7f0e43a39640 10 monclient: get_monmap_and_config 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.809+0000 7f0e43a39640 10 monclient: build_initial_monmap 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.809+0000 7f0e43a39640 10 monclient: monmap: 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:22.816610+0000 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:22.816610+0000 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:22.817 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:22.818 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 Processor -- start 2026-02-21T22:15:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- start start 2026-02-21T22:15:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient: init 2026-02-21T22:15:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-b con 0x7f0e3c13e840 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-c con 0x7f0e3c0590e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-a con 0x7f0e3c08dec0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c13f320 con 0x7f0e3c08dec0 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c057e60 con 0x7f0e3c0590e0 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c058140 con 0x7f0e3c13e840 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient(hunting): authenticate will time out at 1132.052744s 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e417ae640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 0x7f0e3c13ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39074/0 (socket says 10.20.193.102:39074) 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e417ae640 1 -- 10.20.193.102:0/2157018126 learned_addr learned my addr 10.20.193.102:0/2157018126 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e417ae640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e41faf640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.102:6789/0 conn(0x7f0e3c08dec0 legacy=0x7f0e3c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e40fad640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.178:6789/0 conn(0x7f0e3c0590e0 legacy=0x7f0e3c183df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165604214 0 0) 0x7f0e3c058140 con 0x7f0e3c13e840 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): my global_id is 4100 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e24003620 con 0x7f0e3c13e840 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589425189 0 0) 0x7f0e380041c0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): my global_id is 4110 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e3c058140 con 0x7f0e3c08dec0 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879710093 0 0) 0x7f0e3c057e60 con 0x7f0e3c0590e0 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient(hunting): my global_id is 4108 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e380041c0 con 0x7f0e3c0590e0 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901762370 0 0) 0x7f0e3c058140 con 0x7f0e3c08dec0 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.178:6789/0 conn(0x7f0e3c0590e0 legacy=0x7f0e3c183df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 monclient: found mon.noname-a 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c058420 con 0x7f0e3c08dec0 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: _finish_auth 0 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e38004840 con 0x7f0e3c08dec0 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: dump: 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.831 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 5 monclient: authenticate success, global_id 4110 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e38004c20 con 0x7f0e3c08dec0 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 1 -- 10.20.193.102:0/2157018126 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e38004fb0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e2bfff640 10 monclient: dump: 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 4 set_mon_vals no callback set 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 10 monclient: shutdown 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- 10.20.193.102:0/2157018126 >> v1:10.20.193.102:6789/0 conn(0x7f0e3c08dec0 legacy=0x7f0e3c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.813+0000 7f0e43a39640 1 -- 10.20.193.102:0/2157018126 shutdown_connections 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- 10.20.193.102:0/2157018126 >> 10.20.193.102:0/2157018126 conn(0x7f0e3c056cd0 msgr2=0x7f0e3c093d50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- 10.20.193.102:0/2157018126 shutdown_connections 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- 10.20.193.102:0/2157018126 wait complete. 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient: build_initial_monmap 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient: monmap: 2026-02-21T22:15:22.834 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 Processor -- start 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- start start 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient: init 2026-02-21T22:15:22.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-b con 0x7f0e3c0590e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-a con 0x7f0e3c08dec0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient(hunting): picked mon.noname-c con 0x7f0e3c13e840 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c132840 con 0x7f0e3c08dec0 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c12f920 con 0x7f0e3c0590e0 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c0894d0 con 0x7f0e3c13e840 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e43a39640 10 monclient(hunting): authenticate will time out at 1132.057402s 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e40fad640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0e3c08dec0 0x7f0e3c12f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:59590/0 (socket says 10.20.193.102:59590) 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e40fad640 1 -- 10.20.193.102:0/1214501075 learned_addr learned my addr 10.20.193.102:0/1214501075 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:22.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e40fad640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6789/0 conn(0x7f0e3c08dec0 legacy=0x7f0e3c12f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e41faf640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2633360012 0 0) 0x7f0e3c132840 con 0x7f0e3c08dec0 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): my global_id is 4113 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e0c003620 con 0x7f0e3c08dec0 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e417ae640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.178:6789/0 conn(0x7f0e3c0590e0 legacy=0x7f0e3c12eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941778372 0 0) 0x7f0e3c0894d0 con 0x7f0e3c13e840 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): my global_id is 4103 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e3c132840 con 0x7f0e3c13e840 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e41faf640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e41faf640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e41faf640 1 --1- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 0x7f0e3c195790 :-1 s=OPENED pgs=16 gs=3 cs=1 l=1).handle_message read tag failed 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e41faf640 1 --1- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 0x7f0e3c195790 :-1 s=OPENED pgs=16 gs=3 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.102:6790/0 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2765574767 0 0) 0x7f0e0c003620 con 0x7f0e3c08dec0 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.817+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e3c0894d0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e30003490 con 0x7f0e3c08dec0 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): dump: 2026-02-21T22:15:22.838 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 77835979 0 0) 0x7f0e3c12f920 con 0x7f0e3c0590e0 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:22.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient(hunting): my global_id is 4111 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e0c003620 con 0x7f0e3c0590e0 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2785794933 0 0) 0x7f0e3c0894d0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6790/0 conn(0x7f0e3c13e840 legacy=0x7f0e3c195790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.178:6789/0 conn(0x7f0e3c0590e0 legacy=0x7f0e3c12eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 monclient: found mon.a 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c05eac0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 5 monclient: authenticate success, global_id 4113 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _renew_subs 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c1319d0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _renew_subs 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0e3c131040 con 0x7f0e3c08dec0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e30003bf0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:22.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e30003f80 con 0x7f0e3c08dec0 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e2b7fe640 4 set_mon_vals no callback set 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: dump: 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:22.842 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:22.843 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7f0e300058c0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 720+0+0 (unknown 1238907761 0 0) 0x7f0e30006e70 con 0x7f0e3c08dec0 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c131380 con 0x7f0e3c08dec0 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7f0e300063c0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.821+0000 7f0e29ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:22.845 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.841+0000 7f13a8824280 -1 mgr[py] Loading module smb in main interpreter 2026-02-21T22:15:22.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.849+0000 7f0e43a39640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-21T22:15:22.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.849+0000 7f0e43a39640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-21T22:15:22.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.849+0000 7f0e43a39640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:22.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.849+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0e3c062ce0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e29ffb640 1 -- 10.20.193.102:0/1214501075 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (unknown 2277712955 0 0) 0x7f0e30005be0 con 0x7f0e3c08dec0 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e29ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e29ffb640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:adjusted tunables profile to default 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 10 monclient: shutdown 2026-02-21T22:15:22.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:22.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 >> v1:10.20.193.102:6789/0 conn(0x7f0e3c08dec0 legacy=0x7f0e3c12f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:22.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 shutdown_connections 2026-02-21T22:15:22.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 >> 10.20.193.102:0/1214501075 conn(0x7f0e3c056cd0 msgr2=0x7f0e3c13fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:22.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 shutdown_connections 2026-02-21T22:15:22.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:22.853+0000 7f0e43a39640 1 -- 10.20.193.102:0/1214501075 wait complete. 2026-02-21T22:15:22.870 INFO:tasks.ceph:check_enable_crimson: False 2026-02-21T22:15:22.870 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T22:15:22.871 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:22.871 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T22:15:22.882 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:22.882 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T22:15:22.910 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.909+0000 7f13a8824280 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-21T22:15:22.911 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.909+0000 7f13a8824280 -1 mgr[py] Loading module status in main interpreter 2026-02-21T22:15:22.913 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.913+0000 7f13a8824280 -1 mgr[py] Loading module nfs in main interpreter 2026-02-21T22:15:22.923 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.921+0000 7f13a8824280 -1 mgr[py] Loading module localpool in main interpreter 2026-02-21T22:15:22.924 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:22.921+0000 7f13a8824280 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-21T22:15:22.933 DEBUG:teuthology.orchestra.run.trial102:> set -ex 2026-02-21T22:15:22.933 DEBUG:teuthology.orchestra.run.trial102:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T22:15:22.985 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:22.986 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-21T22:15:22.994 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:22.994 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-21T22:15:23.006 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.005+0000 7f13a8824280 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-21T22:15:23.014 DEBUG:teuthology.orchestra.run.trial178:> set -ex 2026-02-21T22:15:23.014 DEBUG:teuthology.orchestra.run.trial178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-21T22:15:23.064 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new 883e925c-449b-4715-a8e9-b2a55b4b2970 0 2026-02-21T22:15:23.141 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.137+0000 7f13a8824280 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-21T22:15:23.142 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.141+0000 7f13a8824280 -1 mgr[py] Loading module stats in main interpreter 2026-02-21T22:15:23.144 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.141+0000 7f13a8824280 -1 mgr[py] Loading module insights in main interpreter 2026-02-21T22:15:23.146 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.145+0000 7f13a8824280 -1 mgr[py] Loading module crash in main interpreter 2026-02-21T22:15:23.149 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.149+0000 7f13a8824280 -1 mgr[py] Loading module selftest in main interpreter 2026-02-21T22:15:23.152 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.149+0000 7f13a8824280 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-21T22:15:23.159 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.157+0000 7f13a8824280 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-21T22:15:23.165 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: get_monmap_and_config 2026-02-21T22:15:23.165 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.165 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: monmap: 2026-02-21T22:15:23.165 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.165 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.166 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:23.164429+0000 2026-02-21T22:15:23.166 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:23.164429+0000 2026-02-21T22:15:23.166 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.166 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.166 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 1 Processor -- start 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 1 -- start start 2026-02-21T22:15:23.167 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: init 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-a con 0x7fb3ec13f950 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-c con 0x7fb3ec163d40 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-b con 0x7fb3ec185840 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.168 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec0574a0 con 0x7fb3ec13f950 2026-02-21T22:15:23.169 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec057780 con 0x7fb3ec163d40 2026-02-21T22:15:23.169 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec057a60 con 0x7fb3ec185840 2026-02-21T22:15:23.169 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.169 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3f175d640 10 monclient(hunting): authenticate will time out at 1167.150527s 2026-02-21T22:15:23.169 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3ea7fc640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 0x7fb3ec184120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54300/0 (socket says 10.20.193.178:54300) 2026-02-21T22:15:23.170 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.161+0000 7fb3ea7fc640 1 -- 10.20.193.178:0/2331366167 learned_addr learned my addr 10.20.193.178:0/2331366167 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.180 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3ea7fc640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 legacy=0x7fb3ec184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3eaffd640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.102:6789/0 conn(0x7fb3ec13f950 legacy=0x7fb3ec13fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3028187188 0 0) 0x7fb3ec057780 con 0x7fb3ec163d40 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient(hunting): my global_id is 4114 2026-02-21T22:15:23.181 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3eb7fe640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.102:6790/0 conn(0x7fb3ec185840 legacy=0x7fb3ec1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d8003620 con 0x7fb3ec163d40 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3417925663 0 0) 0x7fb3d8003620 con 0x7fb3ec163d40 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.102:6790/0 conn(0x7fb3ec185840 legacy=0x7fb3ec1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.102:6789/0 conn(0x7fb3ec13f950 legacy=0x7fb3ec13fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 monclient: found mon.noname-c 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.182 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec0584c0 con 0x7fb3ec163d40 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 5 monclient: authenticate success, global_id 4114 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb3d4003110 con 0x7fb3ec163d40 2026-02-21T22:15:23.183 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: dump: 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.184 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb3d4003450 con 0x7fb3ec163d40 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 1 -- 10.20.193.178:0/2331366167 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb3d4004b50 con 0x7fb3ec163d40 2026-02-21T22:15:23.185 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3e9ffb640 10 monclient: dump: 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.186 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 4 set_mon_vals no callback set 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: shutdown 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.187 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- 10.20.193.178:0/2331366167 >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 legacy=0x7fb3ec184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- 10.20.193.178:0/2331366167 shutdown_connections 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- 10.20.193.178:0/2331366167 >> 10.20.193.178:0/2331366167 conn(0x7fb3ec05e6e0 msgr2=0x7fb3ec095d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- 10.20.193.178:0/2331366167 shutdown_connections 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- 10.20.193.178:0/2331366167 wait complete. 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: monmap: 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:23.188 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 Processor -- start 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- start start 2026-02-21T22:15:23.189 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: init 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-a con 0x7fb3ec13f950 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-b con 0x7fb3ec163d40 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient(hunting): picked mon.noname-c con 0x7fb3ec185840 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec1455a0 con 0x7fb3ec13f950 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec15f1c0 con 0x7fb3ec163d40 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3ec143b70 con 0x7fb3ec185840 2026-02-21T22:15:23.190 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3f175d640 10 monclient(hunting): authenticate will time out at 1167.154381s 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3ea7fc640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 0x7fb3ec12e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54312/0 (socket says 10.20.193.178:54312) 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.165+0000 7fb3ea7fc640 1 -- 10.20.193.178:0/508959454 learned_addr learned my addr 10.20.193.178:0/508959454 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3ea7fc640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 legacy=0x7fb3ec12e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896739604 0 0) 0x7fb3ec15f1c0 con 0x7fb3ec163d40 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): my global_id is 4117 2026-02-21T22:15:23.191 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3eb7fe640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.102:6790/0 conn(0x7fb3ec185840 legacy=0x7fb3ec12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b8003620 con 0x7fb3ec163d40 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3eaffd640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.102:6789/0 conn(0x7fb3ec13f950 legacy=0x7fb3ec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3352827244 0 0) 0x7fb3b8003620 con 0x7fb3ec163d40 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb3ec15f1c0 con 0x7fb3ec163d40 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb3d40048e0 con 0x7fb3ec163d40 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:23.192 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:23.193 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740437382 0 0) 0x7fb3ec143b70 con 0x7fb3ec185840 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): my global_id is 4109 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b8003620 con 0x7fb3ec185840 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 481879277 0 0) 0x7fb3ec1455a0 con 0x7fb3ec13f950 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.194 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient(hunting): my global_id is 4119 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3ec143b70 con 0x7fb3ec13f950 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1981523001 0 0) 0x7fb3ec15f1c0 con 0x7fb3ec163d40 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.102:6790/0 conn(0x7fb3ec185840 legacy=0x7fb3ec12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.102:6789/0 conn(0x7fb3ec13f950 legacy=0x7fb3ec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 monclient: found mon.b 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec1440b0 con 0x7fb3ec163d40 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.195 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 5 monclient: authenticate success, global_id 4117 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _renew_subs 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec146410 con 0x7fb3ec163d40 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _renew_subs 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb3ec144c10 con 0x7fb3ec163d40 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb3d40032f0 con 0x7fb3ec163d40 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.196 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb3d4005320 con 0x7fb3ec163d40 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3e97fa640 4 set_mon_vals no callback set 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: dump: 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.197 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7fb3d40058d0 con 0x7fb3ec163d40 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 720+0+0 (unknown 3161725015 0 0) 0x7fb3d4006da0 con 0x7fb3ec163d40 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ec0008e0 con 0x7fb3ec163d40 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7fb3d4005bf0 con 0x7fb3ec163d40 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.198 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.169+0000 7fb3cb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:23.199 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.177+0000 7f13a8824280 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-21T22:15:23.199 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.185+0000 7f13a8824280 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-21T22:15:23.199 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.185+0000 7f13a8824280 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-21T22:15:23.204 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.201+0000 7fb3f175d640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "883e925c-449b-4715-a8e9-b2a55b4b2970", "id": 0}] 2026-02-21T22:15:23.204 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.201+0000 7fb3f175d640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "883e925c-449b-4715-a8e9-b2a55b4b2970", "id": 0}] 2026-02-21T22:15:23.204 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.201+0000 7fb3f175d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.204 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.201+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd new", "uuid": "883e925c-449b-4715-a8e9-b2a55b4b2970", "id": 0} v 0) -- 0x7fb3ec144f00 con 0x7fb3ec163d40 2026-02-21T22:15:23.205 INFO:tasks.ceph.mgr.x.trial178.stderr:ERROR:root:Module 'jmespath' is not installed. 2026-02-21T22:15:23.206 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3cb7fe640 1 -- 10.20.193.178:0/508959454 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "883e925c-449b-4715-a8e9-b2a55b4b2970", "id": 0}]=0 v3) ==== 112+0+2 (unknown 1601240975 0 732707515) 0x7fb3d4005e70 con 0x7fb3ec163d40 2026-02-21T22:15:23.206 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3cb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "883e925c-449b-4715-a8e9-b2a55b4b2970", "id": 0}] 2026-02-21T22:15:23.206 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3cb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stdout:0 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 10 monclient: shutdown 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 >> v1:10.20.193.178:6789/0 conn(0x7fb3ec163d40 legacy=0x7fb3ec12e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 shutdown_connections 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 >> 10.20.193.178:0/508959454 conn(0x7fb3ec05e6e0 msgr2=0x7fb3ec093080 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 shutdown_connections 2026-02-21T22:15:23.207 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.205+0000 7fb3f175d640 1 -- 10.20.193.178:0/508959454 wait complete. 2026-02-21T22:15:23.216 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new 00fe9fdd-1d81-4b68-8b83-dc19782e2b32 1 2026-02-21T22:15:23.228 INFO:tasks.ceph.mgr.x.trial178.stderr:ERROR:dashboard.controllers.nvmeof:Failed to import NVMeoFClient and related components: cannot import name 'NVMeoFClient' from 'dashboard.services.nvmeof_client' (/usr/share/ceph/mgr/dashboard/services/nvmeof_client.py) 2026-02-21T22:15:23.233 INFO:tasks.ceph.mgr.x.trial178.stderr:ERROR:sso:Module 'jmespath' is not installed. 2026-02-21T22:15:23.280 INFO:tasks.ceph.mgr.x.trial178.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-21T22:15:23.286 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: get_monmap_and_config 2026-02-21T22:15:23.286 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: monmap: 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:23.288293+0000 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:23.288293+0000 2026-02-21T22:15:23.287 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.288 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.288 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.288 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:23.288 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:23.288 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.289 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.285+0000 7f13a8824280 -1 mgr[py] Loading module influx in main interpreter 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 1 Processor -- start 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 1 -- start start 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: init 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:23.292 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-a con 0x7f4a8408e910 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.293 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-b con 0x7f4a84090640 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.293 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-c con 0x7f4a8408df70 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.293 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a84090430 con 0x7f4a8408e910 2026-02-21T22:15:23.293 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a84057f40 con 0x7f4a8408df70 2026-02-21T22:15:23.293 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a84058220 con 0x7f4a84090640 2026-02-21T22:15:23.294 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.294 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a890a5640 10 monclient(hunting): authenticate will time out at 1167.274355s 2026-02-21T22:15:23.294 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a83577640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f4a8408df70 0x7f4a841a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54324/0 (socket says 10.20.193.178:54324) 2026-02-21T22:15:23.294 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.285+0000 7f4a83577640 1 -- 10.20.193.178:0/3467007571 learned_addr learned my addr 10.20.193.178:0/3467007571 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.295 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.285+0000 7f13a8824280 -1 mgr[py] Loading module alerts in main interpreter 2026-02-21T22:15:23.298 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a83577640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.178:6789/0 conn(0x7f4a8408df70 legacy=0x7f4a841a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.298 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a82575640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.102:6790/0 conn(0x7f4a84090640 legacy=0x7f4a841840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531457612 0 0) 0x7f4a84057f40 con 0x7f4a8408df70 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): my global_id is 4120 2026-02-21T22:15:23.299 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a70003620 con 0x7f4a8408df70 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a82d76640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.102:6789/0 conn(0x7f4a8408e910 legacy=0x7f4a8408ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453544220 0 0) 0x7f4a84058220 con 0x7f4a84090640 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.300 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient(hunting): my global_id is 4112 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a84057f40 con 0x7f4a84090640 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3858644194 0 0) 0x7f4a70003620 con 0x7f4a8408df70 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.102:6790/0 conn(0x7f4a84090640 legacy=0x7f4a841840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.102:6789/0 conn(0x7f4a8408e910 legacy=0x7f4a8408ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 monclient: found mon.noname-c 2026-02-21T22:15:23.301 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8405e1a0 con 0x7f4a8408df70 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.302 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 5 monclient: authenticate success, global_id 4120 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4a74004620 con 0x7f4a8408df70 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: dump: 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.303 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.304 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.305 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.305 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.289+0000 7f13a8824280 -1 mgr[py] Loading module volumes in main interpreter 2026-02-21T22:15:23.312 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4a740048f0 con 0x7f4a8408df70 2026-02-21T22:15:23.312 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.312 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 1 -- 10.20.193.178:0/3467007571 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4a74006060 con 0x7f4a8408df70 2026-02-21T22:15:23.312 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:23.312 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 4 set_mon_vals no callback set 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a81d74640 10 monclient: dump: 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: shutdown 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- 10.20.193.178:0/3467007571 >> v1:10.20.193.178:6789/0 conn(0x7f4a8408df70 legacy=0x7f4a841a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- 10.20.193.178:0/3467007571 shutdown_connections 2026-02-21T22:15:23.313 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- 10.20.193.178:0/3467007571 >> 10.20.193.178:0/3467007571 conn(0x7f4a84093f10 msgr2=0x7f4a84094310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- 10.20.193.178:0/3467007571 shutdown_connections 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- 10.20.193.178:0/3467007571 wait complete. 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: monmap: 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 Processor -- start 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- start start 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: init 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:23.314 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-a con 0x7f4a8408df70 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-c con 0x7f4a8408e910 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient(hunting): picked mon.noname-b con 0x7f4a84090640 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8412eed0 con 0x7f4a8408df70 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8405ecf0 con 0x7f4a84090640 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a84130f50 con 0x7f4a8408e910 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a890a5640 10 monclient(hunting): authenticate will time out at 1167.278673s 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a83577640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f4a84090640 0x7f4a84195d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54336/0 (socket says 10.20.193.178:54336) 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.289+0000 7f4a83577640 1 -- 10.20.193.178:0/3159503728 learned_addr learned my addr 10.20.193.178:0/3159503728 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a83577640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.178:6789/0 conn(0x7f4a84090640 legacy=0x7f4a84195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939090494 0 0) 0x7f4a8405e1a0 con 0x7f4a84090640 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): my global_id is 4123 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a50003620 con 0x7f4a84090640 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a82d76640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.102:6789/0 conn(0x7f4a8408df70 legacy=0x7f4a8413f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a82575640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.102:6790/0 conn(0x7f4a8408e910 legacy=0x7f4a84194670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 786506979 0 0) 0x7f4a50003620 con 0x7f4a84090640 2026-02-21T22:15:23.315 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4a8405e1a0 con 0x7f4a84090640 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4a74004080 con 0x7f4a84090640 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): dump: 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629216862 0 0) 0x7f4a8412eed0 con 0x7f4a8408df70 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.316 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient(hunting): my global_id is 4125 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a50003620 con 0x7f4a8408df70 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3713624539 0 0) 0x7f4a8405e1a0 con 0x7f4a84090640 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.102:6790/0 conn(0x7f4a8408e910 legacy=0x7f4a84194670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.102:6789/0 conn(0x7f4a8408df70 legacy=0x7f4a8413f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 monclient: found mon.b 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a84131230 con 0x7f4a84090640 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 5 monclient: authenticate success, global_id 4123 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _renew_subs 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4a841435b0 con 0x7f4a84090640 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _renew_subs 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4a84153720 con 0x7f4a84090640 2026-02-21T22:15:23.317 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4a74006020 con 0x7f4a84090640 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4a74006680 con 0x7f4a84090640 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a81573640 4 set_mon_vals no callback set 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: dump: 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7f4a74006c30 con 0x7f4a84090640 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 835+0+0 (unknown 3818046870 0 0) 0x7f4a74008240 con 0x7f4a84090640 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.318 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.319 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84091e10 con 0x7f4a84090640 2026-02-21T22:15:23.319 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7f4a8405ecf0 con 0x7f4a84090640 2026-02-21T22:15:23.319 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.319 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.293+0000 7f4a637fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:23.319 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.317+0000 7f13a8824280 -1 mgr[py] Loading module rgw in main interpreter 2026-02-21T22:15:23.323 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.321+0000 7f13a8824280 -1 mgr[py] Loading module iostat in main interpreter 2026-02-21T22:15:23.324 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.321+0000 7f13a8824280 -1 mgr[py] Loading module progress in main interpreter 2026-02-21T22:15:23.327 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.325+0000 7f13a8824280 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-21T22:15:23.328 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.325+0000 7f4a890a5640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "00fe9fdd-1d81-4b68-8b83-dc19782e2b32", "id": 1}] 2026-02-21T22:15:23.328 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.325+0000 7f4a890a5640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "00fe9fdd-1d81-4b68-8b83-dc19782e2b32", "id": 1}] 2026-02-21T22:15:23.328 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.325+0000 7f4a890a5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.328 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.325+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd new", "uuid": "00fe9fdd-1d81-4b68-8b83-dc19782e2b32", "id": 1} v 0) -- 0x7f4a8408ff50 con 0x7f4a84090640 2026-02-21T22:15:23.330 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.329+0000 7f13a8824280 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-21T22:15:23.331 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a637fe640 1 -- 10.20.193.178:0/3159503728 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00fe9fdd-1d81-4b68-8b83-dc19782e2b32", "id": 1}]=0 v4) ==== 112+0+2 (unknown 23500934 0 940483276) 0x7f4a74008690 con 0x7f4a84090640 2026-02-21T22:15:23.331 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a637fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "00fe9fdd-1d81-4b68-8b83-dc19782e2b32", "id": 1}] 2026-02-21T22:15:23.331 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a637fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:23.331 INFO:teuthology.orchestra.run.trial178.stdout:1 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 10 monclient: shutdown 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 >> v1:10.20.193.178:6789/0 conn(0x7f4a84090640 legacy=0x7f4a84195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 shutdown_connections 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 >> 10.20.193.178:0/3159503728 conn(0x7f4a84093f10 msgr2=0x7f4a841421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 shutdown_connections 2026-02-21T22:15:23.332 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.329+0000 7f4a890a5640 1 -- 10.20.193.178:0/3159503728 wait complete. 2026-02-21T22:15:23.340 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new 1047409d-0635-4dc6-bf64-7d300e5858de 2 2026-02-21T22:15:23.450 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:23.449+0000 7f13a8824280 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: get_monmap_and_config 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: monmap: 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:23.464055+0000 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:23.464055+0000 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.462 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.463 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.463 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:23.463 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:23.463 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.464 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 1 Processor -- start 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 1 -- start start 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: init 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-a con 0x7fc67413f950 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-b con 0x7fc674163d40 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-c con 0x7fc674185840 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6740574a0 con 0x7fc67413f950 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc674057780 con 0x7fc674185840 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc674057a60 con 0x7fc674163d40 2026-02-21T22:15:23.465 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.466 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc679b30640 10 monclient(hunting): authenticate will time out at 1167.449721s 2026-02-21T22:15:23.466 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc673fff640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 0x7fc6741a5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54350/0 (socket says 10.20.193.178:54350) 2026-02-21T22:15:23.466 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.461+0000 7fc673fff640 1 -- 10.20.193.178:0/2279109438 learned_addr learned my addr 10.20.193.178:0/2279109438 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.466 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc673fff640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 legacy=0x7fc6741a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2620854678 0 0) 0x7fc674057780 con 0x7fc674185840 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient(hunting): my global_id is 4126 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc658003620 con 0x7fc674185840 2026-02-21T22:15:23.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc672ffd640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.102:6790/0 conn(0x7fc674163d40 legacy=0x7fc674184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6737fe640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.102:6789/0 conn(0x7fc67413f950 legacy=0x7fc67413fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1164488339 0 0) 0x7fc658003620 con 0x7fc674185840 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.102:6790/0 conn(0x7fc674163d40 legacy=0x7fc674184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.102:6789/0 conn(0x7fc67413f950 legacy=0x7fc67413fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 monclient: found mon.noname-c 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6740584c0 con 0x7fc674185840 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.472 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 5 monclient: authenticate success, global_id 4126 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc668004620 con 0x7fc674185840 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: dump: 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.473 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc6680048f0 con 0x7fc674185840 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 1 -- 10.20.193.178:0/2279109438 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc668006060 con 0x7fc674185840 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc6727fc640 10 monclient: dump: 2026-02-21T22:15:23.474 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 4 set_mon_vals no callback set 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: shutdown 2026-02-21T22:15:23.475 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- 10.20.193.178:0/2279109438 >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 legacy=0x7fc6741a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- 10.20.193.178:0/2279109438 shutdown_connections 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- 10.20.193.178:0/2279109438 >> 10.20.193.178:0/2279109438 conn(0x7fc67405e6e0 msgr2=0x7fc674095d70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- 10.20.193.178:0/2279109438 shutdown_connections 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- 10.20.193.178:0/2279109438 wait complete. 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: monmap: 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:23.476 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 Processor -- start 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- start start 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: init 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-a con 0x7fc67413f950 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-c con 0x7fc674163d40 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient(hunting): picked mon.noname-b con 0x7fc674185840 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc674145530 con 0x7fc67413f950 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc67415f150 con 0x7fc674185840 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc674143a20 con 0x7fc674163d40 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc679b30640 10 monclient(hunting): authenticate will time out at 1167.453683s 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc673fff640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 0x7fc67412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54358/0 (socket says 10.20.193.178:54358) 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc673fff640 1 -- 10.20.193.178:0/3568426083 learned_addr learned my addr 10.20.193.178:0/3568426083 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.465+0000 7fc673fff640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 legacy=0x7fc67412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967642612 0 0) 0x7fc67415f150 con 0x7fc674185840 2026-02-21T22:15:23.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): my global_id is 4129 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc644003620 con 0x7fc674185840 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc6737fe640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.102:6789/0 conn(0x7fc67413f950 legacy=0x7fc67415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 818014622 0 0) 0x7fc644003620 con 0x7fc674185840 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc672ffd640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.102:6790/0 conn(0x7fc674163d40 legacy=0x7fc67412e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc67415f150 con 0x7fc674185840 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc668005df0 con 0x7fc674185840 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:23.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:23.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3348861047 0 0) 0x7fc67415f150 con 0x7fc674185840 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.102:6790/0 conn(0x7fc674163d40 legacy=0x7fc67412e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.102:6789/0 conn(0x7fc67413f950 legacy=0x7fc67415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 monclient: found mon.b 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc674059050 con 0x7fc674185840 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 5 monclient: authenticate success, global_id 4129 2026-02-21T22:15:23.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _renew_subs 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc674144040 con 0x7fc674185840 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _renew_subs 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc674144ba0 con 0x7fc674185840 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc668006010 con 0x7fc674185840 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc668006680 con 0x7fc674185840 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: dump: 2026-02-21T22:15:23.482 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc671ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 1) ==== 939+0+0 (unknown 246333160 0 0) 0x7fc668006c30 con 0x7fc674185840 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 950+0+0 (unknown 1970012902 0 0) 0x7fc668008300 con 0x7fc674185840 2026-02-21T22:15:23.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6741464b0 con 0x7fc674185840 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7fc668007f50 con 0x7fc674185840 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.469+0000 7fc64ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:23.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.481+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mgrmap(e 2) ==== 50154+0+0 (unknown 4083504518 0 0) 0x7fc668006f50 con 0x7fc674185840 2026-02-21T22:15:23.502 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.501+0000 7fc679b30640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "1047409d-0635-4dc6-bf64-7d300e5858de", "id": 2}] 2026-02-21T22:15:23.503 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.501+0000 7fc679b30640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "1047409d-0635-4dc6-bf64-7d300e5858de", "id": 2}] 2026-02-21T22:15:23.503 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.501+0000 7fc679b30640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:23.503 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.501+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd new", "uuid": "1047409d-0635-4dc6-bf64-7d300e5858de", "id": 2} v 0) -- 0x7fc674144ee0 con 0x7fc674185840 2026-02-21T22:15:23.521 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Starting 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:15:23] 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:15:23, mode upmap, max misplaced 0.050000 2026-02-21T22:15:23.522 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:15:23.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:15:23.523 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:No pools available 2026-02-21T22:15:23.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Starting 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 2 seconds 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:ident set(), fault set() 2026-02-21T22:15:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: {} 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Loading... 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:No stored events to load 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Loaded [] historic events 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Loaded OSDMap, ready. 2026-02-21T22:15:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:recovery thread starting 2026-02-21T22:15:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:starting setup 2026-02-21T22:15:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:15:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:setting log level based on debug_mgr: DEBUG (20/20) 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: queue is initialized 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: starting 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:PerfHandler: starting 2026-02-21T22:15:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:sequence=0, tasks_by_sequence={}, tasks_by_id={} 2026-02-21T22:15:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TaskHandler: starting 2026-02-21T22:15:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:15:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:15:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:15:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:15:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:15:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:15:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:15:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:15:23.548 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc64ffff640 1 -- 10.20.193.178:0/3568426083 <== mon.1 v1:10.20.193.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1047409d-0635-4dc6-bf64-7d300e5858de", "id": 2}]=0 v5) ==== 112+0+2 (unknown 1896103375 0 216599125) 0x7fc668019db0 con 0x7fc674185840 2026-02-21T22:15:23.549 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc64ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "1047409d-0635-4dc6-bf64-7d300e5858de", "id": 2}] 2026-02-21T22:15:23.549 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc64ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:15:23.550 INFO:teuthology.orchestra.run.trial178.stdout:2 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:15:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:15:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:15:23.553 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 10 monclient: shutdown 2026-02-21T22:15:23.553 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:15:23.554 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 >> v1:10.20.193.178:6789/0 conn(0x7fc674185840 legacy=0x7fc67412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:15:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:15:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:23.556 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 shutdown_connections 2026-02-21T22:15:23.556 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 >> 10.20.193.178:0/3568426083 conn(0x7fc67405e6e0 msgr2=0x7fc67408adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:23.557 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 shutdown_connections 2026-02-21T22:15:23.557 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.541+0000 7fc679b30640 1 -- 10.20.193.178:0/3568426083 wait complete. 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning 4 to execute more jobs concurrently 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name cloner.1771712123.5473225.0 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [cloner.1771712123.5473225.0] starting 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name cloner.1771712123.5474763.1 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [cloner.1771712123.5474763.1] starting 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name cloner.1771712123.5477035.2 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [cloner.1771712123.5477035.2] starting 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name cloner.1771712123.5479221.3 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [cloner.1771712123.5479221.3] starting 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:tick thread cloner.tick starting 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning 4 to execute more jobs concurrently 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name purgejob.1771712123.548471.0 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [purgejob.1771712123.548471.0] starting 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name purgejob.1771712123.5488265.1 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [purgejob.1771712123.5488265.1] starting 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name purgejob.1771712123.548979.2 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [purgejob.1771712123.548979.2] starting 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:spawning new thread with name purgejob.1771712123.549155.3 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:thread [purgejob.1771712123.549155.3] starting 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:tick thread purgejob.tick starting 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locking with 5 timeout 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:mgr_util:locked 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: queue is initialized 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: starting 2026-02-21T22:15:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:setup complete 2026-02-21T22:15:23.566 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09 3 2026-02-21T22:15:23.624 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.621+0000 7f44a39ba640 10 monclient: get_monmap_and_config 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.621+0000 7f44a39ba640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.621+0000 7f44a39ba640 10 monclient: monmap: 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:23.627002+0000 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:23.627002+0000 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.625 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.626 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:23.626 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:23.626 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 1 Processor -- start 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 1 -- start start 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient: init 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:23.627 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-c con 0x7f449c08d010 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-a con 0x7f449c08c580 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-b con 0x7f449c185530 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c08aea0 con 0x7f449c08c580 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c088e30 con 0x7f449c08d010 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c08ab50 con 0x7f449c185530 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient(hunting): authenticate will time out at 1167.612620s 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a172f640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f449c08d010 0x7f449c08d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54442/0 (socket says 10.20.193.178:54442) 2026-02-21T22:15:23.628 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a172f640 1 -- 10.20.193.178:0/3339203449 learned_addr learned my addr 10.20.193.178:0/3339203449 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.631 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a172f640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.178:6789/0 conn(0x7f449c08d010 legacy=0x7f449c08d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a1f30640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.102:6790/0 conn(0x7f449c185530 legacy=0x7f449c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3509810489 0 0) 0x7f449c08ab50 con 0x7f449c185530 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient(hunting): my global_id is 4136 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a0f2e640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.102:6789/0 conn(0x7f449c08c580 legacy=0x7f449c183e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4484003620 con 0x7f449c185530 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3254607174 0 0) 0x7f4484003620 con 0x7f449c185530 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.178:6789/0 conn(0x7f449c08d010 legacy=0x7f449c08d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.632 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.102:6789/0 conn(0x7f449c08c580 legacy=0x7f449c183e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 monclient: found mon.noname-b 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c089270 con 0x7f449c185530 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4498004670 con 0x7f449c185530 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: dump: 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.633 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 5 monclient: authenticate success, global_id 4136 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4498004940 con 0x7f449c185530 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 1 -- 10.20.193.178:0/3339203449 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4498004dd0 con 0x7f449c185530 2026-02-21T22:15:23.634 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f448bfff640 10 monclient: dump: 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:23.635 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.625+0000 7f44a39ba640 4 set_mon_vals no callback set 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: shutdown 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- 10.20.193.178:0/3339203449 >> v1:10.20.193.102:6790/0 conn(0x7f449c185530 legacy=0x7f449c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- 10.20.193.178:0/3339203449 shutdown_connections 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- 10.20.193.178:0/3339203449 >> 10.20.193.178:0/3339203449 conn(0x7f449c091a50 msgr2=0x7f449c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- 10.20.193.178:0/3339203449 shutdown_connections 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- 10.20.193.178:0/3339203449 wait complete. 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: build_initial_monmap 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: monmap: 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:23.636 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 Processor -- start 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- start start 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: init 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-b con 0x7f449c08c580 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-a con 0x7f449c08d010 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient(hunting): picked mon.noname-c con 0x7f449c185530 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c080b20 con 0x7f449c08d010 2026-02-21T22:15:23.637 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c080e00 con 0x7f449c08c580 2026-02-21T22:15:23.638 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f449c185320 con 0x7f449c185530 2026-02-21T22:15:23.638 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:23.638 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a39ba640 10 monclient(hunting): authenticate will time out at 1167.617373s 2026-02-21T22:15:23.638 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a172f640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f449c08c580 0x7f449c142060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54458/0 (socket says 10.20.193.178:54458) 2026-02-21T22:15:23.638 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a172f640 1 -- 10.20.193.178:0/3532079453 learned_addr learned my addr 10.20.193.178:0/3532079453 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a172f640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.178:6789/0 conn(0x7f449c08c580 legacy=0x7f449c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a0f2e640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.102:6789/0 conn(0x7f449c08d010 legacy=0x7f449c183c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f44a1f30640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.102:6790/0 conn(0x7f449c185530 legacy=0x7f449c1209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1587925082 0 0) 0x7f449c185320 con 0x7f449c185530 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.641 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): my global_id is 4139 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f446c003620 con 0x7f449c185530 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789899166 0 0) 0x7f449c080b20 con 0x7f449c08d010 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 10 monclient(hunting): my global_id is 4155 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.629+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f449c185320 con 0x7f449c08d010 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 864555178 0 0) 0x7f446c003620 con 0x7f449c185530 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f449c080b20 con 0x7f449c185530 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4498004dd0 con 0x7f449c185530 2026-02-21T22:15:23.642 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): dump: 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.643 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3016618344 0 0) 0x7f449c185320 con 0x7f449c08d010 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f446c003620 con 0x7f449c08d010 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4490003380 con 0x7f449c08d010 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient(hunting): dump: 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.644 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 882442131 0 0) 0x7f449c080b20 con 0x7f449c185530 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.178:6789/0 conn(0x7f449c08c580 legacy=0x7f449c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.102:6789/0 conn(0x7f449c08d010 legacy=0x7f449c183c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 monclient: found mon.c 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c085310 con 0x7f449c185530 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:23.645 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 5 monclient: authenticate success, global_id 4139 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _renew_subs 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f449c080560 con 0x7f449c185530 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _renew_subs 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f449c080230 con 0x7f449c185530 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44980048e0 con 0x7f449c185530 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4498005560 con 0x7f449c185530 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f448b7fe640 4 set_mon_vals no callback set 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c0008e0 con 0x7f449c185530 2026-02-21T22:15:23.646 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: dump: 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:23.647 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 2) ==== 50154+0+0 (unknown 4083504518 0 0) 0x7f44980057b0 con 0x7f449c185530 2026-02-21T22:15:23.648 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(5..5 src has 1..5) ==== 1065+0+0 (unknown 3147974839 0 0) 0x7f449804c930 con 0x7f449c185530 2026-02-21T22:15:23.648 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7f4498018890 con 0x7f449c185530 2026-02-21T22:15:23.648 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:23.648 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.633+0000 7f4489ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:23.669 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.669+0000 7f44a39ba640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09", "id": 3}] 2026-02-21T22:15:23.669 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.669+0000 7f44a39ba640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09", "id": 3}] 2026-02-21T22:15:23.669 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.669+0000 7f44a39ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:23.669 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:23.669+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd new", "uuid": "a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09", "id": 3} v 0) -- 0x7f449c08b5b0 con 0x7f449c185530 2026-02-21T22:15:24.397 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f4489ffb640 1 -- 10.20.193.178:0/3532079453 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09", "id": 3}]=0 v6) ==== 112+0+2 (unknown 929307249 0 525046306) 0x7f449801dae0 con 0x7f449c185530 2026-02-21T22:15:24.398 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f4489ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09", "id": 3}] 2026-02-21T22:15:24.398 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f4489ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:24.398 INFO:teuthology.orchestra.run.trial178.stdout:3 2026-02-21T22:15:24.399 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 10 monclient: shutdown 2026-02-21T22:15:24.399 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:24.399 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 >> v1:10.20.193.102:6790/0 conn(0x7f449c185530 legacy=0x7f449c1209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.400 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 shutdown_connections 2026-02-21T22:15:24.400 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 >> 10.20.193.178:0/3532079453 conn(0x7f449c091a50 msgr2=0x7f449c0860d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:24.400 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 shutdown_connections 2026-02-21T22:15:24.400 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.397+0000 7f44a39ba640 1 -- 10.20.193.178:0/3532079453 wait complete. 2026-02-21T22:15:24.402 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:user is not opted-in 2026-02-21T22:15:24.402 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waiting for mgr to warm up 2026-02-21T22:15:24.408 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new 507e846b-017c-4655-839e-e62916a2128f 4 2026-02-21T22:15:24.468 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: get_monmap_and_config 2026-02-21T22:15:24.468 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: build_initial_monmap 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: monmap: 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:24.469370+0000 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:24.469370+0000 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:24.469 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 1 Processor -- start 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 1 -- start start 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: init 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-c con 0x7f189808d010 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-a con 0x7f189808c580 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-b con 0x7f1898185530 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189808aea0 con 0x7f189808c580 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1898088e30 con 0x7f189808d010 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189808ab50 con 0x7f1898185530 2026-02-21T22:15:24.470 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:24.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f189cf47640 10 monclient(hunting): authenticate will time out at 1168.455014s 2026-02-21T22:15:24.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f1896575640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f189808d010 0x7f189808d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54472/0 (socket says 10.20.193.178:54472) 2026-02-21T22:15:24.471 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.465+0000 7f1896575640 1 -- 10.20.193.178:0/3059058128 learned_addr learned my addr 10.20.193.178:0/3059058128 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:24.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1896575640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.178:6789/0 conn(0x7f189808d010 legacy=0x7f189808d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1896d76640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.102:6790/0 conn(0x7f1898185530 legacy=0x7f18981a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.477 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895d74640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.102:6789/0 conn(0x7f189808c580 legacy=0x7f1898183e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819343678 0 0) 0x7f1898088e30 con 0x7f189808d010 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): my global_id is 4138 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1884003620 con 0x7f189808d010 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231675225 0 0) 0x7f189808ab50 con 0x7f1898185530 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.478 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): my global_id is 4142 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1898088e30 con 0x7f1898185530 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193523308 0 0) 0x7f189808aea0 con 0x7f189808c580 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient(hunting): my global_id is 4158 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f189808ab50 con 0x7f189808c580 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2108023897 0 0) 0x7f1884003620 con 0x7f189808d010 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.102:6790/0 conn(0x7f1898185530 legacy=0x7f18981a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.102:6789/0 conn(0x7f189808c580 legacy=0x7f1898183e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: _finish_hunting 0 2026-02-21T22:15:24.479 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 monclient: found mon.noname-c 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1898089270 con 0x7f189808d010 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: _finish_auth 0 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: _check_auth_tickets 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 5 monclient: authenticate success, global_id 4138 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f188c003110 con 0x7f189808d010 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: dump: 2026-02-21T22:15:24.480 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f188c0033e0 con 0x7f189808d010 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:24.481 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 1 -- 10.20.193.178:0/3059058128 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f188c004b50 con 0x7f189808d010 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1895573640 10 monclient: dump: 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.482 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 4 set_mon_vals no callback set 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: shutdown 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- 10.20.193.178:0/3059058128 >> v1:10.20.193.178:6789/0 conn(0x7f189808d010 legacy=0x7f189808d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- 10.20.193.178:0/3059058128 shutdown_connections 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- 10.20.193.178:0/3059058128 >> 10.20.193.178:0/3059058128 conn(0x7f1898091a50 msgr2=0x7f1898091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- 10.20.193.178:0/3059058128 shutdown_connections 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- 10.20.193.178:0/3059058128 wait complete. 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: build_initial_monmap 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: monmap: 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:24.483 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 Processor -- start 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- start start 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: init 2026-02-21T22:15:24.484 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-a con 0x7f189808c580 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-c con 0x7f189808d010 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient(hunting): picked mon.noname-b con 0x7f1898185530 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1898080f10 con 0x7f189808c580 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1898089f80 con 0x7f1898185530 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1898185320 con 0x7f189808d010 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f189cf47640 10 monclient(hunting): authenticate will time out at 1168.459817s 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1896d76640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f1898185530 0x7f18981209e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54488/0 (socket says 10.20.193.178:54488) 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.469+0000 7f1896d76640 1 -- 10.20.193.178:0/2032149564 learned_addr learned my addr 10.20.193.178:0/2032149564 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:24.485 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f1896d76640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.178:6789/0 conn(0x7f1898185530 legacy=0x7f18981209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f1895d74640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.102:6790/0 conn(0x7f189808d010 legacy=0x7f1898183c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650505876 0 0) 0x7f1898089f80 con 0x7f1898185530 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): my global_id is 4141 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1864003620 con 0x7f1898185530 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f1896575640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.102:6789/0 conn(0x7f189808c580 legacy=0x7f1898142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4000072350 0 0) 0x7f1864003620 con 0x7f1898185530 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1898089f80 con 0x7f1898185530 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1888004900 con 0x7f1898185530 2026-02-21T22:15:24.486 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): dump: 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.487 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888914203 0 0) 0x7f1898185320 con 0x7f189808d010 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): my global_id is 4145 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1864003620 con 0x7f189808d010 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656544520 0 0) 0x7f1898080f10 con 0x7f189808c580 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient(hunting): my global_id is 4161 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1898185320 con 0x7f189808c580 2026-02-21T22:15:24.488 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1229825367 0 0) 0x7f1898089f80 con 0x7f1898185530 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.102:6790/0 conn(0x7f189808d010 legacy=0x7f1898183c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.102:6789/0 conn(0x7f189808c580 legacy=0x7f1898142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 monclient: found mon.b 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189807ebb0 con 0x7f1898185530 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: _finish_auth 0 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 5 monclient: authenticate success, global_id 4141 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _renew_subs 2026-02-21T22:15:24.489 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1898080c30 con 0x7f1898185530 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _renew_subs 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f189807e2b0 con 0x7f1898185530 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1888004ed0 con 0x7f1898185530 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1888005260 con 0x7f1898185530 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 10 monclient: dump: 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.490 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f1894d72640 4 set_mon_vals no callback set 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.491 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189805eaf0 con 0x7f1898185530 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 2) ==== 50154+0+0 (unknown 4083504518 0 0) 0x7f1888005530 con 0x7f1898185530 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.473+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(6..6 src has 1..6) ==== 1180+0+0 (unknown 1119783581 0 0) 0x7f188804cb30 con 0x7f1898185530 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.477+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+77541 (unknown 1092875540 0 2392355004) 0x7f18880189b0 con 0x7f1898185530 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.477+0000 7f187effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.477+0000 7f187effd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:24.492 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.485+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mgrmap(e 3) ==== 50246+0+0 (unknown 3547795549 0 0) 0x7f188804b790 con 0x7f1898185530 2026-02-21T22:15:24.493 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.489+0000 7f1896575640 10 monclient: get_auth_request con 0x7f186403e770 auth_method 0 2026-02-21T22:15:24.498 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:24.497+0000 7f12d29b1640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-21T22:15:24.500 INFO:tasks.ceph.mgr.x.trial178.stderr:2026-02-21T22:15:24.497+0000 7f12d29b1640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-21T22:15:24.509 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.509+0000 7f189cf47640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "507e846b-017c-4655-839e-e62916a2128f", "id": 4}] 2026-02-21T22:15:24.510 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.509+0000 7f189cf47640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "507e846b-017c-4655-839e-e62916a2128f", "id": 4}] 2026-02-21T22:15:24.510 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.509+0000 7f189cf47640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.510 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.509+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd new", "uuid": "507e846b-017c-4655-839e-e62916a2128f", "id": 4} v 0) -- 0x7f1898062ce0 con 0x7f1898185530 2026-02-21T22:15:24.599 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f187effd640 1 -- 10.20.193.178:0/2032149564 <== mon.1 v1:10.20.193.178:6789/0 11 ==== mon_command_ack([{"prefix": "osd new", "uuid": "507e846b-017c-4655-839e-e62916a2128f", "id": 4}]=0 v7) ==== 112+0+2 (unknown 758773586 0 1697012583) 0x7f1888019b40 con 0x7f1898185530 2026-02-21T22:15:24.599 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f187effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "507e846b-017c-4655-839e-e62916a2128f", "id": 4}] 2026-02-21T22:15:24.599 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f187effd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:24.600 INFO:teuthology.orchestra.run.trial178.stdout:4 2026-02-21T22:15:24.600 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.178:6800/2549949439 conn(0x7f186403e770 legacy=0x7f186405eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.600 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 10 monclient: shutdown 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 >> v1:10.20.193.178:6789/0 conn(0x7f1898185530 legacy=0x7f18981209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 shutdown_connections 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 >> 10.20.193.178:0/2032149564 conn(0x7f1898091a50 msgr2=0x7f18981a5910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 shutdown_connections 2026-02-21T22:15:24.601 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.597+0000 7f189cf47640 1 -- 10.20.193.178:0/2032149564 wait complete. 2026-02-21T22:15:24.604 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:15:24.604 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:24.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:24.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:15:24.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:15:24.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:15:24.611 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:15:24.612 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:15:24.613 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:24.614 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:15:24.615 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:15:24.617 DEBUG:teuthology.orchestra.run.trial178:> sudo ceph --cluster ceph osd new c24d73e4-3129-4e17-bde0-940b2c5e2abf 5 2026-02-21T22:15:24.618 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:15:24.618 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:15:24.618 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:15:24.618 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:15:24.618 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:15:24.677 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: get_monmap_and_config 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: build_initial_monmap 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: monmap: 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:24.679414+0000 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:24.679414+0000 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:24.678 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:24.679 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.679 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 1 Processor -- start 2026-02-21T22:15:24.679 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 1 -- start start 2026-02-21T22:15:24.679 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: init 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-a con 0x7f9e78088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-b con 0x7f9e78086260 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-c con 0x7f9e78089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7808a380 con 0x7f9e78088d80 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7808a630 con 0x7f9e78089860 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7808b6d0 con 0x7f9e78086260 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:24.684 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7f0dc640 10 monclient(hunting): authenticate will time out at 1168.665253s 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7d652640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f9e78089860 0x7f9e781a2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54504/0 (socket says 10.20.193.178:54504) 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7d652640 1 -- 10.20.193.178:0/3895523207 learned_addr learned my addr 10.20.193.178:0/3895523207 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7d652640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.178:6789/0 conn(0x7f9e78089860 legacy=0x7f9e781a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6ffff640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.102:6790/0 conn(0x7f9e78086260 legacy=0x7f9e78180c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e7ce51640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.102:6789/0 conn(0x7f9e78088d80 legacy=0x7f9e78089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745496308 0 0) 0x7f9e7808a630 con 0x7f9e78089860 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.685 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): my global_id is 4144 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e64003620 con 0x7f9e78089860 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449283995 0 0) 0x7f9e7808a380 con 0x7f9e78088d80 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.686 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 10 monclient(hunting): my global_id is 4164 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.677+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e7808a630 con 0x7f9e78088d80 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2218370273 0 0) 0x7f9e64003620 con 0x7f9e78089860 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.102:6790/0 conn(0x7f9e78086260 legacy=0x7f9e78180c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.687 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.102:6789/0 conn(0x7f9e78088d80 legacy=0x7f9e78089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 monclient: found mon.noname-c 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7808b9b0 con 0x7f9e78089860 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9e74004620 con 0x7f9e78089860 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.688 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: dump: 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.689 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 5 monclient: authenticate success, global_id 4144 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9e740048f0 con 0x7f9e78089860 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 1 -- 10.20.193.178:0/3895523207 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9e74006060 con 0x7f9e78089860 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6f7fe640 10 monclient: dump: 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.690 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 4 set_mon_vals no callback set 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: shutdown 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/3895523207 >> v1:10.20.193.178:6789/0 conn(0x7f9e78089860 legacy=0x7f9e781a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/3895523207 shutdown_connections 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/3895523207 >> 10.20.193.178:0/3895523207 conn(0x7f9e78091a50 msgr2=0x7f9e78091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:24.691 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/3895523207 shutdown_connections 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/3895523207 wait complete. 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: build_initial_monmap 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: monmap: 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:epoch 0 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 0.000000 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:created 0.000000 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 1 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:24.692 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 Processor -- start 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- start start 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: init 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-b con 0x7f9e78086260 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-c con 0x7f9e78088d80 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient(hunting): picked mon.noname-a con 0x7f9e78089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e78088a70 con 0x7f9e78089860 2026-02-21T22:15:24.693 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7807ec40 con 0x7f9e78086260 2026-02-21T22:15:24.694 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7807ef20 con 0x7f9e78088d80 2026-02-21T22:15:24.694 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:24.694 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7f0dc640 10 monclient(hunting): authenticate will time out at 1168.669525s 2026-02-21T22:15:24.694 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7ce51640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f9e78086260 0x7f9e78084a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.178:54518/0 (socket says 10.20.193.178:54518) 2026-02-21T22:15:24.694 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7ce51640 1 -- 10.20.193.178:0/831958574 learned_addr learned my addr 10.20.193.178:0/831958574 (peer_addr_for_me v1:10.20.193.178:0/0) 2026-02-21T22:15:24.697 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7ce51640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.178:6789/0 conn(0x7f9e78086260 legacy=0x7f9e78084a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.697 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e6ffff640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.102:6790/0 conn(0x7f9e78088d80 legacy=0x7f9e781304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.697 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.681+0000 7f9e7d652640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.102:6789/0 conn(0x7f9e78089860 legacy=0x7f9e78131bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188935463 0 0) 0x7f9e7807ec40 con 0x7f9e78086260 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): my global_id is 4147 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e48003620 con 0x7f9e78086260 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899064666 0 0) 0x7f9e78088a70 con 0x7f9e78089860 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): my global_id is 4167 2026-02-21T22:15:24.698 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e7807ec40 con 0x7f9e78089860 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 546886427 0 0) 0x7f9e48003620 con 0x7f9e78086260 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9e78088a70 con 0x7f9e78086260 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9e68003490 con 0x7f9e78086260 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): dump: 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.699 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2719045648 0 0) 0x7f9e78088a70 con 0x7f9e78086260 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.102:6790/0 conn(0x7f9e78088d80 legacy=0x7f9e781304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.102:6789/0 conn(0x7f9e78089860 legacy=0x7f9e78131bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 monclient: found mon.b 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:24.700 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7807dce0 con 0x7f9e78086260 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 5 monclient: authenticate success, global_id 4147 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _renew_subs 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9e78085160 con 0x7f9e78086260 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _renew_subs 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9e7807d390 con 0x7f9e78086260 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9e68003c30 con 0x7f9e78086260 2026-02-21T22:15:24.701 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9e68003fc0 con 0x7f9e78086260 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: dump: 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:epoch 1 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:election_strategy: 3 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:24.702 INFO:teuthology.orchestra.run.trial178.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6effd640 4 set_mon_vals no callback set 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 3) ==== 50246+0+0 (unknown 3547795549 0 0) 0x7f9e68004210 con 0x7f9e78086260 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e780008e0 con 0x7f9e78086260 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(7..7 src has 1..7) ==== 1295+0+0 (unknown 605759297 0 0) 0x7f9e6804b9a0 con 0x7f9e78086260 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6ffff640 10 monclient: get_auth_request con 0x7f9e4803d380 auth_method 0 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9e68017750 con 0x7f9e78086260 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:24.703 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.685+0000 7f9e6d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:24.786 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.785+0000 7f9e7f0dc640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c24d73e4-3129-4e17-bde0-940b2c5e2abf", "id": 5}] 2026-02-21T22:15:24.786 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.785+0000 7f9e7f0dc640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c24d73e4-3129-4e17-bde0-940b2c5e2abf", "id": 5}] 2026-02-21T22:15:24.786 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.785+0000 7f9e7f0dc640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:24.786 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:24.785+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd new", "uuid": "c24d73e4-3129-4e17-bde0-940b2c5e2abf", "id": 5} v 0) -- 0x7f9e7807f140 con 0x7f9e78086260 2026-02-21T22:15:25.259 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e6d7fa640 1 -- 10.20.193.178:0/831958574 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c24d73e4-3129-4e17-bde0-940b2c5e2abf", "id": 5}]=0 v8) ==== 112+0+2 (unknown 3587015471 0 1988412176) 0x7f9e68017030 con 0x7f9e78086260 2026-02-21T22:15:25.259 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e6d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c24d73e4-3129-4e17-bde0-940b2c5e2abf", "id": 5}] 2026-02-21T22:15:25.259 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e6d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:25.260 INFO:teuthology.orchestra.run.trial178.stdout:5 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9e4803d380 legacy=0x7f9e4805d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 10 monclient: shutdown 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 >> v1:10.20.193.178:6789/0 conn(0x7f9e78086260 legacy=0x7f9e78084a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 shutdown_connections 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 >> 10.20.193.178:0/831958574 conn(0x7f9e78091a50 msgr2=0x7f9e78058510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 shutdown_connections 2026-02-21T22:15:25.261 INFO:teuthology.orchestra.run.trial178.stderr:2026-02-21T22:15:25.257+0000 7f9e7f0dc640 1 -- 10.20.193.178:0/831958574 wait complete. 2026-02-21T22:15:25.273 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T22:15:25.273 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-21T22:15:25.275 INFO:tasks.ceph.osd.0:Started 2026-02-21T22:15:25.275 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T22:15:25.275 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-21T22:15:25.277 INFO:tasks.ceph.osd.1:Started 2026-02-21T22:15:25.278 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T22:15:25.278 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-21T22:15:25.280 INFO:tasks.ceph.osd.2:Started 2026-02-21T22:15:25.280 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-21T22:15:25.280 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-21T22:15:25.282 INFO:tasks.ceph.osd.3:Started 2026-02-21T22:15:25.282 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-21T22:15:25.282 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2026-02-21T22:15:25.284 INFO:tasks.ceph.osd.4:Started 2026-02-21T22:15:25.284 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-21T22:15:25.285 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2026-02-21T22:15:25.286 INFO:tasks.ceph.osd.5:Started 2026-02-21T22:15:25.287 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:25.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: get_monmap_and_config 2026-02-21T22:15:25.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: build_initial_monmap 2026-02-21T22:15:25.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: monmap: 2026-02-21T22:15:25.351 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:25.349769+0000 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:25.349769+0000 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 1 Processor -- start 2026-02-21T22:15:25.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 1 -- start start 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: init 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-a con 0x7f251c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-c con 0x7f251c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-b con 0x7f251c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c086a10 con 0x7f251c08ce00 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c08b130 con 0x7f251c08b7f0 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c08bd60 con 0x7f251c08d8e0 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f2520b6b640 10 monclient(hunting): authenticate will time out at 1134.585648s 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f251a575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f251c08ce00 0x7f251c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:59626/0 (socket says 10.20.193.102:59626) 2026-02-21T22:15:25.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f251a575640 1 -- 10.20.193.102:0/1445264780 learned_addr learned my addr 10.20.193.102:0/1445264780 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:25.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f251a575640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.102:6789/0 conn(0x7f251c08ce00 legacy=0x7f251c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.345+0000 7f251ad76640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.102:6790/0 conn(0x7f251c08d8e0 legacy=0x7f251c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519d74640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.178:6789/0 conn(0x7f251c08b7f0 legacy=0x7f251c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009567853 0 0) 0x7f251c086a10 con 0x7f251c08ce00 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): my global_id is 4188 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2508003620 con 0x7f251c08ce00 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721643857 0 0) 0x7f251c08bd60 con 0x7f251c08d8e0 2026-02-21T22:15:25.363 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): my global_id is 4166 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f251c086a10 con 0x7f251c08d8e0 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711301850 0 0) 0x7f251c08b130 con 0x7f251c08b7f0 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient(hunting): my global_id is 4159 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f251c08bd60 con 0x7f251c08b7f0 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2283475904 0 0) 0x7f2508003620 con 0x7f251c08ce00 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.102:6790/0 conn(0x7f251c08d8e0 legacy=0x7f251c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.178:6789/0 conn(0x7f251c08b7f0 legacy=0x7f251c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: _finish_hunting 0 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 monclient: found mon.noname-a 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c08b370 con 0x7f251c08ce00 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: _finish_auth 0 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: _check_auth_tickets 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2510003110 con 0x7f251c08ce00 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:25.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: dump: 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 5 monclient: authenticate success, global_id 4188 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:25.366 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f25100033e0 con 0x7f251c08ce00 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 4 set_mon_vals no callback set 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 1 -- 10.20.193.102:0/1445264780 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2510004b50 con 0x7f251c08ce00 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519573640 10 monclient: dump: 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:25.367 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: shutdown 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- 10.20.193.102:0/1445264780 >> v1:10.20.193.102:6789/0 conn(0x7f251c08ce00 legacy=0x7f251c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- 10.20.193.102:0/1445264780 shutdown_connections 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- 10.20.193.102:0/1445264780 >> 10.20.193.102:0/1445264780 conn(0x7f251c091a50 msgr2=0x7f251c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- 10.20.193.102:0/1445264780 shutdown_connections 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- 10.20.193.102:0/1445264780 wait complete. 2026-02-21T22:15:25.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: build_initial_monmap 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: monmap: 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 Processor -- start 2026-02-21T22:15:25.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- start start 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: init 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-b con 0x7f251c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-c con 0x7f251c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient(hunting): picked mon.noname-a con 0x7f251c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c085350 con 0x7f251c08d8e0 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c0835b0 con 0x7f251c08b7f0 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251c058940 con 0x7f251c08ce00 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2520b6b640 10 monclient(hunting): authenticate will time out at 1134.589762s 2026-02-21T22:15:25.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f251ad76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f251c08d8e0 0x7f251c195660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:59630/0 (socket says 10.20.193.102:59630) 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f251c08ce00 0x7f251c0894c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39136/0 (socket says 10.20.193.102:39136) 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f251ad76640 1 -- 10.20.193.102:0/891106892 learned_addr learned my addr 10.20.193.102:0/891106892 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f251ad76640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.102:6789/0 conn(0x7f251c08d8e0 legacy=0x7f251c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.349+0000 7f2519d74640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.102:6790/0 conn(0x7f251c08ce00 legacy=0x7f251c0894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922111401 0 0) 0x7f251c085350 con 0x7f251c08d8e0 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): my global_id is 4191 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e8003620 con 0x7f251c08d8e0 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687705835 0 0) 0x7f251c058940 con 0x7f251c08ce00 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): my global_id is 4169 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f251c085350 con 0x7f251c08ce00 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f251a575640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.178:6789/0 conn(0x7f251c08b7f0 legacy=0x7f251c088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1377893350 0 0) 0x7f24e8003620 con 0x7f251c08d8e0 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f251c058940 con 0x7f251c08d8e0 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f250c004960 con 0x7f251c08d8e0 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): dump: 2026-02-21T22:15:25.372 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600027807 0 0) 0x7f251c0835b0 con 0x7f251c08b7f0 2026-02-21T22:15:25.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): my global_id is 4162 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24e8003620 con 0x7f251c08b7f0 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 709985795 0 0) 0x7f251c085350 con 0x7f251c08ce00 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f251c0835b0 con 0x7f251c08ce00 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f25040033c0 con 0x7f251c08ce00 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient(hunting): dump: 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:25.374 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2670303768 0 0) 0x7f251c058940 con 0x7f251c08d8e0 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.102:6790/0 conn(0x7f251c08ce00 legacy=0x7f251c0894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.178:6789/0 conn(0x7f251c08b7f0 legacy=0x7f251c088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 monclient: found mon.a 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c089bd0 con 0x7f251c08d8e0 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 5 monclient: authenticate success, global_id 4191 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _renew_subs 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f251c05e0e0 con 0x7f251c08d8e0 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _renew_subs 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f251c0832d0 con 0x7f251c08d8e0 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f250c004ed0 con 0x7f251c08d8e0 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f250c0052a0 con 0x7f251c08d8e0 2026-02-21T22:15:25.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2518d72640 4 set_mon_vals no callback set 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: dump: 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:25.378 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251c081e90 con 0x7f251c08d8e0 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 3) ==== 50246+0+0 (unknown 3547795549 0 0) 0x7f250c005480 con 0x7f251c08d8e0 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(8..8 src has 1..8) ==== 1410+0+0 (unknown 1574384669 0 0) 0x7f250c04cd20 con 0x7f251c08d8e0 2026-02-21T22:15:25.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f251a575640 10 monclient: get_auth_request con 0x7f24e803d6e0 auth_method 0 2026-02-21T22:15:25.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f250c018ab0 con 0x7f251c08d8e0 2026-02-21T22:15:25.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:25.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.353+0000 7f2502ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:25.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:25.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f251c062ce0 con 0x7f251c08d8e0 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2502ffd640 1 -- 10.20.193.102:0/891106892 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5603 (unknown 150273500 0 2058806927) 0x7f250c018390 con 0x7f251c08d8e0 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2502ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2502ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:25.447 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:25.448 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":8,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:24.786352+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:25.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.178:6800/2549949439 conn(0x7f24e803d6e0 legacy=0x7f24e805db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 10 monclient: shutdown 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 >> v1:10.20.193.102:6789/0 conn(0x7f251c08d8e0 legacy=0x7f251c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 shutdown_connections 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 >> 10.20.193.102:0/891106892 conn(0x7f251c091a50 msgr2=0x7f251c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 shutdown_connections 2026-02-21T22:15:25.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:25.441+0000 7f2520b6b640 1 -- 10.20.193.102:0/891106892 wait complete. 2026-02-21T22:15:25.458 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T22:15:25.458 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T22:15:25.524 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:not enough osds to create mgr pool 2026-02-21T22:15:25.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 2 seconds 2026-02-21T22:15:25.662 INFO:tasks.ceph.osd.5.trial178.stderr:2026-02-21T22:15:25.661+0000 7fc2fdc02a40 -1 Falling back to public interface 2026-02-21T22:15:25.686 INFO:tasks.ceph.osd.1.trial102.stderr:2026-02-21T22:15:25.681+0000 7fe947325a40 -1 Falling back to public interface 2026-02-21T22:15:25.694 INFO:tasks.ceph.osd.3.trial178.stderr:2026-02-21T22:15:25.693+0000 7f8eed033a40 -1 Falling back to public interface 2026-02-21T22:15:25.694 INFO:tasks.ceph.osd.0.trial102.stderr:2026-02-21T22:15:25.689+0000 7fbdcfb5aa40 -1 Falling back to public interface 2026-02-21T22:15:25.709 INFO:tasks.ceph.osd.2.trial102.stderr:2026-02-21T22:15:25.705+0000 7ff6318eba40 -1 Falling back to public interface 2026-02-21T22:15:25.734 INFO:tasks.ceph.osd.4.trial178.stderr:2026-02-21T22:15:25.733+0000 7f6975708a40 -1 Falling back to public interface 2026-02-21T22:15:26.063 DEBUG:teuthology.orchestra.run.trial102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: get_monmap_and_config 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: build_initial_monmap 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: monmap: 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:epoch 0 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:26.118741+0000 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:26.118741+0000 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:26.120 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 1 Processor -- start 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 1 -- start start 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: init 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-b con 0x7f95a413f720 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:26.121 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-c con 0x7f95a40939d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-a con 0x7f95a41851e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a4056170 con 0x7f95a41851e0 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a40570f0 con 0x7f95a40939d0 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a40573d0 con 0x7f95a413f720 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a8f24640 10 monclient(hunting): authenticate will time out at 1135.354705s 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a37fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f95a413f720 0x7f95a413faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39152/0 (socket says 10.20.193.102:39152) 2026-02-21T22:15:26.122 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.113+0000 7f95a37fe640 1 -- 10.20.193.102:0/3750467245 learned_addr learned my addr 10.20.193.102:0/3750467245 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:26.128 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a37fe640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.102:6790/0 conn(0x7f95a413f720 legacy=0x7f95a413faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a3fff640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.102:6789/0 conn(0x7f95a41851e0 legacy=0x7f95a41a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a2ffd640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.178:6789/0 conn(0x7f95a40939d0 legacy=0x7f95a4183ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3669605252 0 0) 0x7f95a40573d0 con 0x7f95a413f720 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): my global_id is 4172 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9588003620 con 0x7f95a413f720 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480047222 0 0) 0x7f95a4056170 con 0x7f95a41851e0 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:26.129 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient(hunting): my global_id is 4194 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a40573d0 con 0x7f95a41851e0 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3198665409 0 0) 0x7f95a40573d0 con 0x7f95a41851e0 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.102:6790/0 conn(0x7f95a413f720 legacy=0x7f95a413faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.178:6789/0 conn(0x7f95a40939d0 legacy=0x7f95a4183ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 monclient: found mon.noname-a 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a40576b0 con 0x7f95a41851e0 2026-02-21T22:15:26.130 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9598004670 con 0x7f95a41851e0 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: dump: 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:26.131 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 5 monclient: authenticate success, global_id 4194 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9598004940 con 0x7f95a41851e0 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 1 -- 10.20.193.102:0/3750467245 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9598004dd0 con 0x7f95a41851e0 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a27fc640 10 monclient: dump: 2026-02-21T22:15:26.132 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 4 set_mon_vals no callback set 2026-02-21T22:15:26.133 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: shutdown 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- 10.20.193.102:0/3750467245 >> v1:10.20.193.102:6789/0 conn(0x7f95a41851e0 legacy=0x7f95a41a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- 10.20.193.102:0/3750467245 shutdown_connections 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- 10.20.193.102:0/3750467245 >> 10.20.193.102:0/3750467245 conn(0x7f95a405e390 msgr2=0x7f95a4095bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- 10.20.193.102:0/3750467245 shutdown_connections 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- 10.20.193.102:0/3750467245 wait complete. 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: build_initial_monmap 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: monmap: 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:epoch 0 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:created 0.000000 2026-02-21T22:15:26.134 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 1 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 Processor -- start 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- start start 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: init 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-b con 0x7f95a40939d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:26.135 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-a con 0x7f95a413f720 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient(hunting): picked mon.noname-c con 0x7f95a41851e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a410e6b0 con 0x7f95a413f720 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a412dbd0 con 0x7f95a40939d0 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a410d840 con 0x7f95a41851e0 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a8f24640 10 monclient(hunting): authenticate will time out at 1135.359347s 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a2ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f95a413f720 0x7f95a412d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:59634/0 (socket says 10.20.193.102:59634) 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.117+0000 7f95a2ffd640 1 -- 10.20.193.102:0/4284876375 learned_addr learned my addr 10.20.193.102:0/4284876375 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a2ffd640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.102:6789/0 conn(0x7f95a413f720 legacy=0x7f95a412d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a3fff640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.102:6790/0 conn(0x7f95a41851e0 legacy=0x7f95a41848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326760731 0 0) 0x7f95a410e6b0 con 0x7f95a413f720 2026-02-21T22:15:26.136 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): my global_id is 4197 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9574003620 con 0x7f95a413f720 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a37fe640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.178:6789/0 conn(0x7f95a40939d0 legacy=0x7f95a412cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462768764 0 0) 0x7f95a410d840 con 0x7f95a41851e0 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): my global_id is 4175 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a410e6b0 con 0x7f95a41851e0 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1647976846 0 0) 0x7f9574003620 con 0x7f95a413f720 2026-02-21T22:15:26.137 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f95a410d840 con 0x7f95a413f720 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9594003400 con 0x7f95a413f720 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:26.138 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207522455 0 0) 0x7f95a412dbd0 con 0x7f95a40939d0 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient(hunting): my global_id is 4168 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9574003620 con 0x7f95a40939d0 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2425678076 0 0) 0x7f95a410d840 con 0x7f95a413f720 2026-02-21T22:15:26.139 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.102:6790/0 conn(0x7f95a41851e0 legacy=0x7f95a41848a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.178:6789/0 conn(0x7f95a40939d0 legacy=0x7f95a412cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 monclient: found mon.a 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a410c350 con 0x7f95a413f720 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 5 monclient: authenticate success, global_id 4197 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _renew_subs 2026-02-21T22:15:26.140 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f95a408ea80 con 0x7f95a413f720 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _renew_subs 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f95a410de10 con 0x7f95a413f720 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f95940039c0 con 0x7f95a413f720 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9594003d50 con 0x7f95a413f720 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a1ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 10 monclient: dump: 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:26.141 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.142 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4091810 con 0x7f95a413f720 2026-02-21T22:15:26.143 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 3) ==== 50246+0+0 (unknown 3547795549 0 0) 0x7f9594003fa0 con 0x7f95a413f720 2026-02-21T22:15:26.143 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(8..8 src has 1..8) ==== 1410+0+0 (unknown 1574384669 0 0) 0x7f959404b8b0 con 0x7f95a413f720 2026-02-21T22:15:26.143 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.121+0000 7f95a37fe640 10 monclient: get_auth_request con 0x7f957403d2a0 auth_method 0 2026-02-21T22:15:26.144 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.125+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9594017640 con 0x7f95a413f720 2026-02-21T22:15:26.144 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.125+0000 7f957ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:26.144 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.125+0000 7f957ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95a405ea70 con 0x7f95a413f720 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f957ffff640 1 -- 10.20.193.102:0/4284876375 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5603 (unknown 150273500 0 2058806927) 0x7f9594016f20 con 0x7f95a413f720 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f957ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:26.215 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f957ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:26.216 INFO:teuthology.misc.health.trial102.stdout: 2026-02-21T22:15:26.216 INFO:teuthology.misc.health.trial102.stdout:{"epoch":8,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:24.786352+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:26.217 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.178:6800/2549949439 conn(0x7f957403d2a0 legacy=0x7f957405d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.217 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 10 monclient: shutdown 2026-02-21T22:15:26.217 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:26.217 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 >> v1:10.20.193.102:6789/0 conn(0x7f95a413f720 legacy=0x7f95a412d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:26.217 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 shutdown_connections 2026-02-21T22:15:26.218 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 >> 10.20.193.102:0/4284876375 conn(0x7f95a405e390 msgr2=0x7f95a41a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:26.218 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 shutdown_connections 2026-02-21T22:15:26.218 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:26.209+0000 7f95a8f24640 1 -- 10.20.193.102:0/4284876375 wait complete. 2026-02-21T22:15:26.226 DEBUG:teuthology.misc:0 of 6 OSDs are up 2026-02-21T22:15:26.334 INFO:tasks.ceph.osd.5.trial178.stderr:2026-02-21T22:15:26.333+0000 7fc2fdc02a40 -1 osd.5 0 log_to_monitors true 2026-02-21T22:15:26.344 INFO:tasks.ceph.osd.3.trial178.stderr:2026-02-21T22:15:26.341+0000 7f8eed033a40 -1 osd.3 0 log_to_monitors true 2026-02-21T22:15:26.365 INFO:tasks.ceph.osd.0.trial102.stderr:2026-02-21T22:15:26.357+0000 7fbdcfb5aa40 -1 osd.0 0 log_to_monitors true 2026-02-21T22:15:26.379 INFO:tasks.ceph.osd.1.trial102.stderr:2026-02-21T22:15:26.373+0000 7fe947325a40 -1 osd.1 0 log_to_monitors true 2026-02-21T22:15:26.391 INFO:tasks.ceph.osd.2.trial102.stderr:2026-02-21T22:15:26.385+0000 7ff6318eba40 -1 osd.2 0 log_to_monitors true 2026-02-21T22:15:26.435 INFO:tasks.ceph.osd.4.trial178.stderr:2026-02-21T22:15:26.433+0000 7f6975708a40 -1 osd.4 0 log_to_monitors true 2026-02-21T22:15:27.411 INFO:tasks.ceph.osd.1.trial102.stderr:2026-02-21T22:15:27.405+0000 7fe9432ad640 -1 osd.1 0 waiting for initial osdmap 2026-02-21T22:15:27.412 INFO:tasks.ceph.osd.2.trial102.stderr:2026-02-21T22:15:27.405+0000 7ff62d873640 -1 osd.2 0 waiting for initial osdmap 2026-02-21T22:15:27.412 INFO:tasks.ceph.osd.0.trial102.stderr:2026-02-21T22:15:27.405+0000 7fbdcbae2640 -1 osd.0 0 waiting for initial osdmap 2026-02-21T22:15:27.413 INFO:tasks.ceph.osd.5.trial178.stderr:2026-02-21T22:15:27.413+0000 7fc2f9b8a640 -1 osd.5 0 waiting for initial osdmap 2026-02-21T22:15:27.414 INFO:tasks.ceph.osd.3.trial178.stderr:2026-02-21T22:15:27.413+0000 7f8ee8fbb640 -1 osd.3 0 waiting for initial osdmap 2026-02-21T22:15:27.416 INFO:tasks.ceph.osd.0.trial102.stderr:2026-02-21T22:15:27.409+0000 7fbdc69c9640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:27.416 INFO:tasks.ceph.osd.2.trial102.stderr:2026-02-21T22:15:27.409+0000 7ff62875a640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:27.416 INFO:tasks.ceph.osd.1.trial102.stderr:2026-02-21T22:15:27.409+0000 7fe93e194640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:27.417 INFO:tasks.ceph.osd.5.trial178.stderr:2026-02-21T22:15:27.417+0000 7fc2f5272640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:27.417 INFO:tasks.ceph.osd.3.trial178.stderr:2026-02-21T22:15:27.417+0000 7f8ee3ea2640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:27.525 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:not enough osds to create mgr pool 2026-02-21T22:15:27.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 2 seconds 2026-02-21T22:15:28.418 INFO:tasks.ceph.osd.4.trial178.stderr:2026-02-21T22:15:28.417+0000 7f6971690640 -1 osd.4 0 waiting for initial osdmap 2026-02-21T22:15:28.422 INFO:tasks.ceph.osd.4.trial178.stderr:2026-02-21T22:15:28.421+0000 7f696cd78640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T22:15:28.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:28.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:29.525 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:creating mgr pool 2026-02-21T22:15:29.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:creating new mgr pool 2026-02-21T22:15:30.423 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:mon_command: 'osd pool create' -> 0 in 0.898s 2026-02-21T22:15:31.427 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:mon_command: 'osd pool application enable' -> 0 in 1.004s 2026-02-21T22:15:31.427 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:using uri file:///.mgr:devicehealth/main.db?vfs=ceph 2026-02-21T22:15:31.434 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:new libcephsqlite addrv = 10.20.193.178:0/1615536903 2026-02-21T22:15:31.440 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:creating main.db for devicehealth 2026-02-21T22:15:31.445 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:15:31.459 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:finished reading legacy pool, complete = True 2026-02-21T22:15:31.459 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:get_kv('last_scrape') 2026-02-21T22:15:31.461 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Last scrape None 2026-02-21T22:15:31.461 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Check health 2026-02-21T22:15:31.464 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Last scrape never, next scrape due 20260221-221531 2026-02-21T22:15:31.464 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.0 2026-02-21T22:15:31.503 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:created device MTFDKCC1T9TGP-1BK1JABYY_012510B47DB8 2026-02-21T22:15:31.519 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:updating MTFDKCC1T9TGP-1BK1JABYY_012510B47DB8 wear level to 0.0 2026-02-21T22:15:31.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.1 2026-02-21T22:15:31.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:skipping duplicate MTFDKCC1T9TGP-1BK1JABYY_012510B47DB8 2026-02-21T22:15:31.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.2 2026-02-21T22:15:31.613 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:created device MTFDKCC1T9TGP-1BK1JABYY_012510B47B0E 2026-02-21T22:15:31.634 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:updating MTFDKCC1T9TGP-1BK1JABYY_012510B47B0E wear level to 0.0 2026-02-21T22:15:31.636 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.3 2026-02-21T22:15:31.677 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:created device MTFDKCC1T9TGP-1BK1JABYY_012510B002B9 2026-02-21T22:15:31.699 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:updating MTFDKCC1T9TGP-1BK1JABYY_012510B002B9 wear level to 0.0 2026-02-21T22:15:31.702 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.4 2026-02-21T22:15:31.736 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:skipping duplicate MTFDKCC1T9TGP-1BK1JABYY_012510B002B9 2026-02-21T22:15:31.736 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon osd.5 2026-02-21T22:15:31.770 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:created device MTFDKCC1T9TGP-1BK1JABYY_012510B002C3 2026-02-21T22:15:31.780 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:updating MTFDKCC1T9TGP-1BK1JABYY_012510B002C3 wear level to 0.0 2026-02-21T22:15:31.782 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon mon.a 2026-02-21T22:15:31.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:skipping duplicate MTFDKCC1T9TGP-1BK1JABYY_012510B47DB8 2026-02-21T22:15:31.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon mon.b 2026-02-21T22:15:31.847 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:skipping duplicate MTFDKCC1T9TGP-1BK1JABYY_012510B002B9 2026-02-21T22:15:31.847 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_scrape_daemon mon.c 2026-02-21T22:15:31.880 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:skipping duplicate MTFDKCC1T9TGP-1BK1JABYY_012510B47DB8 2026-02-21T22:15:31.880 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:set_kv('last_scrape', '20260221-221531') 2026-02-21T22:15:31.898 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 600 seconds 2026-02-21T22:15:32.831 DEBUG:teuthology.orchestra.run.trial102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: get_monmap_and_config 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: build_initial_monmap 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: monmap: 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:epoch 0 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:32.887273+0000 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:32.887273+0000 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:32.887 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.888 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:32.888 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:32.888 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:32.888 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 1 Processor -- start 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 1 -- start start 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: init 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-a con 0x7f95d808c370 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-c con 0x7f95d808ce50 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-b con 0x7f95d8183560 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d808d680 con 0x7f95d808c370 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8085a60 con 0x7f95d808ce50 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d808dee0 con 0x7f95d8183560 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95df0a3640 10 monclient(hunting): authenticate will time out at 1142.123169s 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95dce18640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f95d808c370 0x7f95d808c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58206/0 (socket says 10.20.193.102:58206) 2026-02-21T22:15:32.889 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.881+0000 7f95dce18640 1 -- 10.20.193.102:0/2465681454 learned_addr learned my addr 10.20.193.102:0/2465681454 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95dce18640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.102:6789/0 conn(0x7f95d808c370 legacy=0x7f95d808c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95dd619640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.102:6790/0 conn(0x7f95d8183560 legacy=0x7f95d81a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010386227 0 0) 0x7f95d808d680 con 0x7f95d808c370 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:32.891 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): my global_id is 4227 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95c4003620 con 0x7f95d808c370 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901324471 0 0) 0x7f95d808dee0 con 0x7f95d8183560 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient(hunting): my global_id is 4205 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d808d680 con 0x7f95d8183560 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cffff640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.178:6789/0 conn(0x7f95d808ce50 legacy=0x7f95d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747949849 0 0) 0x7f95c4003620 con 0x7f95d808c370 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.102:6790/0 conn(0x7f95d8183560 legacy=0x7f95d81a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.178:6789/0 conn(0x7f95d808ce50 legacy=0x7f95d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 monclient: found mon.noname-a 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d80857a0 con 0x7f95d808c370 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:32.892 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f95c8003110 con 0x7f95d808c370 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 5 monclient: authenticate success, global_id 4227 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: dump: 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f95c80033e0 con 0x7f95d808c370 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:32.893 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 1 -- 10.20.193.102:0/2465681454 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f95c8004b50 con 0x7f95d808c370 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95cf7fe640 10 monclient: dump: 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 4 set_mon_vals no callback set 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: shutdown 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- 10.20.193.102:0/2465681454 >> v1:10.20.193.102:6789/0 conn(0x7f95d808c370 legacy=0x7f95d808c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- 10.20.193.102:0/2465681454 shutdown_connections 2026-02-21T22:15:32.894 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- 10.20.193.102:0/2465681454 >> 10.20.193.102:0/2465681454 conn(0x7f95d80917a0 msgr2=0x7f95d8091ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:32.895 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- 10.20.193.102:0/2465681454 shutdown_connections 2026-02-21T22:15:32.895 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- 10.20.193.102:0/2465681454 wait complete. 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: build_initial_monmap 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: monmap: 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:epoch 0 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:created 0.000000 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 1 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 Processor -- start 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- start start 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: init 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-c con 0x7f95d808c370 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-a con 0x7f95d808ce50 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient(hunting): picked mon.noname-b con 0x7f95d8183560 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:32.897 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d80802a0 con 0x7f95d808ce50 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8080b20 con 0x7f95d8183560 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d8080e00 con 0x7f95d808c370 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95df0a3640 10 monclient(hunting): authenticate will time out at 1142.127826s 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95dce18640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f95d808c370 0x7f95d81411a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59990/0 (socket says 10.20.193.102:59990) 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.885+0000 7f95dce18640 1 -- 10.20.193.102:0/4083469124 learned_addr learned my addr 10.20.193.102:0/4083469124 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95dce18640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.102:6790/0 conn(0x7f95d808c370 legacy=0x7f95d81411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cffff640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.102:6789/0 conn(0x7f95d808ce50 legacy=0x7f95d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959441319 0 0) 0x7f95d8080e00 con 0x7f95d808c370 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): my global_id is 4208 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a8003620 con 0x7f95d808c370 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447949956 0 0) 0x7f95d80802a0 con 0x7f95d808ce50 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): my global_id is 4230 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d8080e00 con 0x7f95d808ce50 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95dd619640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.178:6789/0 conn(0x7f95d8183560 legacy=0x7f95d8123930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4010705209 0 0) 0x7f95a8003620 con 0x7f95d808c370 2026-02-21T22:15:32.898 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f95d80802a0 con 0x7f95d808c370 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f95c8002b70 con 0x7f95d808c370 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): dump: 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1352439947 0 0) 0x7f95d8080e00 con 0x7f95d808ce50 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f95a8003620 con 0x7f95d808ce50 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f95c0003380 con 0x7f95d808ce50 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): dump: 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:32.899 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614052580 0 0) 0x7f95d8080b20 con 0x7f95d8183560 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient(hunting): my global_id is 4201 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95d8080e00 con 0x7f95d8183560 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4016910053 0 0) 0x7f95a8003620 con 0x7f95d808ce50 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.102:6790/0 conn(0x7f95d808c370 legacy=0x7f95d81411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.178:6789/0 conn(0x7f95d8183560 legacy=0x7f95d8123930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 monclient: found mon.a 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.900 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d805df50 con 0x7f95d808ce50 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 5 monclient: authenticate success, global_id 4230 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _renew_subs 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f95d8088990 con 0x7f95d808ce50 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _renew_subs 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f95d8080840 con 0x7f95d808ce50 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f95c00039c0 con 0x7f95d808ce50 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f95c0003d20 con 0x7f95d808ce50 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 10 monclient: dump: 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:epoch 1 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:election_strategy: 3 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:32.901 INFO:teuthology.misc.health.trial102.stderr: 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95ceffd640 4 set_mon_vals no callback set 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d80008e0 con 0x7f95d808ce50 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f95c0003ec0 con 0x7f95d808ce50 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(15..15 src has 1..15) ==== 3147+0+0 (unknown 3885361510 0 0) 0x7f95c004c310 con 0x7f95d808ce50 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.889+0000 7f95dce18640 10 monclient: get_auth_request con 0x7f95a803d800 auth_method 0 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.893+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f95c0017560 con 0x7f95d808ce50 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.893+0000 7f95cd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:32.902 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.893+0000 7f95cd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95d8090810 con 0x7f95d808ce50 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95cd7fa640 1 -- 10.20.193.102:0/4083469124 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+9080 (unknown 4221056075 0 1954999753) 0x7f95c001c880 con 0x7f95d808ce50 2026-02-21T22:15:32.984 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95cd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:32.985 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95cd7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:32.985 INFO:teuthology.misc.health.trial102.stdout: 2026-02-21T22:15:32.985 INFO:teuthology.misc.health.trial102.stdout:{"epoch":15,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:32.425421+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.178:6800/2549949439 conn(0x7f95a803d800 legacy=0x7f95a805dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 10 monclient: shutdown 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 >> v1:10.20.193.102:6789/0 conn(0x7f95d808ce50 legacy=0x7f95d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 shutdown_connections 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 >> 10.20.193.102:0/4083469124 conn(0x7f95d80917a0 msgr2=0x7f95d808e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:32.986 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 shutdown_connections 2026-02-21T22:15:32.987 INFO:teuthology.misc.health.trial102.stderr:2026-02-21T22:15:32.977+0000 7f95df0a3640 1 -- 10.20.193.102:0/4083469124 wait complete. 2026-02-21T22:15:32.993 DEBUG:teuthology.misc:6 of 6 OSDs are up 2026-02-21T22:15:32.993 INFO:tasks.ceph:Creating RBD pool 2026-02-21T22:15:32.994 DEBUG:teuthology.orchestra.run.trial102:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T22:15:33.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: get_monmap_and_config 2026-02-21T22:15:33.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: build_initial_monmap 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: monmap: 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:33.055333+0000 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:33.055333+0000 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:33.057 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 1 Processor -- start 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 1 -- start start 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: init 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-c con 0x7f1200142ea0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-a con 0x7f120005e010 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-b con 0x7f12001854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120005e490 con 0x7f120005e010 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1200142a50 con 0x7f1200142ea0 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120008e700 con 0x7f12001854b0 2026-02-21T22:15:33.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:33.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f12081fb640 10 monclient(hunting): authenticate will time out at 1142.291237s 2026-02-21T22:15:33.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f120576f640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f120005e010 0x7f1200183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58230/0 (socket says 10.20.193.102:58230) 2026-02-21T22:15:33.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.049+0000 7f120576f640 1 -- 10.20.193.102:0/2731763811 learned_addr learned my addr 10.20.193.102:0/2731763811 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f120576f640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.102:6789/0 conn(0x7f120005e010 legacy=0x7f1200183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1206771640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.102:6790/0 conn(0x7f12001854b0 legacy=0x7f12001a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182802542 0 0) 0x7f120005e490 con 0x7f120005e010 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): my global_id is 4233 2026-02-21T22:15:33.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11e8003620 con 0x7f120005e010 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1205f70640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.178:6789/0 conn(0x7f1200142ea0 legacy=0x7f1200143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405370417 0 0) 0x7f120008e700 con 0x7f12001854b0 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient(hunting): my global_id is 4211 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f120005e490 con 0x7f12001854b0 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3542630725 0 0) 0x7f11e8003620 con 0x7f120005e010 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.102:6790/0 conn(0x7f12001854b0 legacy=0x7f12001a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.178:6789/0 conn(0x7f1200142ea0 legacy=0x7f1200143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: _finish_hunting 0 2026-02-21T22:15:33.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 monclient: found mon.noname-a 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120005cdb0 con 0x7f120005e010 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: _finish_auth 0 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: _check_auth_tickets 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f11f4003110 con 0x7f120005e010 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: dump: 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.064 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 5 monclient: authenticate success, global_id 4233 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f11f40033e0 con 0x7f120005e010 2026-02-21T22:15:33.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 1 -- 10.20.193.102:0/2731763811 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f11f4004b50 con 0x7f120005e010 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f1204f6e640 10 monclient: dump: 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.066 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 4 set_mon_vals no callback set 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: shutdown 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- 10.20.193.102:0/2731763811 >> v1:10.20.193.102:6789/0 conn(0x7f120005e010 legacy=0x7f1200183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- 10.20.193.102:0/2731763811 shutdown_connections 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- 10.20.193.102:0/2731763811 >> 10.20.193.102:0/2731763811 conn(0x7f1200054f80 msgr2=0x7f1200091f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- 10.20.193.102:0/2731763811 shutdown_connections 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- 10.20.193.102:0/2731763811 wait complete. 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: build_initial_monmap 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: monmap: 2026-02-21T22:15:33.067 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 Processor -- start 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- start start 2026-02-21T22:15:33.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: init 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-b con 0x7f120005e010 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-a con 0x7f1200142ea0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient(hunting): picked mon.noname-c con 0x7f12001854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120010ff50 con 0x7f1200142ea0 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120010f0e0 con 0x7f120005e010 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f120010dbf0 con 0x7f12001854b0 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f12081fb640 10 monclient(hunting): authenticate will time out at 1142.295694s 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f120576f640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1200142ea0 0x7f12001401a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58246/0 (socket says 10.20.193.102:58246) 2026-02-21T22:15:33.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.053+0000 7f120576f640 1 -- 10.20.193.102:0/1178894580 learned_addr learned my addr 10.20.193.102:0/1178894580 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:33.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f120576f640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.102:6789/0 conn(0x7f1200142ea0 legacy=0x7f12001401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f1206771640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.102:6790/0 conn(0x7f12001854b0 legacy=0x7f1200184b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145136440 0 0) 0x7f120010ff50 con 0x7f1200142ea0 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): my global_id is 4236 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11d0003620 con 0x7f1200142ea0 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222763371 0 0) 0x7f120010dbf0 con 0x7f12001854b0 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 10 monclient(hunting): my global_id is 4214 2026-02-21T22:15:33.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f120010ff50 con 0x7f12001854b0 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f1205f70640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.178:6789/0 conn(0x7f120005e010 legacy=0x7f120013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.057+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 433181455 0 0) 0x7f11d0003620 con 0x7f1200142ea0 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f120010dbf0 con 0x7f1200142ea0 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3677668115 0 0) 0x7f120010ff50 con 0x7f12001854b0 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f11d0003620 con 0x7f12001854b0 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694980148 0 0) 0x7f120010f0e0 con 0x7f120005e010 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): my global_id is 4207 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f120010ff50 con 0x7f120005e010 2026-02-21T22:15:33.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f11f4002b70 con 0x7f1200142ea0 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): dump: 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.076 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f11fc004930 con 0x7f12001854b0 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient(hunting): dump: 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.077 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2456988707 0 0) 0x7f120010dbf0 con 0x7f1200142ea0 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.102:6790/0 conn(0x7f12001854b0 legacy=0x7f1200184b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.178:6789/0 conn(0x7f120005e010 legacy=0x7f120013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 monclient: found mon.a 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120010f6b0 con 0x7f1200142ea0 2026-02-21T22:15:33.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 5 monclient: authenticate success, global_id 4236 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _renew_subs 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f120010d690 con 0x7f1200142ea0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _renew_subs 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f120010e750 con 0x7f1200142ea0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f11f40033c0 con 0x7f1200142ea0 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:33.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f11f40052e0 con 0x7f1200142ea0 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 10 monclient: dump: 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.080 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11effff640 4 set_mon_vals no callback set 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12000008e0 con 0x7f1200142ea0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f11f4011970 con 0x7f1200142ea0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(15..15 src has 1..15) ==== 3147+0+0 (unknown 3885361510 0 0) 0x7f11f404c3d0 con 0x7f1200142ea0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.061+0000 7f1205f70640 10 monclient: get_auth_request con 0x7f11d003d870 auth_method 0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.065+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f11f4017740 con 0x7f1200142ea0 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.065+0000 7f11ee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:33.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.065+0000 7f11ee7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:33.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.145+0000 7f12081fb640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-21T22:15:33.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.145+0000 7f12081fb640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-21T22:15:33.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.145+0000 7f12081fb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:33.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.145+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f1200055470 con 0x7f1200142ea0 2026-02-21T22:15:33.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.517+0000 7f11ee7fc640 1 -- 10.20.193.102:0/1178894580 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) ==== 109+0+0 (unknown 231814880 0 0) 0x7f11f4017020 con 0x7f1200142ea0 2026-02-21T22:15:33.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.517+0000 7f11ee7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-21T22:15:33.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.517+0000 7f11ee7fc640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-21T22:15:33.525 INFO:teuthology.orchestra.run.trial102.stderr:pool 'rbd' created 2026-02-21T22:15:33.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.178:6800/2549949439 conn(0x7f11d003d870 legacy=0x7f11d005dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 10 monclient: shutdown 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 >> v1:10.20.193.102:6789/0 conn(0x7f1200142ea0 legacy=0x7f12001401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 shutdown_connections 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 >> 10.20.193.102:0/1178894580 conn(0x7f1200054f80 msgr2=0x7f1200142230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 shutdown_connections 2026-02-21T22:15:33.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.521+0000 7f12081fb640 1 -- 10.20.193.102:0/1178894580 wait complete. 2026-02-21T22:15:33.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:33.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:33.537 DEBUG:teuthology.orchestra.run.trial102:> rbd --cluster ceph pool init rbd 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 10 monclient: get_monmap_and_config 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 10 monclient: build_initial_monmap 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 10 monclient: monmap: 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:33.552031+0000 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:33.552031+0000 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:33.552 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 1 Processor -- start 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 1 -- start start 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 10 monclient: init 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.545+0000 7f5021206200 10 monclient: _reopen_session rank -1 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-b con 0x55b296c9b500 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-c con 0x55b296c98fd0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-a con 0x55b296c9a530 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296b24630 con 0x55b296c9a530 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296b0b570 con 0x55b296c98fd0 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296b0b020 con 0x55b296c9b500 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): _renew_subs 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): authenticate will time out at 1142.788070s 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501fcdd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x55b296c9b500 0x55b296c9bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60010/0 (socket says 10.20.193.102:60010) 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5020f66640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x55b296c9a530 0x55b296db0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58262/0 (socket says 10.20.193.102:58262) 2026-02-21T22:15:33.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501fcdd640 1 -- 10.20.193.102:0/1349595507 learned_addr learned my addr 10.20.193.102:0/1349595507 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501fcdd640 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.102:6790/0 conn(0x55b296c9b500 legacy=0x55b296c9bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5020f66640 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.102:6789/0 conn(0x55b296c9a530 legacy=0x55b296db0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501f4dc640 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.178:6789/0 conn(0x55b296c98fd0 legacy=0x55b296c9c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553629528 0 0) 0x55b296b0b020 con 0x55b296c9b500 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): my global_id is 4217 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f500c001e20 con 0x55b296c9b500 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454884418 0 0) 0x55b296b24630 con 0x55b296c9a530 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): my global_id is 4239 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55b296b0b020 con 0x55b296c9a530 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014012703 0 0) 0x55b296b0b570 con 0x55b296c98fd0 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient(hunting): my global_id is 4210 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55b296b24630 con 0x55b296c98fd0 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 209641493 0 0) 0x7f500c001e20 con 0x55b296c9b500 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.178:6789/0 conn(0x55b296c98fd0 legacy=0x55b296c9c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.102:6789/0 conn(0x55b296c9a530 legacy=0x55b296db0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 monclient: found mon.noname-b 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b296ad1690 con 0x55b296c9b500 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: _finish_auth 0 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 5 monclient: authenticate success, global_id 4217 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5014001910 con 0x55b296c9b500 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: dump: 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5014001be0 con 0x55b296c9b500 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 1 -- 10.20.193.102:0/1349595507 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5014003350 con 0x55b296c9b500 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f501ecdb640 10 monclient: dump: 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.557 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: get_monmap_and_config success 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 4 set_mon_vals no callback set 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: shutdown 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- 10.20.193.102:0/1349595507 >> v1:10.20.193.102:6790/0 conn(0x55b296c9b500 legacy=0x55b296c9bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- 10.20.193.102:0/1349595507 shutdown_connections 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- 10.20.193.102:0/1349595507 >> 10.20.193.102:0/1349595507 conn(0x55b296c9ea00 msgr2=0x55b296ca1580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- 10.20.193.102:0/1349595507 shutdown_connections 2026-02-21T22:15:33.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- 10.20.193.102:0/1349595507 wait complete. 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: build_initial_monmap 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: monmap: 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 Processor -- start 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- start start 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: init 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: _reopen_session rank -1 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-a con 0x55b296c98fd0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-c con 0x55b296c9a530 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:33.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 10 monclient(hunting): picked mon.noname-b con 0x55b296c9b500 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.549+0000 7f5021206200 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296c98700 con 0x55b296c98fd0 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296d19150 con 0x55b296c9b500 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55b296d340b0 con 0x55b296c9a530 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient(hunting): _renew_subs 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient(hunting): authenticate will time out at 1142.792008s 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f501fcdd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x55b296c98fd0 0x55b296d300d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58268/0 (socket says 10.20.193.102:58268) 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f501fcdd640 1 -- 10.20.193.102:0/1214669133 learned_addr learned my addr 10.20.193.102:0/1214669133 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f501fcdd640 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.102:6789/0 conn(0x55b296c98fd0 legacy=0x55b296d300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f501f4dc640 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.102:6790/0 conn(0x55b296c9a530 legacy=0x55b296d9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.178:6789/0 conn(0x55b296c9b500 legacy=0x55b296dbff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030876346 0 0) 0x55b296d340b0 con 0x55b296c9a530 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): my global_id is 4220 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5000001e20 con 0x55b296c9a530 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430947325 0 0) 0x55b296ad1690 con 0x55b296c98fd0 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): my global_id is 4242 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55b296d340b0 con 0x55b296c98fd0 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982677176 0 0) 0x55b296d19150 con 0x55b296c9b500 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): my global_id is 4213 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x55b296ad1690 con 0x55b296c9b500 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1324162861 0 0) 0x7f5000001e20 con 0x55b296c9a530 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x55b296d19150 con 0x55b296c9a530 2026-02-21T22:15:33.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5008001bc0 con 0x55b296c9a530 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): dump: 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3396647558 0 0) 0x55b296d340b0 con 0x55b296c98fd0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5000001e20 con 0x55b296c98fd0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f50140030e0 con 0x55b296c98fd0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): dump: 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 261383211 0 0) 0x55b296ad1690 con 0x55b296c9b500 2026-02-21T22:15:33.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x55b296d340b0 con 0x55b296c9b500 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f50100030d0 con 0x55b296c9b500 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient(hunting): dump: 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2350191659 0 0) 0x55b296d19150 con 0x55b296c9a530 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.178:6789/0 conn(0x55b296c9b500 legacy=0x55b296dbff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.102:6789/0 conn(0x55b296c98fd0 legacy=0x55b296d300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 monclient: found mon.c 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b296d19710 con 0x55b296c9a530 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: _finish_auth 0 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 5 monclient: authenticate success, global_id 4220 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _renew_subs 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x55b296d1ba70 con 0x55b296c9a530 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _renew_subs 2026-02-21T22:15:33.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x55b296d307e0 con 0x55b296c9a530 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f50080021c0 con 0x55b296c9a530 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5008002520 con 0x55b296c9a530 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 10 monclient: dump: 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f501e4da640 4 set_mon_vals no callback set 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5008002780 con 0x55b296c9a530 2026-02-21T22:15:33.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(16..16 src has 1..16) ==== 3492+0+0 (unknown 1207316518 0 0) 0x7f500804acd0 con 0x55b296c9a530 2026-02-21T22:15:33.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-21T22:15:33.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-21T22:15:33.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:33.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.553+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55b296d18370 con 0x55b296c9a530 2026-02-21T22:15:33.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:33.557+0000 7f501fcdd640 10 monclient: get_auth_request con 0x7f500003c1f0 auth_method 0 2026-02-21T22:15:34.412 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:15:34.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v17) ==== 141+0+0 (unknown 1851909647 0 0) 0x7f5008015e20 con 0x55b296c9a530 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5021206200 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f5014001be0 con 0x55b296c9a530 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (unknown 391748328 0 0) 0x7f5008002aa0 con 0x55b296c9a530 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 10 monclient: handle_get_version_reply finishing 1 version 17 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f501e4da640 10 monclient: _renew_subs 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f501e4da640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f501e4da640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=17}) -- 0x7f4ff8002af0 con 0x55b296c9a530 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 11 ==== osd_map(17..17 src has 1..17) ==== 661+0+0 (unknown 330623341 0 0) 0x7f5008020070 con 0x55b296c9a530 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6801/2402357529 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) -- 0x55b296de3b00 con 0x55b296dc2000 2026-02-21T22:15:34.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.525+0000 7f5020f66640 10 monclient: get_auth_request con 0x55b296dc2000 auth_method 0 2026-02-21T22:15:34.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.529+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 <== osd.5 v1:10.20.193.178:6801/2402357529 1 ==== osd_op_reply(1 rbd_trash [create] v17'1 uv1 ondisk = 0) ==== 153+0+0 (unknown 1703302865 0 0) 0x7f5010005700 con 0x55b296dc2000 2026-02-21T22:15:34.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6801/2402357529 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) -- 0x55b296e1f570 con 0x55b296dc2000 2026-02-21T22:15:34.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 <== osd.5 v1:10.20.193.178:6801/2402357529 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 4274215525 0 0) 0x7f5010005700 con 0x55b296dc2000 2026-02-21T22:15:34.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6801/2402357529 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17) -- 0x55b296e1e680 con 0x55b296dc2000 2026-02-21T22:15:34.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 <== osd.5 v1:10.20.193.178:6801/2402357529 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (unknown 4287864937 0 0) 0x7f5010005700 con 0x55b296dc2000 2026-02-21T22:15:34.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f501dcd9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:34.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:34.533+0000 7f501dcd9640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f4ffc001f50 con 0x55b296c9a530 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 12 ==== pool_op_reply(tid 4 (0) Success v18) ==== 55+0+0 (unknown 955820729 0 0) 0x7f500801b0e0 con 0x55b296c9a530 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f5007fff640 10 monclient: _renew_subs 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f5007fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=18}) -- 0x55b296d19150 con 0x55b296c9a530 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 13 ==== osd_map(18..18 src has 1..18) ==== 629+0+0 (unknown 1725346916 0 0) 0x7f50080160b0 con 0x55b296c9a530 2026-02-21T22:15:35.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.525+0000 7f501e4da640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6801/2402357529 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) -- 0x7f4ff8005860 con 0x55b296dc2000 2026-02-21T22:15:35.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.529+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 <== osd.5 v1:10.20.193.178:6801/2402357529 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v18'1 uv1 ondisk = 0) ==== 194+0+0 (unknown 2910594948 0 0) 0x7f5010005700 con 0x55b296dc2000 2026-02-21T22:15:35.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.529+0000 7f501dcd9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:35.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:35.529+0000 7f501dcd9640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- pool_op(delete unmanaged snap pool 2 tid 6 name v18) -- 0x7f4ffc0033c0 con 0x55b296c9a530 2026-02-21T22:15:36.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 14 ==== pool_op_reply(tid 6 (0) Success v19) ==== 43+0+0 (unknown 3613622959 0 0) 0x7f500801d6c0 con 0x55b296c9a530 2026-02-21T22:15:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f5007fff640 10 monclient: _renew_subs 2026-02-21T22:15:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f5007fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=19}) -- 0x55b296ad1690 con 0x55b296c9a530 2026-02-21T22:15:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f5007fff640 1 -- 10.20.193.102:0/1214669133 <== mon.2 v1:10.20.193.102:6790/0 15 ==== osd_map(19..19 src has 1..19) ==== 657+0+0 (unknown 3028789517 0 0) 0x7f50080162b0 con 0x55b296c9a530 2026-02-21T22:15:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.533+0000 7f501e4da640 1 -- 10.20.193.102:0/1214669133 --> v1:10.20.193.178:6801/2402357529 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19) -- 0x7f4ff8005c40 con 0x55b296dc2000 2026-02-21T22:15:36.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 <== osd.5 v1:10.20.193.178:6801/2402357529 5 ==== osd_op_reply(7 rbd_info [write 0~19] v19'2 uv2 ondisk = 0) ==== 152+0+0 (unknown 900577457 0 0) 0x7f5010005700 con 0x55b296dc2000 2026-02-21T22:15:36.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.178:6801/2402357529 conn(0x55b296dc2000 legacy=0x55b296de23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.178:6800/2549949439 conn(0x7f500003c1f0 legacy=0x7f500005c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 10 monclient: shutdown 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 >> v1:10.20.193.102:6790/0 conn(0x55b296c9a530 legacy=0x55b296d9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5020f66640 1 -- 10.20.193.102:0/1214669133 reap_dead start 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 shutdown_connections 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 >> 10.20.193.102:0/1214669133 conn(0x55b296c9ea00 msgr2=0x55b296c919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 shutdown_connections 2026-02-21T22:15:36.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.545+0000 7f5021206200 1 -- 10.20.193.102:0/1214669133 wait complete. 2026-02-21T22:15:36.554 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T22:15:36.555 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T22:15:36.556 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T22:15:36.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 10 monclient: get_monmap_and_config 2026-02-21T22:15:36.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 10 monclient: build_initial_monmap 2026-02-21T22:15:36.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 10 monclient: monmap: 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:36.656327+0000 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:36.656327+0000 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 1 Processor -- start 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 1 -- start start 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.649+0000 7fcab1456640 10 monclient: init 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-a con 0x7fcaac08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-c con 0x7fcaac08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-b con 0x7fcaac08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac086a10 con 0x7fcaac08ce00 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac08b130 con 0x7fcaac08b7f0 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac08bd60 con 0x7fcaac08d8e0 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:36.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient(hunting): authenticate will time out at 1145.892230s 2026-02-21T22:15:36.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaaaffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 0x7fcaac08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58294/0 (socket says 10.20.193.102:58294) 2026-02-21T22:15:36.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaaaffd640 1 -- 10.20.193.102:0/83047654 learned_addr learned my addr 10.20.193.102:0/83047654 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:36.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaaaffd640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 legacy=0x7fcaac08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaab7fe640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.102:6790/0 conn(0x7fcaac08d8e0 legacy=0x7fcaac1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070037551 0 0) 0x7fcaac086a10 con 0x7fcaac08ce00 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): my global_id is 4245 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca98003620 con 0x7fcaac08ce00 2026-02-21T22:15:36.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666056526 0 0) 0x7fcaac08bd60 con 0x7fcaac08d8e0 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient(hunting): my global_id is 4223 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaaa7fc640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.178:6789/0 conn(0x7fcaac08b7f0 legacy=0x7fcaac181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaac086a10 con 0x7fcaac08d8e0 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2668030244 0 0) 0x7fca98003620 con 0x7fcaac08ce00 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.102:6790/0 conn(0x7fcaac08d8e0 legacy=0x7fcaac1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.178:6789/0 conn(0x7fcaac08b7f0 legacy=0x7fcaac181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 monclient: found mon.noname-a 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac08b370 con 0x7fcaac08ce00 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 5 monclient: authenticate success, global_id 4245 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcaa0003110 con 0x7fcaac08ce00 2026-02-21T22:15:36.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: dump: 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.670 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcaa00033e0 con 0x7fcaac08ce00 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 1 -- 10.20.193.102:0/83047654 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcaa0004b50 con 0x7fcaac08ce00 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcaa9ffb640 10 monclient: dump: 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.671 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 4 set_mon_vals no callback set 2026-02-21T22:15:36.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 10 monclient: shutdown 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- 10.20.193.102:0/83047654 >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 legacy=0x7fcaac08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- 10.20.193.102:0/83047654 shutdown_connections 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- 10.20.193.102:0/83047654 >> 10.20.193.102:0/83047654 conn(0x7fcaac091a50 msgr2=0x7fcaac091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- 10.20.193.102:0/83047654 shutdown_connections 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.653+0000 7fcab1456640 1 -- 10.20.193.102:0/83047654 wait complete. 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: build_initial_monmap 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: monmap: 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:36.673 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 Processor -- start 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- start start 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: init 2026-02-21T22:15:36.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-c con 0x7fcaac08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-a con 0x7fcaac08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient(hunting): picked mon.noname-b con 0x7fcaac08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac083190 con 0x7fcaac08ce00 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac153eb0 con 0x7fcaac08d8e0 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac088a70 con 0x7fcaac08b7f0 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient(hunting): authenticate will time out at 1145.896833s 2026-02-21T22:15:36.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaaa7fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 0x7fcaac0846c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58308/0 (socket says 10.20.193.102:58308) 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaaa7fc640 1 -- 10.20.193.102:0/3345272162 learned_addr learned my addr 10.20.193.102:0/3345272162 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaaa7fc640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 legacy=0x7fcaac0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaaaffd640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.102:6790/0 conn(0x7fcaac08b7f0 legacy=0x7fcaac152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaab7fe640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.178:6789/0 conn(0x7fcaac08d8e0 legacy=0x7fcaac195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3381445047 0 0) 0x7fcaac083190 con 0x7fcaac08ce00 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): my global_id is 4248 2026-02-21T22:15:36.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca78003620 con 0x7fcaac08ce00 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404698250 0 0) 0x7fcaac088a70 con 0x7fcaac08b7f0 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): my global_id is 4226 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaac083190 con 0x7fcaac08b7f0 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397494589 0 0) 0x7fcaac153eb0 con 0x7fcaac08d8e0 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): my global_id is 4219 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaac088a70 con 0x7fcaac08d8e0 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3262936883 0 0) 0x7fcaac083190 con 0x7fcaac08b7f0 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcaac153eb0 con 0x7fcaac08b7f0 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2782779389 0 0) 0x7fca78003620 con 0x7fcaac08ce00 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcaac083190 con 0x7fcaac08ce00 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fca94003400 con 0x7fcaac08ce00 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.678 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcaa0002b70 con 0x7fcaac08b7f0 2026-02-21T22:15:36.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.680 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4266820170 0 0) 0x7fcaac088a70 con 0x7fcaac08d8e0 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca78003620 con 0x7fcaac08d8e0 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fca9c004900 con 0x7fcaac08d8e0 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.681 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 480918921 0 0) 0x7fcaac153eb0 con 0x7fcaac08b7f0 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.178:6789/0 conn(0x7fcaac08d8e0 legacy=0x7fcaac195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.102:6789/0 conn(0x7fcaac08ce00 legacy=0x7fcaac0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:36.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 monclient: found mon.c 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac080e30 con 0x7fcaac08b7f0 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 5 monclient: authenticate success, global_id 4226 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _renew_subs 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcaac0828f0 con 0x7fcaac08b7f0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _renew_subs 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fcaac081990 con 0x7fcaac08b7f0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcaa0004a90 con 0x7fcaac08b7f0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcaa0005170 con 0x7fcaac08b7f0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fcaa97fa640 4 set_mon_vals no callback set 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.657+0000 7fca8b7fe640 10 monclient: dump: 2026-02-21T22:15:36.684 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fcab1456640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fcab1456640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fcab1456640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fcaa0011800 con 0x7fcaac08b7f0 2026-02-21T22:15:36.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaac0008e0 con 0x7fcaac08b7f0 2026-02-21T22:15:36.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fcaa004c430 con 0x7fcaac08b7f0 2026-02-21T22:15:36.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fcaaa7fc640 10 monclient: get_auth_request con 0x7fca7803d890 auth_method 0 2026-02-21T22:15:36.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fcaa0017600 con 0x7fcaac08b7f0 2026-02-21T22:15:36.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fca8b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.661+0000 7fca8b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:36.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 10 start_mon_command cmd=[{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-21T22:15:36.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 10 monclient: _send_command 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-21T22:15:36.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:36.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fcaac090810 con 0x7fcaac08b7f0 2026-02-21T22:15:36.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fca8b7fe640 1 -- 10.20.193.102:0/3345272162 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v1) ==== 86+0+61 (unknown 4269106227 0 102270264) 0x7fcaa0016ee0 con 0x7fcaac08b7f0 2026-02-21T22:15:36.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fca8b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config log", "num": 1, "format": "json"}] 2026-02-21T22:15:36.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fca8b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:36.756 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.178:6800/2549949439 conn(0x7fca7803d890 legacy=0x7fca7805dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 10 monclient: shutdown 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 >> v1:10.20.193.102:6790/0 conn(0x7fcaac08b7f0 legacy=0x7fcaac152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 shutdown_connections 2026-02-21T22:15:36.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 >> 10.20.193.102:0/3345272162 conn(0x7fcaac091a50 msgr2=0x7fcaac05c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:36.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 shutdown_connections 2026-02-21T22:15:36.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.749+0000 7fcab1456640 1 -- 10.20.193.102:0/3345272162 wait complete. 2026-02-21T22:15:36.765 INFO:teuthology.orchestra.run.trial102.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-21T22:15:36.766 INFO:tasks.ceph_manager:config epoch is 1 2026-02-21T22:15:36.766 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T22:15:36.766 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T22:15:36.766 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T22:15:36.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 10 monclient: get_monmap_and_config 2026-02-21T22:15:36.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 10 monclient: build_initial_monmap 2026-02-21T22:15:36.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 10 monclient: monmap: 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:36.828314+0000 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:36.828314+0000 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:36.829 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 1 Processor -- start 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 1 -- start start 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.821+0000 7fedd4f90640 10 monclient: init 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:36.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-c con 0x7fedd0091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-b con 0x7fedd00909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-a con 0x7fedd01854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0057a20 con 0x7fedd01854b0 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0091d20 con 0x7fedd0091240 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0057740 con 0x7fedd00909a0 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient(hunting): authenticate will time out at 1146.064471s 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcdd74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 0x7fedd0183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60036/0 (socket says 10.20.193.102:60036) 2026-02-21T22:15:36.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcdd74640 1 -- 10.20.193.102:0/1235842588 learned_addr learned my addr 10.20.193.102:0/1235842588 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:36.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedced76640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.102:6789/0 conn(0x7fedd01854b0 legacy=0x7fedd01a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcdd74640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 legacy=0x7fedd0183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedce575640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.178:6789/0 conn(0x7fedd0091240 legacy=0x7fedd0091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879460618 0 0) 0x7fedd0057a20 con 0x7fedd01854b0 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): my global_id is 4251 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedbc003620 con 0x7fedd01854b0 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847163213 0 0) 0x7fedd0057740 con 0x7fedd00909a0 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): my global_id is 4229 2026-02-21T22:15:36.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0057a20 con 0x7fedd00909a0 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492205228 0 0) 0x7fedd0091d20 con 0x7fedd0091240 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient(hunting): my global_id is 4222 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd0057740 con 0x7fedd0091240 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4037181238 0 0) 0x7fedbc003620 con 0x7fedd01854b0 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 legacy=0x7fedd0183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.178:6789/0 conn(0x7fedd0091240 legacy=0x7fedd0091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: _finish_hunting 0 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 monclient: found mon.noname-a 2026-02-21T22:15:36.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0090dc0 con 0x7fedd01854b0 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: _finish_auth 0 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: _check_auth_tickets 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fedc0004670 con 0x7fedd01854b0 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: dump: 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.843 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 5 monclient: authenticate success, global_id 4251 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:36.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fedc0004940 con 0x7fedd01854b0 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 1 -- 10.20.193.102:0/1235842588 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fedc0004dd0 con 0x7fedd01854b0 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedcd573640 10 monclient: dump: 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.845 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 4 set_mon_vals no callback set 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 10 monclient: shutdown 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- 10.20.193.102:0/1235842588 >> v1:10.20.193.102:6789/0 conn(0x7fedd01854b0 legacy=0x7fedd01a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- 10.20.193.102:0/1235842588 shutdown_connections 2026-02-21T22:15:36.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- 10.20.193.102:0/1235842588 >> 10.20.193.102:0/1235842588 conn(0x7fedd005e660 msgr2=0x7fedd0095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- 10.20.193.102:0/1235842588 shutdown_connections 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.825+0000 7fedd4f90640 1 -- 10.20.193.102:0/1235842588 wait complete. 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient: build_initial_monmap 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient: monmap: 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:36.847 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 1 Processor -- start 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 1 -- start start 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient: init 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-c con 0x7fedd00909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-a con 0x7fedd0091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient(hunting): picked mon.noname-b con 0x7fedd01854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:36.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd012bd50 con 0x7fedd0091240 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0112fa0 con 0x7fedd01854b0 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd0112130 con 0x7fedd00909a0 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedd4f90640 10 monclient(hunting): authenticate will time out at 1146.069264s 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedce575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 0x7fedd012b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60048/0 (socket says 10.20.193.102:60048) 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedce575640 1 -- 10.20.193.102:0/188196462 learned_addr learned my addr 10.20.193.102:0/188196462 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedcdd74640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.102:6789/0 conn(0x7fedd0091240 legacy=0x7fedd0172a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedce575640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 legacy=0x7fedd012b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452108277 0 0) 0x7fedd012bd50 con 0x7fedd0091240 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): my global_id is 4254 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c003620 con 0x7fedd0091240 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 502956092 0 0) 0x7fedd0112130 con 0x7fedd00909a0 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): my global_id is 4232 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedd012bd50 con 0x7fedd00909a0 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedced76640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.178:6789/0 conn(0x7fedd01854b0 legacy=0x7fedd0174120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2947622630 0 0) 0x7fed9c003620 con 0x7fedd0091240 2026-02-21T22:15:36.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0112130 con 0x7fedd0091240 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fedb80032c0 con 0x7fedd0091240 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): dump: 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.851 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655980910 0 0) 0x7fedd0112fa0 con 0x7fedd01854b0 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): my global_id is 4225 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c003620 con 0x7fedd01854b0 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2839140422 0 0) 0x7fedd012bd50 con 0x7fedd00909a0 2026-02-21T22:15:36.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedd0112fa0 con 0x7fedd00909a0 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fedc4003710 con 0x7fedd00909a0 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.829+0000 7fedb6ffd640 10 monclient(hunting): dump: 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.853 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1287489983 0 0) 0x7fedd0112130 con 0x7fedd0091240 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.102:6790/0 conn(0x7fedd00909a0 legacy=0x7fedd012b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.178:6789/0 conn(0x7fedd01854b0 legacy=0x7fedd0174120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 monclient: found mon.a 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0110c40 con 0x7fedd0091240 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:36.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 5 monclient: authenticate success, global_id 4254 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _renew_subs 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fedd0112700 con 0x7fedd0091240 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _renew_subs 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fedd01117a0 con 0x7fedd0091240 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fedb8004b90 con 0x7fedd0091240 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fedb8004ef0 con 0x7fedd0091240 2026-02-21T22:15:36.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedccd72640 4 set_mon_vals no callback set 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 10 monclient: dump: 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:36.856 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd00008e0 con 0x7fedd0091240 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fedb8011580 con 0x7fedd0091240 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fedb804c4e0 con 0x7fedd0091240 2026-02-21T22:15:36.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.833+0000 7fedce575640 10 monclient: get_auth_request con 0x7fed9c03d8d0 auth_method 0 2026-02-21T22:15:36.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.837+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fedb80175f0 con 0x7fedd0091240 2026-02-21T22:15:36.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.837+0000 7fedb6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:36.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.837+0000 7fedb6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:36.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedd4f90640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-21T22:15:36.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedd4f90640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-21T22:15:36.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedd4f90640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:36.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fedd0095010 con 0x7fedd0091240 2026-02-21T22:15:36.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedb6ffd640 1 -- 10.20.193.102:0/188196462 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+95498 (unknown 481731318 0 3355067984) 0x7fedb8016ed0 con 0x7fedd0091240 2026-02-21T22:15:36.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedb6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-21T22:15:36.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.937+0000 7fedb6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:36.946 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:36.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.178:6800/2549949439 conn(0x7fed9c03d8d0 legacy=0x7fed9c05dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 10 monclient: shutdown 2026-02-21T22:15:36.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:36.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 >> v1:10.20.193.102:6789/0 conn(0x7fedd0091240 legacy=0x7fedd0172a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:36.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 shutdown_connections 2026-02-21T22:15:36.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 >> 10.20.193.102:0/188196462 conn(0x7fedd005e660 msgr2=0x7fedd008b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:36.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 shutdown_connections 2026-02-21T22:15:36.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:36.941+0000 7fedd4f90640 1 -- 10.20.193.102:0/188196462 wait complete. 2026-02-21T22:15:36.955 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6800","nonce":2549949439}]},"active_addr":"10.20.193.178:6800/2549949439","active_change":"2026-02-21T22:15:23.478435+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.178:0","nonce":3156900188}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.178:0","nonce":1615536903}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.178:0","nonce":700003767}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.178:0","nonce":2797236064}]}]} 2026-02-21T22:15:36.962 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T22:15:36.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T22:15:36.962 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:37.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 10 monclient: monmap: 2026-02-21T22:15:37.032 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.032263+0000 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.032263+0000 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.033 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 1 Processor -- start 2026-02-21T22:15:37.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 1 -- start start 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.025+0000 7f2678ced640 10 monclient: init 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-c con 0x7f2674091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-a con 0x7f26740909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-b con 0x7f26741854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674057a20 con 0x7f26740909a0 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674091d20 con 0x7f2674091240 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674057740 con 0x7f26741854b0 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient(hunting): authenticate will time out at 1146.268158s 2026-02-21T22:15:37.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f26740909a0 0x7f2674183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58324/0 (socket says 10.20.193.102:58324) 2026-02-21T22:15:37.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671d74640 1 -- 10.20.193.102:0/1826289184 learned_addr learned my addr 10.20.193.102:0/1826289184 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671d74640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.102:6789/0 conn(0x7f26740909a0 legacy=0x7f2674183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2672d76640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.102:6790/0 conn(0x7f26741854b0 legacy=0x7f26741a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2672575640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.178:6789/0 conn(0x7f2674091240 legacy=0x7f2674091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419472509 0 0) 0x7f2674057a20 con 0x7f26740909a0 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): my global_id is 4257 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2660003620 con 0x7f26740909a0 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664261593 0 0) 0x7f2674057740 con 0x7f26741854b0 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): my global_id is 4235 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674057a20 con 0x7f26741854b0 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1721193492 0 0) 0x7f2674091d20 con 0x7f2674091240 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient(hunting): my global_id is 4228 2026-02-21T22:15:37.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674057740 con 0x7f2674091240 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030071072 0 0) 0x7f2674057a20 con 0x7f26741854b0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.178:6789/0 conn(0x7f2674091240 legacy=0x7f2674091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.102:6789/0 conn(0x7f26740909a0 legacy=0x7f2674183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674090dc0 con 0x7f26741854b0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2664004670 con 0x7f26741854b0 2026-02-21T22:15:37.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: dump: 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.046 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 5 monclient: authenticate success, global_id 4235 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2664004940 con 0x7f26741854b0 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 1 -- 10.20.193.102:0/1826289184 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2664004dd0 con 0x7f26741854b0 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2671573640 10 monclient: dump: 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.047 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 4 set_mon_vals no callback set 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: shutdown 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- 10.20.193.102:0/1826289184 >> v1:10.20.193.102:6790/0 conn(0x7f26741854b0 legacy=0x7f26741a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- 10.20.193.102:0/1826289184 shutdown_connections 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- 10.20.193.102:0/1826289184 >> 10.20.193.102:0/1826289184 conn(0x7f267405e660 msgr2=0x7f2674095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- 10.20.193.102:0/1826289184 shutdown_connections 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 -- 10.20.193.102:0/1826289184 wait complete. 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 10 monclient: monmap: 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.049 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.029+0000 7f2678ced640 1 Processor -- start 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- start start 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: init 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-c con 0x7f26740909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-a con 0x7f2674091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient(hunting): picked mon.noname-b con 0x7f26741854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267412bbb0 con 0x7f2674091240 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674112e00 con 0x7f26741854b0 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2674111f90 con 0x7f26740909a0 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient(hunting): authenticate will time out at 1146.272312s 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2671d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2674091240 0x7f2674172860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58326/0 (socket says 10.20.193.102:58326) 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2671d74640 1 -- 10.20.193.102:0/18011100 learned_addr learned my addr 10.20.193.102:0/18011100 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2672575640 1 --1- 10.20.193.102:0/18011100 >> v1:10.20.193.102:6790/0 conn(0x7f26740909a0 0x7f267412b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60066/0 (socket says 10.20.193.102:60066) 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2672575640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.102:6790/0 conn(0x7f26740909a0 legacy=0x7f267412b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2671d74640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.102:6789/0 conn(0x7f2674091240 legacy=0x7f2674172860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458795495 0 0) 0x7f2674111f90 con 0x7f26740909a0 2026-02-21T22:15:37.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): my global_id is 4238 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2640003620 con 0x7f26740909a0 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626017215 0 0) 0x7f267412bbb0 con 0x7f2674091240 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): my global_id is 4260 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674111f90 con 0x7f2674091240 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2672d76640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.178:6789/0 conn(0x7f26741854b0 legacy=0x7f2674173f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 443093832 0 0) 0x7f2640003620 con 0x7f26740909a0 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f267412bbb0 con 0x7f26740909a0 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26680033c0 con 0x7f26740909a0 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): dump: 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.053 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3657633644 0 0) 0x7f2674111f90 con 0x7f2674091240 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2640003620 con 0x7f2674091240 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f265c0048e0 con 0x7f2674091240 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): dump: 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.054 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117977034 0 0) 0x7f2674112e00 con 0x7f26741854b0 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient(hunting): my global_id is 4231 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2674111f90 con 0x7f26741854b0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3402867381 0 0) 0x7f267412bbb0 con 0x7f26740909a0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.178:6789/0 conn(0x7f26741854b0 legacy=0x7f2674173f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.102:6789/0 conn(0x7f2674091240 legacy=0x7f2674172860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 monclient: found mon.c 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674110aa0 con 0x7f26740909a0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 5 monclient: authenticate success, global_id 4238 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _renew_subs 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2674112560 con 0x7f26740909a0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _renew_subs 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f26741117d0 con 0x7f26740909a0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f26680039c0 con 0x7f26740909a0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2668003d50 con 0x7f26740909a0 2026-02-21T22:15:37.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 10 monclient: dump: 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.058 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2670d72640 4 set_mon_vals no callback set 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26741104a0 con 0x7f26740909a0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.033+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2668004020 con 0x7f26740909a0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.037+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f266804c540 con 0x7f26740909a0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.037+0000 7f2671d74640 10 monclient: get_auth_request con 0x7f264003d870 auth_method 0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.037+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f2668017650 con 0x7f26740909a0 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.037+0000 7f265affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.037+0000 7f265affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f267405eb70 con 0x7f26740909a0 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f265affd640 1 -- 10.20.193.102:0/18011100 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+11009 (unknown 941590788 0 586439007) 0x7f2668016f30 con 0x7f26740909a0 2026-02-21T22:15:37.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f265affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f265affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":19,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:36.532998+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T22:15:33.157956+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.178:6800/2549949439 conn(0x7f264003d870 legacy=0x7f264005dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 10 monclient: shutdown 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 >> v1:10.20.193.102:6790/0 conn(0x7f26740909a0 legacy=0x7f267412b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 shutdown_connections 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 >> 10.20.193.102:0/18011100 conn(0x7f267405e660 msgr2=0x7f26741a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 shutdown_connections 2026-02-21T22:15:37.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.121+0000 7f2678ced640 1 -- 10.20.193.102:0/18011100 wait complete. 2026-02-21T22:15:37.138 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T22:15:37.138 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:37.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: monmap: 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.206154+0000 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.206154+0000 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.207 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 1 Processor -- start 2026-02-21T22:15:37.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 1 -- start start 2026-02-21T22:15:37.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: init 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-a con 0x7f35a4086880 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-b con 0x7f35a4086c50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-c con 0x7f35a4087060 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a4057d50 con 0x7f35a4086880 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a4057a70 con 0x7f35a4087060 2026-02-21T22:15:37.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a4058030 con 0x7f35a4086c50 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35aa83b640 10 monclient(hunting): authenticate will time out at 1146.442334s 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35a9839640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f35a4086880 0x7f35a40587a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58328/0 (socket says 10.20.193.102:58328) 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35a9038640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 0x7f35a4184120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60078/0 (socket says 10.20.193.102:60078) 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.201+0000 7f35a9839640 1 -- 10.20.193.102:0/3718094544 learned_addr learned my addr 10.20.193.102:0/3718094544 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a9839640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.102:6789/0 conn(0x7f35a4086880 legacy=0x7f35a40587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a9038640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 legacy=0x7f35a4184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa03a640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.178:6789/0 conn(0x7f35a4087060 legacy=0x7f35a41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089038680 0 0) 0x7f3594002c60 con 0x7f35a4086880 2026-02-21T22:15:37.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): my global_id is 4263 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f358c003620 con 0x7f35a4086880 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061073350 0 0) 0x7f35a4058030 con 0x7f35a4086c50 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): my global_id is 4241 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3594002c60 con 0x7f35a4086c50 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1661901681 0 0) 0x7f35a4057a70 con 0x7f35a4087060 2026-02-21T22:15:37.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient(hunting): my global_id is 4234 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35a4058030 con 0x7f35a4087060 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1009445859 0 0) 0x7f358c003620 con 0x7f35a4086880 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 legacy=0x7f35a4184120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.178:6789/0 conn(0x7f35a4087060 legacy=0x7f35a41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 monclient: found mon.noname-a 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a405e650 con 0x7f35a4086880 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 5 monclient: authenticate success, global_id 4263 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f35940032e0 con 0x7f35a4086880 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: dump: 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.217 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f35940049a0 con 0x7f35a4086880 2026-02-21T22:15:37.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 1 -- 10.20.193.102:0/3718094544 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3594004d30 con 0x7f35a4086880 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a8837640 10 monclient: dump: 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.219 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 4 set_mon_vals no callback set 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: shutdown 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- 10.20.193.102:0/3718094544 >> v1:10.20.193.102:6789/0 conn(0x7f35a4086880 legacy=0x7f35a40587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- 10.20.193.102:0/3718094544 shutdown_connections 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- 10.20.193.102:0/3718094544 >> 10.20.193.102:0/3718094544 conn(0x7f35a4056c00 msgr2=0x7f35a4093a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- 10.20.193.102:0/3718094544 shutdown_connections 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- 10.20.193.102:0/3718094544 wait complete. 2026-02-21T22:15:37.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: monmap: 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 Processor -- start 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- start start 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: init 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-b con 0x7f35a4086880 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-c con 0x7f35a4086c50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient(hunting): picked mon.noname-a con 0x7f35a4087060 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a413a960 con 0x7f35a4087060 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a413b400 con 0x7f35a4086880 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a413a4a0 con 0x7f35a4086c50 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35aa83b640 10 monclient(hunting): authenticate will time out at 1146.446486s 2026-02-21T22:15:37.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a9038640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 0x7f35a41945e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60086/0 (socket says 10.20.193.102:60086) 2026-02-21T22:15:37.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.205+0000 7f35a9038640 1 -- 10.20.193.102:0/1773547838 learned_addr learned my addr 10.20.193.102:0/1773547838 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35a9038640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 legacy=0x7f35a41945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa03a640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.102:6789/0 conn(0x7f35a4087060 legacy=0x7f35a4195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35a9839640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.178:6789/0 conn(0x7f35a4086880 legacy=0x7f35a4140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462120407 0 0) 0x7f35a413a960 con 0x7f35a4087060 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): my global_id is 4266 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3574003620 con 0x7f35a4087060 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084414828 0 0) 0x7f35a413a4a0 con 0x7f35a4086c50 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): my global_id is 4244 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35a413a960 con 0x7f35a4086c50 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927685542 0 0) 0x7f35a413b400 con 0x7f35a4086880 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): my global_id is 4237 2026-02-21T22:15:37.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35a413a4a0 con 0x7f35a4086880 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2553845718 0 0) 0x7f3574003620 con 0x7f35a4087060 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f35a413b400 con 0x7f35a4087060 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f35a00048d0 con 0x7f35a4087060 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): dump: 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.229 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2815743188 0 0) 0x7f35a413a960 con 0x7f35a4086c50 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3574003620 con 0x7f35a4086c50 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3598002f00 con 0x7f35a4086c50 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): dump: 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.230 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2274912104 0 0) 0x7f35a413a4a0 con 0x7f35a4086880 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f35a413a960 con 0x7f35a4086880 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3594002b50 con 0x7f35a4086880 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient(hunting): dump: 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1192133020 0 0) 0x7f35a413b400 con 0x7f35a4087060 2026-02-21T22:15:37.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.102:6790/0 conn(0x7f35a4086c50 legacy=0x7f35a41945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.178:6789/0 conn(0x7f35a4086880 legacy=0x7f35a4140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 monclient: found mon.a 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a4153740 con 0x7f35a4087060 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 5 monclient: authenticate success, global_id 4266 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _renew_subs 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f35a4140910 con 0x7f35a4087060 2026-02-21T22:15:37.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _renew_subs 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f35a4089f10 con 0x7f35a4087060 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f35a0004ed0 con 0x7f35a4087060 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f35a0005260 con 0x7f35a4087060 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 10 monclient: dump: 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.234 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f3593fff640 4 set_mon_vals no callback set 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a408ab80 con 0x7f35a4087060 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f35a0005530 con 0x7f35a4087060 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f35a004d950 con 0x7f35a4087060 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.209+0000 7f35a9839640 10 monclient: get_auth_request con 0x7f357403d9a0 auth_method 0 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.213+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f35a0018ac0 con 0x7f35a4087060 2026-02-21T22:15:37.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.213+0000 7f35927fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.213+0000 7f35927fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35a408f420 con 0x7f35a4087060 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35927fc640 1 -- 10.20.193.102:0/1773547838 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+11009 (unknown 941590788 0 586439007) 0x7f35a00183a0 con 0x7f35a4087060 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35927fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35927fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.303 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:37.304 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":19,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:36.532998+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T22:15:33.157956+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:37.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.178:6800/2549949439 conn(0x7f357403d9a0 legacy=0x7f357405de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 10 monclient: shutdown 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 >> v1:10.20.193.102:6789/0 conn(0x7f35a4087060 legacy=0x7f35a4195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 shutdown_connections 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 >> 10.20.193.102:0/1773547838 conn(0x7f35a4056c00 msgr2=0x7f35a408ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 shutdown_connections 2026-02-21T22:15:37.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.297+0000 7f35aa83b640 1 -- 10.20.193.102:0/1773547838 wait complete. 2026-02-21T22:15:37.313 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T22:15:37.314 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T22:15:37.315 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T22:15:37.315 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T22:15:37.316 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-21T22:15:37.316 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-21T22:15:37.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: monmap: 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.376890+0000 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.376890+0000 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.377 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 1 Processor -- start 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 1 -- start start 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: init 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-a con 0x7fa3ec13fb80 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-c con 0x7fa3ec13ff50 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-b con 0x7fa3ec1855d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec093e30 con 0x7fa3ec13fb80 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec092470 con 0x7fa3ec13ff50 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec056470 con 0x7fa3ec1855d0 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f31eb640 10 monclient(hunting): authenticate will time out at 1146.613266s 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f0f60640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa3ec13fb80 0x7fa3ec058530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58336/0 (socket says 10.20.193.102:58336) 2026-02-21T22:15:37.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f0f60640 1 -- 10.20.193.102:0/397229456 learned_addr learned my addr 10.20.193.102:0/397229456 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: monmap: 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.378574+0000 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.378574+0000 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: monmap: 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.379 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.378609+0000 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.378609+0000 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 1 Processor -- start 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 1 -- start start 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: init 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 1 Processor -- start 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 1 -- start start 2026-02-21T22:15:37.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: init 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient(hunting): picked mon.noname-a con 0x7fad4c091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient(hunting): picked mon.noname-c con 0x7fad4c0909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient(hunting): picked mon.noname-b con 0x7fad4c1854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c057a20 con 0x7fad4c091240 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c091d20 con 0x7fad4c0909a0 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c057740 con 0x7fad4c1854b0 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad51398640 10 monclient(hunting): authenticate will time out at 1146.614545s 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad4b7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 0x7fad4c1a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60108/0 (socket says 10.20.193.102:60108) 2026-02-21T22:15:37.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fad4b7fe640 1 -- 10.20.193.102:0/1281281561 learned_addr learned my addr 10.20.193.102:0/1281281561 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-a con 0x7f379013f9c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-c con 0x7f379013fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-b con 0x7f37901401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3790092490 con 0x7f379013f9c0 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37900571f0 con 0x7f379013fd90 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37900574d0 con 0x7f37901401f0 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f37965c7640 10 monclient(hunting): authenticate will time out at 1146.614956s 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f3794dc4640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f379013fd90 0x7f3790183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.102:38846/0 (socket says 10.20.193.102:38846) 2026-02-21T22:15:37.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7f3794dc4640 1 -- 10.20.193.102:0/1428899557 learned_addr learned my addr 10.20.193.102:0/1428899557 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f0f60640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.102:6789/0 conn(0x7fa3ec13fb80 legacy=0x7fa3ec058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 10 monclient: monmap: 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.380312+0000 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.380312+0000 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.383 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3f1761640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.102:6790/0 conn(0x7fa3ec1855d0 legacy=0x7fa3ec1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e3fff640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.178:6789/0 conn(0x7fa3ec13ff50 legacy=0x7fa3ec183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386673580 0 0) 0x7fa3ec056470 con 0x7fa3ec1855d0 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): my global_id is 4247 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3d8003620 con 0x7fa3ec1855d0 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 1 Processor -- start 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850289647 0 0) 0x7fa3ec092470 con 0x7fa3ec13ff50 2026-02-21T22:15:37.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 1 -- start start 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fed866d6640 10 monclient: init 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 10 monclient(hunting): my global_id is 4240 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.373+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3ec056470 con 0x7fa3ec13ff50 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-c con 0x7fed80143040 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2097411717 0 0) 0x7fa3ec056470 con 0x7fa3ec13ff50 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.102:6790/0 conn(0x7fa3ec1855d0 legacy=0x7fa3ec1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-a con 0x7fed80143410 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-b con 0x7fed80185550 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.102:6789/0 conn(0x7fa3ec13fb80 legacy=0x7fa3ec058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 monclient: found mon.noname-c 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8005e440 con 0x7fed80143410 2026-02-21T22:15:37.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec057510 con 0x7fa3ec13ff50 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3d4003160 con 0x7fa3ec13ff50 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed80057010 con 0x7fed80143040 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8008d790 con 0x7fed80185550 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient(hunting): authenticate will time out at 1146.616274s 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: dump: 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.387 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 5 monclient: authenticate success, global_id 4240 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed84c4c640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fed80185550 0x7fed801a5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60130/0 (socket says 10.20.193.102:60130) 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed84c4c640 1 -- 10.20.193.102:0/4169153087 learned_addr learned my addr 10.20.193.102:0/4169153087 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3d4003430 con 0x7fa3ec13ff50 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 1 -- 10.20.193.102:0/397229456 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3d40038c0 con 0x7fa3ec13ff50 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3e37fe640 10 monclient: dump: 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 4 set_mon_vals no callback set 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad4b7fe640 1 -- 10.20.193.102:0/1281281561 >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 legacy=0x7fad4c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191757706 0 0) 0x7fad4c057740 con 0x7fad4c1854b0 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient(hunting): my global_id is 4250 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad38003620 con 0x7fad4c1854b0 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: shutdown 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- 10.20.193.102:0/397229456 >> v1:10.20.193.178:6789/0 conn(0x7fa3ec13ff50 legacy=0x7fa3ec183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad4a7fc640 1 -- 10.20.193.102:0/1281281561 >> v1:10.20.193.178:6789/0 conn(0x7fad4c0909a0 legacy=0x7fad4c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- 10.20.193.102:0/397229456 shutdown_connections 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- 10.20.193.102:0/397229456 >> 10.20.193.102:0/397229456 conn(0x7fa3ec096bc0 msgr2=0x7fa3ec092a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- 10.20.193.102:0/397229456 shutdown_connections 2026-02-21T22:15:37.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- 10.20.193.102:0/397229456 wait complete. 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3023421651 0 0) 0x7fad38003620 con 0x7fad4c1854b0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 >> v1:10.20.193.178:6789/0 conn(0x7fad4c0909a0 legacy=0x7fad4c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 >> v1:10.20.193.102:6789/0 conn(0x7fad4c091240 legacy=0x7fad4c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c090dc0 con 0x7fad4c1854b0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fad3c004620 con 0x7fad4c1854b0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: dump: 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.391 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 5 monclient: authenticate success, global_id 4250 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fad3c0048f0 con 0x7fad4c1854b0 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 1 -- 10.20.193.102:0/1281281561 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fad3c006060 con 0x7fad4c1854b0 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad49ffb640 10 monclient: dump: 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 4 set_mon_vals no callback set 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f3794dc4640 1 -- 10.20.193.102:0/1428899557 >> v1:10.20.193.178:6789/0 conn(0x7f379013fd90 legacy=0x7f3790183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: monmap: 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: shutdown 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- 10.20.193.102:0/1281281561 >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 legacy=0x7fad4c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 Processor -- start 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- 10.20.193.102:0/1281281561 shutdown_connections 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- 10.20.193.102:0/1281281561 >> 10.20.193.102:0/1281281561 conn(0x7fad4c05e660 msgr2=0x7fad4c095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- start start 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: init 2026-02-21T22:15:37.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768113063 0 0) 0x7f37900571f0 con 0x7f379013fd90 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient(hunting): my global_id is 4246 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- 10.20.193.102:0/1281281561 shutdown_connections 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- 10.20.193.102:0/1281281561 wait complete. 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3778003620 con 0x7f379013fd90 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37955c5640 1 -- 10.20.193.102:0/1428899557 >> v1:10.20.193.102:6789/0 conn(0x7f379013f9c0 legacy=0x7f37900906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-b con 0x7fa3ec13fb80 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-c con 0x7fa3ec13ff50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient(hunting): picked mon.noname-a con 0x7fa3ec1855d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec12c0c0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec059240 con 0x7fa3ec13fb80 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ec08f700 con 0x7fa3ec13ff50 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f31eb640 10 monclient(hunting): authenticate will time out at 1146.617822s 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 539070276 0 0) 0x7f3778003620 con 0x7f379013fd90 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 >> v1:10.20.193.102:6790/0 conn(0x7f37901401f0 legacy=0x7f37901a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 >> v1:10.20.193.102:6789/0 conn(0x7f379013f9c0 legacy=0x7f37900906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 monclient: found mon.noname-c 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37900577b0 con 0x7f379013fd90 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 5 monclient: authenticate success, global_id 4246 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3784003110 con 0x7f379013fd90 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: dump: 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f1761640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa3ec1855d0 0x7fa3ec184c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58370/0 (socket says 10.20.193.102:58370) 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fa3f1761640 1 -- 10.20.193.102:0/2214955909 learned_addr learned my addr 10.20.193.102:0/2214955909 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: monmap: 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f37840033e0 con 0x7f379013fd90 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 1 -- 10.20.193.102:0/1428899557 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3784004b50 con 0x7f379013fd90 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f377ffff640 10 monclient: dump: 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 4 set_mon_vals no callback set 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 Processor -- start 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- start start 2026-02-21T22:15:37.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: init 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: shutdown 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- 10.20.193.102:0/1428899557 >> v1:10.20.193.178:6789/0 conn(0x7f379013fd90 legacy=0x7f3790183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- 10.20.193.102:0/1428899557 shutdown_connections 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- 10.20.193.102:0/1428899557 >> 10.20.193.102:0/1428899557 conn(0x7f379005e3f0 msgr2=0x7f3790095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- 10.20.193.102:0/1428899557 shutdown_connections 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- 10.20.193.102:0/1428899557 wait complete. 2026-02-21T22:15:37.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed84c4c640 1 -- 10.20.193.102:0/4169153087 >> v1:10.20.193.102:6790/0 conn(0x7fed80185550 legacy=0x7fed801a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient(hunting): picked mon.noname-a con 0x7fad4c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient(hunting): picked mon.noname-b con 0x7fad4c091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient(hunting): picked mon.noname-c con 0x7fad4c1854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c12bcc0 con 0x7fad4c0909a0 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c112f10 con 0x7fad4c091240 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c1120a0 con 0x7fad4c1854b0 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad51398640 10 monclient(hunting): authenticate will time out at 1146.618612s 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595826259 0 0) 0x7fed8008d790 con 0x7fed80185550 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient(hunting): my global_id is 4253 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed64003620 con 0x7fed80185550 2026-02-21T22:15:37.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad4b7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 0x7fad4c174090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60146/0 (socket says 10.20.193.102:60146) 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fad4b7fe640 1 -- 10.20.193.102:0/305759715 learned_addr learned my addr 10.20.193.102:0/305759715 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3571924782 0 0) 0x7fed64003620 con 0x7fed80185550 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 >> v1:10.20.193.178:6789/0 conn(0x7fed80143040 legacy=0x7fed8008ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 >> v1:10.20.193.102:6789/0 conn(0x7fed80143410 legacy=0x7fed80183e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed80143870 con 0x7fed80185550 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: monmap: 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.402 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 5 monclient: authenticate success, global_id 4253 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fed74004620 con 0x7fed80185550 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: dump: 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.403 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fed740048f0 con 0x7fed80185550 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 1 -- 10.20.193.102:0/4169153087 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fed74006060 con 0x7fed80185550 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed7effd640 10 monclient: dump: 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 4 set_mon_vals no callback set 2026-02-21T22:15:37.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 Processor -- start 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- start start 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: init 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-b con 0x7f379013f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-a con 0x7f379013fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient(hunting): picked mon.noname-c con 0x7f37901401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37901312f0 con 0x7f379013fd90 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3790130cd0 con 0x7f379013f9c0 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3790133650 con 0x7f37901401f0 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7f37965c7640 10 monclient(hunting): authenticate will time out at 1146.619290s 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: shutdown 2026-02-21T22:15:37.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- 10.20.193.102:0/4169153087 >> v1:10.20.193.102:6790/0 conn(0x7fed80185550 legacy=0x7fed801a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- 10.20.193.102:0/4169153087 shutdown_connections 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- 10.20.193.102:0/4169153087 >> 10.20.193.102:0/4169153087 conn(0x7fed80054e20 msgr2=0x7fed80092a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- 10.20.193.102:0/4169153087 shutdown_connections 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 1 -- 10.20.193.102:0/4169153087 wait complete. 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.377+0000 7fed866d6640 10 monclient: monmap: 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 Processor -- start 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- start start 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: init 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f1761640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.102:6789/0 conn(0x7fa3ec1855d0 legacy=0x7fa3ec184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f3794dc4640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f379013fd90 0x7f3790194030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58396/0 (socket says 10.20.193.102:58396) 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f3794dc4640 1 -- 10.20.193.102:0/3522119298 learned_addr learned my addr 10.20.193.102:0/3522119298 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f0f60640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.178:6789/0 conn(0x7fa3ec13fb80 legacy=0x7fa3ec12b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-c con 0x7fed80143040 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-b con 0x7fed80143410 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient(hunting): picked mon.noname-a con 0x7fed80185550 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8007f860 con 0x7fed80185550 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8007cb80 con 0x7fed80143410 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8007fb40 con 0x7fed80143040 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient(hunting): authenticate will time out at 1146.620330s 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7ffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fed80143040 0x7fed8007bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60164/0 (socket says 10.20.193.102:60164) 2026-02-21T22:15:37.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7ffff640 1 -- 10.20.193.102:0/858702076 learned_addr learned my addr 10.20.193.102:0/858702076 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234028907 0 0) 0x7fa3ec059240 con 0x7fa3ec13fb80 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): my global_id is 4249 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003620 con 0x7fa3ec13fb80 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: monmap: 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.385248+0000 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.385248+0000 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.408 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f0f60640 1 --1- 10.20.193.102:0/2214955909 >> v1:10.20.193.178:6789/0 conn(0x7fa3ec13fb80 0x7fa3ec12b2a0 :-1 s=OPENED pgs=78 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fa3dc003270 mon_map magic: 0 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 109820260 0 0) 0x7fa3bc003620 con 0x7fa3ec13fb80 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e3fff640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.102:6790/0 conn(0x7fa3ec13ff50 legacy=0x7fa3ec12b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa3ec059240 con 0x7fa3ec13fb80 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad4b7fe640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 legacy=0x7fad4c174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 1 Processor -- start 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3dc003270 con 0x7fa3ec13fb80 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): dump: 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 1 -- start start 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: init 2026-02-21T22:15:37.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad4a7fc640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.178:6789/0 conn(0x7fad4c091240 legacy=0x7fad4c172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient(hunting): picked mon.noname-c con 0x7f4480056270 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient(hunting): picked mon.noname-a con 0x7f448008da90 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient(hunting): picked mon.noname-b con 0x7f4480056d50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448008ecc0 con 0x7f448008da90 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448008cdf0 con 0x7f4480056270 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448008f0e0 con 0x7f4480056d50 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f4486076640 10 monclient(hunting): authenticate will time out at 1146.621207s 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447f7fe640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f4480056270 0x7f4480056640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.102:38918/0 (socket says 10.20.193.102:38918) 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447f7fe640 1 -- 10.20.193.102:0/1207982526 learned_addr learned my addr 10.20.193.102:0/1207982526 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: monmap: 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.386881+0000 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.386881+0000 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.411 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad4affd640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.102:6789/0 conn(0x7fad4c0909a0 legacy=0x7fad4c12b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f3794dc4640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.102:6789/0 conn(0x7f379013fd90 legacy=0x7f3790194030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290717307 0 0) 0x7fa3ec12c0c0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): my global_id is 4275 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003620 con 0x7fa3ec1855d0 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047470954 0 0) 0x7f37901312f0 con 0x7f379013fd90 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): my global_id is 4281 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 1 Processor -- start 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3760003620 con 0x7f379013fd90 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 1 -- start start 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: init 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7ffff640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.102:6790/0 conn(0x7fed80143040 legacy=0x7fed8007bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1294028567 0 0) 0x7fa3bc003620 con 0x7fa3ec1855d0 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa3ec12c0c0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3e8004930 con 0x7fa3ec1855d0 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): dump: 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.413 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3105532844 0 0) 0x7f3760003620 con 0x7f379013fd90 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f37901312f0 con 0x7f379013fd90 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f37840048e0 con 0x7f379013fd90 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): dump: 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694593938 0 0) 0x7fed8007fb40 con 0x7fed80143040 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): my global_id is 4262 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed50003620 con 0x7fed80143040 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f3795dc6640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.102:6790/0 conn(0x7f37901401f0 legacy=0x7f3790195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952193507 0 0) 0x7fad4c1120a0 con 0x7fad4c1854b0 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): my global_id is 4259 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad18003620 con 0x7fad4c1854b0 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-c con 0x7f5b84091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334920569 0 0) 0x7fad4c12bcc0 con 0x7fad4c0909a0 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): my global_id is 4278 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-b con 0x7f5b840909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad4c1120a0 con 0x7fad4c0909a0 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045089974 0 0) 0x7fad4c112f10 con 0x7fad4c091240 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-a con 0x7f5b841854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): my global_id is 4252 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad4c12bcc0 con 0x7fad4c091240 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b84057a20 con 0x7f5b841854b0 2026-02-21T22:15:37.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3087680293 0 0) 0x7fa3ec08f700 con 0x7fa3ec13ff50 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b84091d20 con 0x7f5b84091240 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient(hunting): my global_id is 4256 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b84057740 con 0x7f5b840909a0 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003620 con 0x7fa3ec13ff50 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2573484477 0 0) 0x7fa3ec12c0c0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.102:6790/0 conn(0x7fa3ec13ff50 legacy=0x7fa3ec12b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b8a9c5640 10 monclient(hunting): authenticate will time out at 1146.622992s 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2259899422 0 0) 0x7f37901312f0 con 0x7f379013fd90 2026-02-21T22:15:37.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.102:6790/0 conn(0x7f37901401f0 legacy=0x7f3790195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.178:6789/0 conn(0x7fa3ec13fb80 legacy=0x7fa3ec12b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 monclient: found mon.a 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.178:6789/0 conn(0x7f379013f9c0 legacy=0x7f379013ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 monclient: found mon.a 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec08e210 con 0x7fa3ec1855d0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3790132db0 con 0x7f379013fd90 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f377dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1937424214 0 0) 0x7fa3ec059240 con 0x7fa3ec13fb80 2026-02-21T22:15:37.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 5 monclient: authenticate success, global_id 4275 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 10 monclient: _renew_subs 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 5 monclient: authenticate success, global_id 4281 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 10 monclient: _renew_subs 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f379013f330 con 0x7f379013fd90 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec08dbc0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b88f3b640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5b841854b0 0x7f5b841a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58430/0 (socket says 10.20.193.102:58430) 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f5b88f3b640 1 -- 10.20.193.102:0/739622163 learned_addr learned my addr 10.20.193.102:0/739622163 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 10 monclient: _renew_subs 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3790132020 con 0x7f379013fd90 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 10 monclient: _renew_subs 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa3ec08fcd0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2828398317 0 0) 0x7fed50003620 con 0x7fed80143040 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed8007fb40 con 0x7fed80143040 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fed6c0043f0 con 0x7fed80143040 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): dump: 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1908453444 0 0) 0x7fad4c12bcc0 con 0x7fad4c091240 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad4c112f10 con 0x7fad4c091240 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fad34003340 con 0x7fad4c091240 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.420 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fad2b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3676651338 0 0) 0x7fed8007fb40 con 0x7fed80143040 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.178:6789/0 conn(0x7fed80143410 legacy=0x7fed8007c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.102:6789/0 conn(0x7fed80185550 legacy=0x7fed80184c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 monclient: found mon.c 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8008c4a0 con 0x7fed80143040 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 5 monclient: authenticate success, global_id 4262 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _renew_subs 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fed801424c0 con 0x7fed80143040 2026-02-21T22:15:37.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.102:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f37900590d0 con 0x7f379013fd90 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _renew_subs 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fed8007f580 con 0x7fed80143040 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.102:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa3ac000f30 con 0x7fa3ec1855d0 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fed6c002990 con 0x7fed80143040 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fed6c002cf0 con 0x7fed80143040 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 10 monclient: dump: 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7e7fc640 4 set_mon_vals no callback set 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447f7fe640 1 -- 10.20.193.102:0/1207982526 >> v1:10.20.193.178:6789/0 conn(0x7f4480056270 legacy=0x7f4480056640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fed80080020 con 0x7fed80143040 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4104986688 0 0) 0x7f448008cdf0 con 0x7f4480056270 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 10 monclient(hunting): my global_id is 4255 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4464003620 con 0x7f4480056270 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3ac000f30 con 0x7fa3ec1855d0 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3e8005080 con 0x7fa3ec1855d0 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e2ffd640 4 set_mon_vals no callback set 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 10 monclient: dump: 2026-02-21T22:15:37.423 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa3e80052d0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.381+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fed6c002f10 con 0x7fed80143040 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2586617142 0 0) 0x7f4464003620 con 0x7f4480056270 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 >> v1:10.20.193.102:6790/0 conn(0x7f4480056d50 legacy=0x7f44801a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 >> v1:10.20.193.102:6789/0 conn(0x7f448008da90 legacy=0x7f4480181e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 monclient: found mon.noname-c 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448008c3f0 con 0x7f4480056270 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f446c003110 con 0x7f4480056270 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 5 monclient: authenticate success, global_id 4255 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: dump: 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fed6c04c650 con 0x7fed80143040 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.178:6801/2402357529 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed6c002b40 con 0x7fed50061fd0 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7fed6c04ca40 con 0x7fed80143040 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed7cff9640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f446c003450 con 0x7f4480056270 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 1 -- 10.20.193.102:0/1207982526 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f446c004b50 con 0x7f4480056270 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447e7fc640 10 monclient: dump: 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 4 set_mon_vals no callback set 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fa3e804d7a0 con 0x7fa3ec1855d0 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed7f7fe640 10 monclient: get_auth_request con 0x7fed5003d320 auth_method 0 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3f0f60640 10 monclient: get_auth_request con 0x7fa3bc03d870 auth_method 0 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.178:6809/3629312922 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3e8004ed0 con 0x7fa3bc062450 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fed84c4c640 10 monclient: get_auth_request con 0x7fed50061fd0 auth_method 0 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: shutdown 2026-02-21T22:15:37.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- 10.20.193.102:0/1207982526 >> v1:10.20.193.178:6789/0 conn(0x7f4480056270 legacy=0x7f4480056640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- 10.20.193.102:0/1207982526 shutdown_connections 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- 10.20.193.102:0/1207982526 >> 10.20.193.102:0/1207982526 conn(0x7f4480054e20 msgr2=0x7f4480092a00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- 10.20.193.102:0/1207982526 shutdown_connections 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- 10.20.193.102:0/1207982526 wait complete. 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3e3fff640 10 monclient: get_auth_request con 0x7fa3bc062450 auth_method 0 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1309016148 0 0) 0x7fad4c1120a0 con 0x7fad4c0909a0 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad4c12bcc0 con 0x7fad4c0909a0 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fad40004260 con 0x7fad4c0909a0 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.427 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2126813338 0 0) 0x7fad4c12bcc0 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.102:6790/0 conn(0x7fad4c1854b0 legacy=0x7fad4c174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.178:6789/0 conn(0x7fad4c091240 legacy=0x7fad4c172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 monclient: found mon.a 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c110bb0 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 5 monclient: authenticate success, global_id 4278 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 10 monclient: _renew_subs 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c112670 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 10 monclient: _renew_subs 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fad4c1118e0 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fad40002990 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fad400045a0 con 0x7fad4c0909a0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad497fa640 4 set_mon_vals no callback set 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: dump: 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: monmap: 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.428 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b837fe640 1 -- 10.20.193.102:0/739622163 >> v1:10.20.193.102:6790/0 conn(0x7f5b840909a0 legacy=0x7f5b84183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b83fff640 1 -- 10.20.193.102:0/739622163 >> v1:10.20.193.178:6789/0 conn(0x7f5b84091240 legacy=0x7f5b84091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fad4c08c250 con 0x7fad4c0909a0 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292806279 0 0) 0x7f5b84057740 con 0x7f5b840909a0 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): my global_id is 4268 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b68003620 con 0x7f5b840909a0 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 Processor -- start 2026-02-21T22:15:37.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fad40002e10 con 0x7fad4c0909a0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- start start 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: init 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223655354 0 0) 0x7f5b84091d20 con 0x7f5b84091240 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient(hunting): my global_id is 4258 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b84057740 con 0x7f5b84091240 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2592696399 0 0) 0x7f5b68003620 con 0x7f5b840909a0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 >> v1:10.20.193.178:6789/0 conn(0x7f5b84091240 legacy=0x7f5b84091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 >> v1:10.20.193.102:6789/0 conn(0x7f5b841854b0 legacy=0x7f5b841a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b84090dc0 con 0x7f5b840909a0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 5 monclient: authenticate success, global_id 4268 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b74003160 con 0x7f5b840909a0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: dump: 2026-02-21T22:15:37.430 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b740033b0 con 0x7f5b840909a0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 1 -- 10.20.193.102:0/739622163 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b740038c0 con 0x7f5b840909a0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b82ffd640 10 monclient: dump: 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 4 set_mon_vals no callback set 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fad4004c2a0 con 0x7fad4c0909a0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6800/4138446215 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad40002b70 con 0x7fad180625a0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7fad4004c690 con 0x7fad4c0909a0 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad2b7fe640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad4b7fe640 10 monclient: get_auth_request con 0x7fad180625a0 auth_method 0 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: shutdown 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/739622163 >> v1:10.20.193.102:6790/0 conn(0x7f5b840909a0 legacy=0x7f5b84183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient(hunting): picked mon.noname-b con 0x7f4480056270 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient(hunting): picked mon.noname-c con 0x7f4480056d50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient(hunting): picked mon.noname-a con 0x7f448008da90 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4480089bb0 con 0x7f448008da90 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f448013cd60 con 0x7f4480056270 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/739622163 shutdown_connections 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4480059730 con 0x7f4480056d50 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f4486076640 10 monclient(hunting): authenticate will time out at 1146.626108s 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/739622163 >> 10.20.193.102:0/739622163 conn(0x7f5b8405e660 msgr2=0x7f5b84095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/739622163 shutdown_connections 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/739622163 wait complete. 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4480056d50 0x7f4480192190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60188/0 (socket says 10.20.193.102:60188) 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f447effd640 1 -- 10.20.193.102:0/3780543083 learned_addr learned my addr 10.20.193.102:0/3780543083 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7fa3e8018040 con 0x7fa3ec1855d0 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fa3e17fa640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f37900590d0 con 0x7f379013fd90 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3784005170 con 0x7f379013fd90 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 10 monclient: dump: 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377f7fe640 4 set_mon_vals no callback set 2026-02-21T22:15:37.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f3784011800 con 0x7f379013fd90 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: monmap: 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f378404c4f0 con 0x7f379013fd90 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.178:6805/653951931 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3784003380 con 0x7f3760062110 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7f378404c830 con 0x7f379013fd90 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f377dffb640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 Processor -- start 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7fad4a7fc640 10 monclient: get_auth_request con 0x7fad1803d940 auth_method 0 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f37955c5640 10 monclient: get_auth_request con 0x7f376003d460 auth_method 0 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- start start 2026-02-21T22:15:37.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: init 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f3795dc6640 10 monclient: get_auth_request con 0x7f3760062110 auth_method 0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-a con 0x7f5b840909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-c con 0x7f5b84091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient(hunting): picked mon.noname-b con 0x7f5b841854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b841120a0 con 0x7f5b840909a0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8412bcc0 con 0x7f5b841854b0 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8412efc0 con 0x7f5b84091240 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b8a9c5640 10 monclient(hunting): authenticate will time out at 1146.627133s 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b83fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5b840909a0 0x7f5b8412b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58438/0 (socket says 10.20.193.102:58438) 2026-02-21T22:15:37.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.385+0000 7f5b83fff640 1 -- 10.20.193.102:0/472766735 learned_addr learned my addr 10.20.193.102:0/472766735 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447effd640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.102:6790/0 conn(0x7f4480056d50 legacy=0x7f4480192190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447ffff640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.102:6789/0 conn(0x7f448008da90 legacy=0x7f44801938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447f7fe640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.178:6789/0 conn(0x7f4480056270 legacy=0x7f44801511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265707068 0 0) 0x7f4480059730 con 0x7f4480056d50 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): my global_id is 4271 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4450003620 con 0x7f4480056d50 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023969209 0 0) 0x7f4480089bb0 con 0x7f448008da90 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): my global_id is 4284 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4480059730 con 0x7f448008da90 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 236496780 0 0) 0x7f4450003620 con 0x7f4480056d50 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4480089bb0 con 0x7f4480056d50 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f44700043c0 con 0x7f4480056d50 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): dump: 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.436 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585275603 0 0) 0x7f448013cd60 con 0x7f4480056270 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): my global_id is 4261 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4450003620 con 0x7f4480056270 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 257136864 0 0) 0x7f4480059730 con 0x7f448008da90 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f448013cd60 con 0x7f448008da90 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f44740057c0 con 0x7f448008da90 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient(hunting): dump: 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3624067570 0 0) 0x7f4480089bb0 con 0x7f4480056d50 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.178:6789/0 conn(0x7f4480056270 legacy=0x7f44801511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.102:6789/0 conn(0x7f448008da90 legacy=0x7f44801938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 monclient: found mon.c 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4480141960 con 0x7f4480056d50 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 5 monclient: authenticate success, global_id 4271 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 10 monclient: _renew_subs 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f448013dcc0 con 0x7f4480056d50 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 10 monclient: _renew_subs 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f448013b820 con 0x7f4480056d50 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f44800008e0 con 0x7f4480056d50 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b83fff640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.102:6789/0 conn(0x7f5b840909a0 legacy=0x7f5b8412b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b837fe640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.102:6790/0 conn(0x7f5b84091240 legacy=0x7f5b84172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662080897 0 0) 0x7f5b84090dc0 con 0x7f5b84091240 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): my global_id is 4274 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b54003620 con 0x7f5b84091240 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965949609 0 0) 0x7f5b841120a0 con 0x7f5b840909a0 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): my global_id is 4287 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b88f3b640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.178:6789/0 conn(0x7f5b841854b0 legacy=0x7f5b84174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b84090dc0 con 0x7f5b840909a0 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44800008e0 con 0x7f4480056d50 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4470002b40 con 0x7f4480056d50 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: dump: 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.438 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447dffb640 4 set_mon_vals no callback set 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4470002d50 con 0x7f4480056d50 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2839128287 0 0) 0x7f5b54003620 con 0x7f5b84091240 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b841120a0 con 0x7f5b84091240 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b74002bc0 con 0x7f5b84091240 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== osd.5 v1:10.20.193.178:6801/2402357529 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7fed6c002b40 con 0x7fed50061fd0 2026-02-21T22:15:37.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744552573 0 0) 0x7f5b8412bcc0 con 0x7f5b841854b0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): my global_id is 4264 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b54003620 con 0x7f5b841854b0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 557306740 0 0) 0x7f5b84090dc0 con 0x7f5b840909a0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b8412bcc0 con 0x7f5b840909a0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b70003420 con 0x7f5b840909a0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f447004c420 con 0x7f4480056d50 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== osd.4 v1:10.20.193.178:6809/3629312922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7fa3e8004ed0 con 0x7fa3bc062450 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6804/604370691 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4470002990 con 0x7f4450062530 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7f447004c760 con 0x7f4480056d50 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f445bfff640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3927010895 0 0) 0x7f5b841120a0 con 0x7f5b84091240 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.178:6789/0 conn(0x7f5b841854b0 legacy=0x7f5b84174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.102:6789/0 conn(0x7f5b840909a0 legacy=0x7f5b8412b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 monclient: found mon.c 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b84112670 con 0x7f5b84091240 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447f7fe640 10 monclient: get_auth_request con 0x7f445003d8d0 auth_method 0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 5 monclient: authenticate success, global_id 4274 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 10 monclient: _renew_subs 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5b84110bb0 con 0x7f5b84091240 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 10 monclient: _renew_subs 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5b841118e0 con 0x7f5b84091240 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f447ffff640 10 monclient: get_auth_request con 0x7f4450062530 auth_method 0 2026-02-21T22:15:37.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b740032a0 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b74004090 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b827fc640 4 set_mon_vals no callback set 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: dump: 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f5b84112fc0 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5b74004330 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f5b7404c510 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6808/579535315 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b74003ee0 con 0x7f5b54062450 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (unknown 4138596325 0 0) 0x7f5b7404c900 con 0x7f5b84091240 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b80ff9640 10 monclient: handle_get_version_reply finishing 1 version 19 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== osd.1 v1:10.20.193.102:6800/4138446215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7fad40002b70 con 0x7fad180625a0 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f5b88f3b640 10 monclient: get_auth_request con 0x7f5b54062450 auth_method 0 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.389+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== osd.3 v1:10.20.193.178:6805/653951931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f3784003380 con 0x7f3760062110 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.393+0000 7f5b83fff640 10 monclient: get_auth_request con 0x7f5b5403d820 auth_method 0 2026-02-21T22:15:37.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.393+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== osd.0 v1:10.20.193.102:6804/604370691 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f4470002990 con 0x7f4450062530 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.393+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== osd.2 v1:10.20.193.102:6808/579535315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f5b74003ee0 con 0x7f5b54062450 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 --> v1:10.20.193.178:6801/2402357529 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed8008f270 con 0x7fed50061fd0 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed7cff9640 1 -- 10.20.193.102:0/858702076 <== osd.5 v1:10.20.193.178:6801/2402357529 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1800444376) 0x7fed8008f270 con 0x7fed50061fd0 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.178:6801/2402357529 conn(0x7fed50061fd0 legacy=0x7fed50082400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.178:6800/2549949439 conn(0x7fed5003d320 legacy=0x7fed5005d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 10 monclient: shutdown 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 >> v1:10.20.193.102:6790/0 conn(0x7fed80143040 legacy=0x7fed8007bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed84c4c640 1 -- 10.20.193.102:0/858702076 reap_dead start 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 shutdown_connections 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 >> 10.20.193.102:0/858702076 conn(0x7fed80054e20 msgr2=0x7fed8008b380 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 shutdown_connections 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fed866d6640 1 -- 10.20.193.102:0/858702076 wait complete. 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 --> v1:10.20.193.102:6800/4138446215 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad4c1106c0 con 0x7fad180625a0 2026-02-21T22:15:37.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad2b7fe640 1 -- 10.20.193.102:0/305759715 <== osd.1 v1:10.20.193.102:6800/4138446215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 654699149) 0x7fad4c1106c0 con 0x7fad180625a0 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.102:6800/4138446215 conn(0x7fad180625a0 legacy=0x7fad180829d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.178:6800/2549949439 conn(0x7fad1803d940 legacy=0x7fad1805ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 10 monclient: shutdown 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 >> v1:10.20.193.102:6789/0 conn(0x7fad4c0909a0 legacy=0x7fad4c12b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad4b7fe640 1 -- 10.20.193.102:0/305759715 reap_dead start 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 shutdown_connections 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 >> 10.20.193.102:0/305759715 conn(0x7fad4c05e660 msgr2=0x7fad4c1a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 shutdown_connections 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fad51398640 1 -- 10.20.193.102:0/305759715 wait complete. 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 --> v1:10.20.193.178:6809/3629312922 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3ac002c90 con 0x7fa3bc062450 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3e17fa640 1 -- 10.20.193.102:0/2214955909 <== osd.4 v1:10.20.193.178:6809/3629312922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 839772338) 0x7fa3ac002c90 con 0x7fa3bc062450 2026-02-21T22:15:37.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.178:6809/3629312922 conn(0x7fa3bc062450 legacy=0x7fa3bc082880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa3bc03d870 legacy=0x7fa3bc05dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 10 monclient: shutdown 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 >> v1:10.20.193.102:6789/0 conn(0x7fa3ec1855d0 legacy=0x7fa3ec184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f1761640 1 -- 10.20.193.102:0/2214955909 reap_dead start 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 shutdown_connections 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 >> 10.20.193.102:0/2214955909 conn(0x7fa3ec096bc0 msgr2=0x7fa3ec141770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 shutdown_connections 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.397+0000 7fa3f31eb640 1 -- 10.20.193.102:0/2214955909 wait complete. 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 --> v1:10.20.193.178:6805/653951931 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3790093ce0 con 0x7f3760062110 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f377dffb640 1 -- 10.20.193.102:0/3522119298 <== osd.3 v1:10.20.193.178:6805/653951931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1800444376) 0x7f3790093ce0 con 0x7f3760062110 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.178:6805/653951931 conn(0x7f3760062110 legacy=0x7f3760082540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.178:6800/2549949439 conn(0x7f376003d460 legacy=0x7f376005d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 10 monclient: shutdown 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 >> v1:10.20.193.102:6789/0 conn(0x7f379013fd90 legacy=0x7f3790194030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f3795dc6640 1 -- 10.20.193.102:0/3522119298 reap_dead start 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 shutdown_connections 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 >> 10.20.193.102:0/3522119298 conn(0x7f379005e3f0 msgr2=0x7f379008a820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 shutdown_connections 2026-02-21T22:15:37.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f37965c7640 1 -- 10.20.193.102:0/3522119298 wait complete. 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 --> v1:10.20.193.102:6804/604370691 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44800864e0 con 0x7f4450062530 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f445bfff640 1 -- 10.20.193.102:0/3780543083 <== osd.0 v1:10.20.193.102:6804/604370691 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1800444376) 0x7f44800864e0 con 0x7f4450062530 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.102:6804/604370691 conn(0x7f4450062530 legacy=0x7f4450082960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.178:6800/2549949439 conn(0x7f445003d8d0 legacy=0x7f445005dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 10 monclient: shutdown 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 >> v1:10.20.193.102:6790/0 conn(0x7f4480056d50 legacy=0x7f4480192190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f447ffff640 1 -- 10.20.193.102:0/3780543083 reap_dead start 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 shutdown_connections 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 >> 10.20.193.102:0/3780543083 conn(0x7f4480054e20 msgr2=0x7f448005a900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 shutdown_connections 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.401+0000 7f4486076640 1 -- 10.20.193.102:0/3780543083 wait complete. 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 --> v1:10.20.193.102:6808/579535315 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b84058480 con 0x7f5b54062450 2026-02-21T22:15:37.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b80ff9640 1 -- 10.20.193.102:0/472766735 <== osd.2 v1:10.20.193.102:6808/579535315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1800444376) 0x7f5b84058480 con 0x7f5b54062450 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.102:6808/579535315 conn(0x7f5b54062450 legacy=0x7f5b54082880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5b5403d820 legacy=0x7f5b5405dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 10 monclient: shutdown 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 >> v1:10.20.193.102:6790/0 conn(0x7f5b84091240 legacy=0x7f5b84172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b88f3b640 1 -- 10.20.193.102:0/472766735 reap_dead start 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 shutdown_connections 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 >> 10.20.193.102:0/472766735 conn(0x7f5b8405e660 msgr2=0x7f5b841a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.405+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 shutdown_connections 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.409+0000 7f5b8a9c5640 1 -- 10.20.193.102:0/472766735 wait complete. 2026-02-21T22:15:37.447 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.447 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T22:15:37.448 INFO:teuthology.orchestra.run.trial102.stdout:47244640260 2026-02-21T22:15:37.448 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T22:15:37.448 INFO:teuthology.orchestra.run.trial102.stdout:51539607555 2026-02-21T22:15:37.448 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T22:15:37.448 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.448 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T22:15:37.449 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.449 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T22:15:37.449 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.449 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T22:15:37.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: monmap: 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.553764+0000 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.553764+0000 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 1 Processor -- start 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 1 -- start start 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: init 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-b con 0x7fddf01407d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-c con 0x7fddf0140ba0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-a con 0x7fddf005df40 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf0141080 con 0x7fddf005df40 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf005aa50 con 0x7fddf0140ba0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf005ad30 con 0x7fddf01407d0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf6eda640 10 monclient(hunting): authenticate will time out at 1146.789719s 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf5450640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fddf005df40 0x7fddf01a3240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58450/0 (socket says 10.20.193.102:58450) 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf5450640 1 -- 10.20.193.102:0/1719892963 learned_addr learned my addr 10.20.193.102:0/1719892963 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: monmap: 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.555 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.555445+0000 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.555445+0000 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 1 Processor -- start 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 1 -- start start 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: init 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-b con 0x7f4cfc13fb80 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-a con 0x7f4cfc13ff50 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-c con 0x7f4cfc1855d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc093e30 con 0x7f4cfc13ff50 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc092470 con 0x7f4cfc1855d0 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc056470 con 0x7f4cfc13fb80 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4d0089e640 10 monclient(hunting): authenticate will time out at 1146.791525s 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4cfa575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13fb80 0x7f4cfc058530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60216/0 (socket says 10.20.193.102:60216) 2026-02-21T22:15:37.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7f4cfa575640 1 -- 10.20.193.102:0/2305539325 learned_addr learned my addr 10.20.193.102:0/2305539325 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.549+0000 7fddf5450640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.102:6789/0 conn(0x7fddf005df40 legacy=0x7fddf01a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf4c4f640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.102:6790/0 conn(0x7fddf01407d0 legacy=0x7fddf005d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde7fff640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.178:6789/0 conn(0x7fddf0140ba0 legacy=0x7fddf0181b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612010985 0 0) 0x7fddf0141080 con 0x7fddf005df40 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): my global_id is 4290 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdddc003620 con 0x7fddf005df40 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813578980 0 0) 0x7fddf005ad30 con 0x7fddf01407d0 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): my global_id is 4277 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf0141080 con 0x7fddf01407d0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 319399253 0 0) 0x7fddf005aa50 con 0x7fddf0140ba0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient(hunting): my global_id is 4267 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf005ad30 con 0x7fddf0140ba0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 246169169 0 0) 0x7fdddc003620 con 0x7fddf005df40 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.102:6790/0 conn(0x7fddf01407d0 legacy=0x7fddf005d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.178:6789/0 conn(0x7fddf0140ba0 legacy=0x7fddf0181b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 monclient: found mon.noname-a 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf005b010 con 0x7fddf005df40 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fddec0046a0 con 0x7fddf005df40 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 5 monclient: authenticate success, global_id 4290 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: dump: 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.559 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf5450640 1 --1- 10.20.193.102:0/1719892963 >> v1:10.20.193.102:6789/0 conn(0x7fddf005df40 0x7fddf01a3240 :-1 s=OPENED pgs=95 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fddec006060 mon_map magic: 0 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fddec004930 con 0x7fddf005df40 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 4 set_mon_vals no callback set 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: shutdown 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- 10.20.193.102:0/1719892963 >> v1:10.20.193.102:6789/0 conn(0x7fddf005df40 legacy=0x7fddf01a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 1 -- 10.20.193.102:0/1719892963 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fddec006060 con 0x7fddf005df40 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde77fe640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- 10.20.193.102:0/1719892963 shutdown_connections 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- 10.20.193.102:0/1719892963 >> 10.20.193.102:0/1719892963 conn(0x7fddf0056cd0 msgr2=0x7fddf0093bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- 10.20.193.102:0/1719892963 shutdown_connections 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- 10.20.193.102:0/1719892963 wait complete. 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: monmap: 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 Processor -- start 2026-02-21T22:15:37.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- start start 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: init 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cfa575640 1 -- 10.20.193.102:0/2305539325 >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13fb80 legacy=0x7f4cfc058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-b con 0x7fddf005df40 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-a con 0x7fddf01407d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient(hunting): picked mon.noname-c con 0x7fddf0140ba0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf0058e10 con 0x7fddf01407d0 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf01a4730 con 0x7fddf005df40 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf012ddf0 con 0x7fddf0140ba0 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fddf6eda640 10 monclient(hunting): authenticate will time out at 1146.793808s 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde7fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fddf01407d0 0x7fddf01a18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58476/0 (socket says 10.20.193.102:58476) 2026-02-21T22:15:37.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7fdde7fff640 1 -- 10.20.193.102:0/3539348249 learned_addr learned my addr 10.20.193.102:0/3539348249 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cfad76640 1 -- 10.20.193.102:0/2305539325 >> v1:10.20.193.178:6789/0 conn(0x7f4cfc1855d0 legacy=0x7f4cfc1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576464681 0 0) 0x7f4cfc056470 con 0x7f4cfc13fb80 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): my global_id is 4280 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce8003620 con 0x7f4cfc13fb80 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610306074 0 0) 0x7f4cfc092470 con 0x7f4cfc1855d0 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: monmap: 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.558885+0000 2026-02-21T22:15:37.562 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.558885+0000 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient(hunting): my global_id is 4270 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cfc056470 con 0x7f4cfc1855d0 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3406740269 0 0) 0x7f4ce8003620 con 0x7f4cfc13fb80 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 >> v1:10.20.193.178:6789/0 conn(0x7f4cfc1855d0 legacy=0x7f4cfc1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 >> v1:10.20.193.102:6789/0 conn(0x7f4cfc13ff50 legacy=0x7f4cfc183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc057510 con 0x7f4cfc13fb80 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4cf0003110 con 0x7f4cfc13fb80 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 5 monclient: authenticate success, global_id 4280 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: dump: 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4cf00033e0 con 0x7f4cfc13fb80 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 1 -- 10.20.193.102:0/2305539325 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4cf0004b50 con 0x7f4cfc13fb80 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9573640 10 monclient: dump: 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 4 set_mon_vals no callback set 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: monmap: 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.565 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.559265+0000 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.559265+0000 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 1 Processor -- start 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 1 -- start start 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: init 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: shutdown 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- 10.20.193.102:0/2305539325 >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13fb80 legacy=0x7f4cfc058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- 10.20.193.102:0/2305539325 shutdown_connections 2026-02-21T22:15:37.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- 10.20.193.102:0/2305539325 >> 10.20.193.102:0/2305539325 conn(0x7f4cfc096bc0 msgr2=0x7f4cfc092a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- 10.20.193.102:0/2305539325 shutdown_connections 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- 10.20.193.102:0/2305539325 wait complete. 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 1 Processor -- start 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 1 -- start start 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: init 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-c con 0x7f8d34091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-a con 0x7f3374091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-a con 0x7f8d340909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-c con 0x7f33740909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-b con 0x7f8d341854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-b con 0x7f33741854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3374057a20 con 0x7f3374091240 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d34057a20 con 0x7f8d340909a0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3374091d20 con 0x7f33740909a0 2026-02-21T22:15:37.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d34091d20 con 0x7f8d34091240 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3374057740 con 0x7f33741854b0 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d34057740 con 0x7f8d341854b0 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d3a182640 10 monclient(hunting): authenticate will time out at 1146.795261s 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3379d91640 10 monclient(hunting): authenticate will time out at 1146.795249s 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: monmap: 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.569 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d337fe640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7f8d34091240 0x7f8d34091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.102:38992/0 (socket says 10.20.193.102:38992) 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d337fe640 1 -- 10.20.193.102:0/1219840227 learned_addr learned my addr 10.20.193.102:0/1219840227 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3373fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f33741854b0 0x7f33741a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60244/0 (socket says 10.20.193.102:60244) 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f3373fff640 1 -- 10.20.193.102:0/1315691991 learned_addr learned my addr 10.20.193.102:0/1315691991 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d337fe640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.178:6789/0 conn(0x7f8d34091240 legacy=0x7f8d34091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 10 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d337fe640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.178:6789/0 conn(0x7f8d34091240 legacy=0x7f8d34091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-21T22:15:37.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f8d337fe640 1 --1- 10.20.193.102:0/1219840227 >> v1:10.20.193.178:6789/0 conn(0x7f8d34091240 0x7f8d34091610 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 gs=1 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 Processor -- start 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- start start 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: init 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-b con 0x7f4cfc13fb80 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-c con 0x7f4cfc13ff50 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient(hunting): picked mon.noname-a con 0x7f4cfc1855d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc08f5d0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc12bf90 con 0x7f4cfc13fb80 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cfc1286b0 con 0x7f4cfc13ff50 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4d0089e640 10 monclient(hunting): authenticate will time out at 1146.795766s 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13ff50 0x7f4cfc12b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60254/0 (socket says 10.20.193.102:60254) 2026-02-21T22:15:37.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.553+0000 7f4cf9d74640 1 -- 10.20.193.102:0/4270773280 learned_addr learned my addr 10.20.193.102:0/4270773280 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde7fff640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.102:6789/0 conn(0x7fddf01407d0 legacy=0x7fddf01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf5450640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.102:6790/0 conn(0x7fddf0140ba0 legacy=0x7fddf01a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf4c4f640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.178:6789/0 conn(0x7fddf005df40 legacy=0x7fddf01136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642073953 0 0) 0x7fddf01a4730 con 0x7fddf005df40 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): my global_id is 4273 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc0003620 con 0x7fddf005df40 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2543427166 0 0) 0x7fddc0003620 con 0x7fddf005df40 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fddf01a4730 con 0x7fddf005df40 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdde0003420 con 0x7fddf005df40 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): dump: 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.572 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2268693869 0 0) 0x7fddf01a4730 con 0x7fddf005df40 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.102:6790/0 conn(0x7fddf0140ba0 legacy=0x7fddf01a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.102:6789/0 conn(0x7fddf01407d0 legacy=0x7fddf01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 monclient: found mon.b 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf012c9c0 con 0x7fddf005df40 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 5 monclient: authenticate success, global_id 4273 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _renew_subs 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fddf012d550 con 0x7fddf005df40 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _renew_subs 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fddf012c5f0 con 0x7fddf005df40 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: monmap: 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.573 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.561787+0000 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.561787+0000 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdde00039c0 con 0x7fddf005df40 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdde0003d60 con 0x7fddf005df40 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 10 monclient: dump: 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde6ffd640 4 set_mon_vals no callback set 2026-02-21T22:15:37.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 1 Processor -- start 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 1 -- start start 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: init 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf00008e0 con 0x7fddf005df40 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3373fff640 1 -- 10.20.193.102:0/1315691991 >> v1:10.20.193.102:6790/0 conn(0x7f33741854b0 legacy=0x7f33741a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdde0004030 con 0x7fddf005df40 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: get_monmap_and_config 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: monmap: 2026-02-21T22:15:37.575 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:37.562369+0000 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:37.562369+0000 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639212857 0 0) 0x7f3374057740 con 0x7f33741854b0 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient(hunting): my global_id is 4283 2026-02-21T22:15:37.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3358003620 con 0x7f33741854b0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-a con 0x7fc05008cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-b con 0x7fc05008d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-c con 0x7fc05008b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0500890e0 con 0x7fc05008cd00 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc050088ea0 con 0x7fc05008b8d0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3372ffd640 1 -- 10.20.193.102:0/1315691991 >> v1:10.20.193.178:6789/0 conn(0x7f33740909a0 legacy=0x7f3374183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05008aea0 con 0x7fc05008d7e0 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc05747d640 10 monclient(hunting): authenticate will time out at 1146.797736s 2026-02-21T22:15:37.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fdde004c510 con 0x7fddf005df40 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4035256143 0 0) 0x7f3358003620 con 0x7f33741854b0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 >> v1:10.20.193.178:6789/0 conn(0x7f33740909a0 legacy=0x7f3374183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 >> v1:10.20.193.102:6789/0 conn(0x7f3374091240 legacy=0x7f3374091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3374090dc0 con 0x7f33741854b0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 5 monclient: authenticate success, global_id 4283 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3368004620 con 0x7f33741854b0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: dump: 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.578 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f33680048f0 con 0x7f33741854b0 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 1 -- 10.20.193.102:0/1315691991 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3368006060 con 0x7f33741854b0 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33727fc640 10 monclient: dump: 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 4 set_mon_vals no callback set 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4cf9d74640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13ff50 legacy=0x7f4cfc12b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fdde7fff640 10 monclient: get_auth_request con 0x7fddc003d450 auth_method 0 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc0559f3640 1 --1- >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 0x7fc0501a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.178:6789/0 says I am v1:10.20.193.102:39008/0 (socket says 10.20.193.102:39008) 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7fc0559f3640 1 -- 10.20.193.102:0/1839207871 learned_addr learned my addr 10.20.193.102:0/1839207871 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 1 Processor -- start 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 1 -- start start 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: init 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d33fff640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.102:6790/0 conn(0x7f8d341854b0 legacy=0x7f8d341a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: shutdown 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- 10.20.193.102:0/1315691991 >> v1:10.20.193.102:6790/0 conn(0x7f33741854b0 legacy=0x7f33741a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- 10.20.193.102:0/1315691991 shutdown_connections 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- 10.20.193.102:0/1315691991 >> 10.20.193.102:0/1315691991 conn(0x7f337405e660 msgr2=0x7f3374095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d32ffd640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.102:6789/0 conn(0x7f8d340909a0 legacy=0x7f8d34183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4cfad76640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.102:6789/0 conn(0x7f4cfc1855d0 legacy=0x7f4cfc184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- 10.20.193.102:0/1315691991 shutdown_connections 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- 10.20.193.102:0/1315691991 wait complete. 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974902417 0 0) 0x7f4cfc08f5d0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): my global_id is 4296 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cc8003620 con 0x7f4cfc1855d0 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:37.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-a con 0x7ffa78091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090183559 0 0) 0x7f8d34057a20 con 0x7f8d340909a0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 10 monclient(hunting): my global_id is 4293 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-b con 0x7ffa780909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d18003620 con 0x7f8d340909a0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-c con 0x7ffa781854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa78057a20 con 0x7ffa78091240 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa78091d20 con 0x7ffa781854b0 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa77fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ffa78091240 0x7ffa78091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58526/0 (socket says 10.20.193.102:58526) 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa77fff640 1 -- 10.20.193.102:0/1000694425 learned_addr learned my addr 10.20.193.102:0/1000694425 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa78057740 con 0x7ffa780909a0 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7ffa7e3ba640 10 monclient(hunting): authenticate will time out at 1146.798592s 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: monmap: 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 Processor -- start 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2681962226 0 0) 0x7f4cc8003620 con 0x7f4cfc1855d0 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4cfc08f5d0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4cec004900 con 0x7f4cfc1855d0 2026-02-21T22:15:37.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): dump: 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- start start 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: init 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3372021314 0 0) 0x7f4cfc08f5d0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.102:6790/0 conn(0x7f4cfc13ff50 legacy=0x7f4cfc12b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.178:6789/0 conn(0x7f4cfc13fb80 legacy=0x7f4cfc12b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 monclient: found mon.a 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc08e0e0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 5 monclient: authenticate success, global_id 4296 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _renew_subs 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc1853a0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _renew_subs 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-c con 0x7f33740909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-a con 0x7f3374091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4cfc08d8f0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient(hunting): picked mon.noname-b con 0x7f33741854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f337412bcc0 con 0x7f3374091240 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3374112f10 con 0x7f33741854b0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33741120a0 con 0x7f33740909a0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f3379d91640 10 monclient(hunting): authenticate will time out at 1146.799108s 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4cec004ed0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4cec005230 con 0x7f4cfc1855d0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4cf8d72640 4 set_mon_vals no callback set 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 10 monclient: dump: 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.586 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33737fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f33740909a0 0x7f337412b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60290/0 (socket says 10.20.193.102:60290) 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f33737fe640 1 -- 10.20.193.102:0/409106914 learned_addr learned my addr 10.20.193.102:0/409106914 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc059220 con 0x7f4cfc1855d0 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4cec005500 con 0x7f4cfc1855d0 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.557+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f4cec04da50 con 0x7f4cfc1855d0 2026-02-21T22:15:37.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f4cfa575640 10 monclient: get_auth_request con 0x7f4cc803d3e0 auth_method 0 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0559f3640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 legacy=0x7fc0501a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0549f1640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.102:6790/0 conn(0x7fc05008d7e0 legacy=0x7fc0501840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0559f3640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 legacy=0x7fc0501a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0559f3640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 legacy=0x7fc0501a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0559f3640 1 --1- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 0x7fc0501a57e0 :-1 s=OPENED pgs=92 gs=3 cs=1 l=1).handle_message read tag failed 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0559f3640 1 --1- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 0x7fc0501a57e0 :-1 s=OPENED pgs=92 gs=3 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.178:6789/0 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646275947 0 0) 0x7fc05008aea0 con 0x7fc05008d7e0 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient(hunting): my global_id is 4289 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc038003620 con 0x7fc05008d7e0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 228073994 0 0) 0x7fc038003620 con 0x7fc05008d7e0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.178:6789/0 conn(0x7fc05008b8d0 legacy=0x7fc0501a57e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.102:6789/0 conn(0x7fc05008cd00 legacy=0x7fc05008d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05008a600 con 0x7fc05008d7e0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc044003160 con 0x7fc05008d7e0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 5 monclient: authenticate success, global_id 4289 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: dump: 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc044003430 con 0x7fc05008d7e0 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 1 -- 10.20.193.102:0/1839207871 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc0440038c0 con 0x7fc05008d7e0 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc03ffff640 10 monclient: dump: 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 4 set_mon_vals no callback set 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa777fe640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 legacy=0x7ffa78183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa77fff640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.102:6789/0 conn(0x7ffa78091240 legacy=0x7ffa78091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099075894 0 0) 0x7f8d34057740 con 0x7f8d341854b0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient(hunting): my global_id is 4286 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d34057a20 con 0x7f8d341854b0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3137739894 0 0) 0x7f8d18003620 con 0x7f8d340909a0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.102:6790/0 conn(0x7f8d341854b0 legacy=0x7f8d341a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.178:6789/0 conn(0x7f8d34091240 legacy=0x7f8d34091610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 monclient: found mon.noname-a 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d34090dc0 con 0x7f8d340909a0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 5 monclient: authenticate success, global_id 4293 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8d24003160 con 0x7f8d340909a0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: dump: 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.591 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: shutdown 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- 10.20.193.102:0/1839207871 >> v1:10.20.193.102:6790/0 conn(0x7fc05008d7e0 legacy=0x7fc0501840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7c930640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.178:6789/0 conn(0x7ffa781854b0 legacy=0x7ffa781a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- 10.20.193.102:0/1839207871 shutdown_connections 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- 10.20.193.102:0/1839207871 >> 10.20.193.102:0/1839207871 conn(0x7fc050091a50 msgr2=0x7fc050091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098956224 0 0) 0x7ffa78057740 con 0x7ffa780909a0 2026-02-21T22:15:37.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient(hunting): my global_id is 4292 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- 10.20.193.102:0/1839207871 shutdown_connections 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa5c003620 con 0x7ffa780909a0 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- 10.20.193.102:0/1839207871 wait complete. 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094624352 0 0) 0x7ffa5c003620 con 0x7ffa780909a0 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.178:6789/0 conn(0x7ffa781854b0 legacy=0x7ffa781a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fdde0017630 con 0x7fddf005df40 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fdde57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fdde57fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.102:6789/0 conn(0x7ffa78091240 legacy=0x7ffa78091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 monclient: found mon.noname-b 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78090dc0 con 0x7ffa780909a0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa68003160 con 0x7ffa780909a0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: dump: 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 5 monclient: authenticate success, global_id 4292 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffa68003430 con 0x7ffa780909a0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 1 -- 10.20.193.102:0/1000694425 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa680038c0 con 0x7ffa780909a0 2026-02-21T22:15:37.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa76ffd640 10 monclient: dump: 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 4 set_mon_vals no callback set 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f33737fe640 1 -- 10.20.193.102:0/409106914 >> v1:10.20.193.102:6790/0 conn(0x7f33740909a0 legacy=0x7f337412b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: monmap: 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.595 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470164672 0 0) 0x7f33741120a0 con 0x7f33740909a0 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): my global_id is 4295 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3344003620 con 0x7f33740909a0 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: shutdown 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 legacy=0x7ffa78183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 shutdown_connections 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 >> 10.20.193.102:0/1000694425 conn(0x7ffa7805e660 msgr2=0x7ffa78095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 shutdown_connections 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 Processor -- start 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/1000694425 wait complete. 2026-02-21T22:15:37.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1843350889 0 0) 0x7f3344003620 con 0x7f33740909a0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- start start 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: init 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f33741120a0 con 0x7f33740909a0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f33600043f0 con 0x7f33740909a0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8d24003430 con 0x7f8d340909a0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 1 -- 10.20.193.102:0/1219840227 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8d240038c0 con 0x7f8d340909a0 2026-02-21T22:15:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d327fc640 10 monclient: dump: 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 4 set_mon_vals no callback set 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 502993113 0 0) 0x7f33741120a0 con 0x7f33740909a0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 >> v1:10.20.193.178:6789/0 conn(0x7f33741854b0 legacy=0x7f3374174090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 >> v1:10.20.193.102:6789/0 conn(0x7f3374091240 legacy=0x7f3374172970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 monclient: found mon.c 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3374110bb0 con 0x7f33740909a0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 5 monclient: authenticate success, global_id 4295 2026-02-21T22:15:37.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _renew_subs 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3374112670 con 0x7f33740909a0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _renew_subs 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f33741118e0 con 0x7f33740909a0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-c con 0x7fc05008b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f4cec018bc0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f4ce2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f4ce2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-b con 0x7fc05008cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient(hunting): picked mon.noname-a con 0x7fc05008d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc050088a70 con 0x7fc05008d7e0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc050081cb0 con 0x7fc05008cd00 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc050087d20 con 0x7fc05008b8d0 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc05747d640 10 monclient(hunting): authenticate will time out at 1146.801943s 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3360002990 con 0x7f33740909a0 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3360002cf0 con 0x7f33740909a0 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 10 monclient: dump: 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3371ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: shutdown 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0551f2640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fc05008b8d0 0x7fc0501533e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60306/0 (socket says 10.20.193.102:60306) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7fc0551f2640 1 -- 10.20.193.102:0/2946608704 learned_addr learned my addr 10.20.193.102:0/2946608704 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- 10.20.193.102:0/1219840227 >> v1:10.20.193.102:6789/0 conn(0x7f8d340909a0 legacy=0x7f8d34183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: monmap: 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- 10.20.193.102:0/1219840227 shutdown_connections 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- 10.20.193.102:0/1219840227 >> 10.20.193.102:0/1219840227 conn(0x7f8d3405e660 msgr2=0x7f8d34095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- 10.20.193.102:0/1219840227 shutdown_connections 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- 10.20.193.102:0/1219840227 wait complete. 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 Processor -- start 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- start start 2026-02-21T22:15:37.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: init 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337408c250 con 0x7f33740909a0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f3360002f10 con 0x7f33740909a0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: build_initial_monmap 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: monmap: 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-c con 0x7ffa780909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-a con 0x7ffa78091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient(hunting): picked mon.noname-b con 0x7ffa781854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa78112100 con 0x7ffa78091240 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7812bd20 con 0x7ffa781854b0 2026-02-21T22:15:37.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7812f020 con 0x7ffa780909a0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f336004c630 con 0x7f33740909a0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa7e3ba640 10 monclient(hunting): authenticate will time out at 1146.802678s 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa77fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 0x7ffa7812b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60318/0 (socket says 10.20.193.102:60318) 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7ffa77fff640 1 -- 10.20.193.102:0/4004724774 learned_addr learned my addr 10.20.193.102:0/4004724774 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 Processor -- start 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- start start 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: init 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f3372ffd640 10 monclient: get_auth_request con 0x7f334403d320 auth_method 0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-b con 0x7f8d340909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-a con 0x7f8d34091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient(hunting): picked mon.noname-c con 0x7f8d341854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:37.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d341120c0 con 0x7f8d34091240 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d3412bce0 con 0x7f8d340909a0 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d3412efe0 con 0x7f8d341854b0 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.561+0000 7f8d3a182640 10 monclient(hunting): authenticate will time out at 1146.803062s 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc0551f2640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.102:6790/0 conn(0x7fc05008b8d0 legacy=0x7fc0501533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc0559f3640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.102:6789/0 conn(0x7fc05008d7e0 legacy=0x7fc050195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d33fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f8d341854b0 0x7f8d341740b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60330/0 (socket says 10.20.193.102:60330) 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d33fff640 1 -- 10.20.193.102:0/462064475 learned_addr learned my addr 10.20.193.102:0/462064475 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc0549f1640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.178:6789/0 conn(0x7fc05008cd00 legacy=0x7fc0501945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297332092 0 0) 0x7fc050081cb0 con 0x7fc05008cd00 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): my global_id is 4279 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc020003620 con 0x7fc05008cd00 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f3360017740 con 0x7f33740909a0 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f334ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f334ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1429652353 0 0) 0x7fc020003620 con 0x7fc05008cd00 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc050081cb0 con 0x7fc05008cd00 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc044003710 con 0x7fc05008cd00 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa77fff640 1 -- 10.20.193.102:0/4004724774 >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 legacy=0x7ffa7812b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): dump: 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727976615 0 0) 0x7ffa78057a20 con 0x7ffa780909a0 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3074376380 0 0) 0x7fc050081cb0 con 0x7fc05008cd00 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): my global_id is 4301 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa44003620 con 0x7ffa780909a0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.102:6790/0 conn(0x7fc05008b8d0 legacy=0x7fc0501533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.102:6789/0 conn(0x7fc05008d7e0 legacy=0x7fc050195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 monclient: found mon.b 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05005d4b0 con 0x7fc05008cd00 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 626466530 0 0) 0x7ffa44003620 con 0x7ffa780909a0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa78057a20 con 0x7ffa780909a0 2026-02-21T22:15:37.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa64003330 con 0x7ffa780909a0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 5 monclient: authenticate success, global_id 4279 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _renew_subs 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc0500819d0 con 0x7fc05008cd00 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _renew_subs 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc05007f220 con 0x7fc05008cd00 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1138512409 0 0) 0x7ffa78057a20 con 0x7ffa780909a0 2026-02-21T22:15:37.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 >> v1:10.20.193.178:6789/0 conn(0x7ffa781854b0 legacy=0x7ffa781740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 >> v1:10.20.193.102:6789/0 conn(0x7ffa78091240 legacy=0x7ffa781729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 monclient: found mon.c 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78110c10 con 0x7ffa780909a0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 5 monclient: authenticate success, global_id 4301 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _renew_subs 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc044003410 con 0x7fc05008cd00 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781126d0 con 0x7ffa780909a0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc044004090 con 0x7fc05008cd00 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 10 monclient: dump: 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.608 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _renew_subs 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ffa78111940 con 0x7ffa780909a0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa77fff640 1 --1- 10.20.193.102:0/4004724774 >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 0x7ffa7812b610 :-1 s=OPENED pgs=98 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7ffa64004b90 config(0 keys) 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffa64004b90 con 0x7ffa780909a0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0500008e0 con 0x7fc05008cd00 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa780008e0 con 0x7ffa780909a0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa64004f30 con 0x7ffa780909a0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 10 monclient: dump: 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.609 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ffa64005200 con 0x7ffa780909a0 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fc044004270 con 0x7fc05008cd00 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03f7fe640 4 set_mon_vals no callback set 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d33fff640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.102:6790/0 conn(0x7f8d341854b0 legacy=0x7f8d341740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa777fe640 10 monclient: get_auth_request con 0x7ffa4403d300 auth_method 0 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa767fc640 4 set_mon_vals no callback set 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fc04404c450 con 0x7fc05008cd00 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d32ffd640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.102:6789/0 conn(0x7f8d34091240 legacy=0x7f8d34172990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d337fe640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.178:6789/0 conn(0x7f8d340909a0 legacy=0x7f8d3412b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:37.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7fc0551f2640 10 monclient: get_auth_request con 0x7fc02003d450 auth_method 0 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3517880660 0 0) 0x7f8d341120c0 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): my global_id is 4302 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d04003620 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4070001748 0 0) 0x7f8d04003620 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8d341120c0 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8d24002bc0 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3397112117 0 0) 0x7f8d341120c0 con 0x7f8d34091240 2026-02-21T22:15:37.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.102:6790/0 conn(0x7f8d341854b0 legacy=0x7f8d341740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.178:6789/0 conn(0x7f8d340909a0 legacy=0x7f8d3412b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 monclient: found mon.a 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d34112690 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 5 monclient: authenticate success, global_id 4302 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _renew_subs 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8d34110bd0 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _renew_subs 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8d34111900 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8d24003430 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8d24004110 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 10 monclient: dump: 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d31ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d34112fe0 con 0x7f8d34091240 2026-02-21T22:15:37.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7ffa6404c760 con 0x7ffa780909a0 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.565+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8d240043b0 con 0x7f8d34091240 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f8d2404c600 con 0x7f8d34091240 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7f8d337fe640 10 monclient: get_auth_request con 0x7f8d0403d400 auth_method 0 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fc0440175c0 con 0x7fc05008cd00 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7fc03dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7fc03dffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f8d34112690 con 0x7f8d34091240 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7f8d0ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:37.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.569+0000 7f8d0ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fddf6eda640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fddf6eda640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fddf6eda640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fddf012d930 con 0x7fddf005df40 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fdde57fa640 1 -- 10.20.193.102:0/3539348249 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 847495621) 0x7fdde001c8e0 con 0x7fddf005df40 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fdde57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.645+0000 7fdde57fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.652 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.178:6800/2549949439 conn(0x7fddc003d450 legacy=0x7fddc005d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 10 monclient: shutdown 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 >> v1:10.20.193.178:6789/0 conn(0x7fddf005df40 legacy=0x7fddf01136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 shutdown_connections 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 >> 10.20.193.102:0/3539348249 conn(0x7fddf0056cd0 msgr2=0x7fddf0088de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 shutdown_connections 2026-02-21T22:15:37.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.649+0000 7fddf6eda640 1 -- 10.20.193.102:0/3539348249 wait complete. 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f337405eb70 con 0x7f33740909a0 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f334ffff640 1 -- 10.20.193.102:0/409106914 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 847495621) 0x7f3360017020 con 0x7f33740909a0 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f334ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f334ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.657 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-21T22:15:37.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-21T22:15:37.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:15:37.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc05007f510 con 0x7fc05008cd00 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc03dffb640 1 -- 10.20.193.102:0/2946608704 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 1444633209) 0x7fc044016ea0 con 0x7fc05008cd00 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc03dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc03dffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stdout:47244640260 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 >> v1:10.20.193.178:6800/2549949439 conn(0x7f334403d320 legacy=0x7f334405d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 10 monclient: shutdown 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 >> v1:10.20.193.102:6790/0 conn(0x7f33740909a0 legacy=0x7f337412b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 shutdown_connections 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 >> 10.20.193.102:0/409106914 conn(0x7f337405e660 msgr2=0x7f33741a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 shutdown_connections 2026-02-21T22:15:37.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7f3379d91640 1 -- 10.20.193.102:0/409106914 wait complete. 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.178:6800/2549949439 conn(0x7fc02003d450 legacy=0x7fc02005d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 10 monclient: shutdown 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 >> v1:10.20.193.178:6789/0 conn(0x7fc05008cd00 legacy=0x7fc0501945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 shutdown_connections 2026-02-21T22:15:37.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 >> 10.20.193.102:0/2946608704 conn(0x7fc050091a50 msgr2=0x7fc050085740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 shutdown_connections 2026-02-21T22:15:37.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.653+0000 7fc05747d640 1 -- 10.20.193.102:0/2946608704 wait complete. 2026-02-21T22:15:37.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.3 2026-02-21T22:15:37.664 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4d0089e640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4d0089e640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4d0089e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4cfc095010 con 0x7f4cfc1855d0 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4ce2ffd640 1 -- 10.20.193.102:0/4270773280 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 956180462) 0x7f4cec0184a0 con 0x7f4cfc1855d0 2026-02-21T22:15:37.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4ce2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-21T22:15:37.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.661+0000 7f4ce2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.669 INFO:teuthology.orchestra.run.trial102.stdout:51539607555 2026-02-21T22:15:37.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.2 2026-02-21T22:15:37.670 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.178:6800/2549949439 conn(0x7f4cc803d3e0 legacy=0x7f4cc805d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 10 monclient: shutdown 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 >> v1:10.20.193.102:6789/0 conn(0x7f4cfc1855d0 legacy=0x7f4cfc184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 shutdown_connections 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 >> 10.20.193.102:0/4270773280 conn(0x7f4cfc096bc0 msgr2=0x7f4cfc141770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 shutdown_connections 2026-02-21T22:15:37.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.665+0000 7f4d0089e640 1 -- 10.20.193.102:0/4270773280 wait complete. 2026-02-21T22:15:37.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.1 2026-02-21T22:15:37.672 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:37.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d3a182640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-21T22:15:37.675 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d3a182640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d3a182640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8d3405eb70 con 0x7f8d34091240 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d0ffff640 1 -- 10.20.193.102:0/462064475 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 847495621) 0x7f8d240177d0 con 0x7f8d34091240 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d0ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.669+0000 7f8d0ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:37.676 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.178:6800/2549949439 conn(0x7f8d0403d400 legacy=0x7f8d0405d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 10 monclient: shutdown 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 >> v1:10.20.193.102:6789/0 conn(0x7f8d34091240 legacy=0x7f8d34172990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 shutdown_connections 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 >> 10.20.193.102:0/462064475 conn(0x7f8d3405e660 msgr2=0x7f8d341a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 shutdown_connections 2026-02-21T22:15:37.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:37.673+0000 7f8d3a182640 1 -- 10.20.193.102:0/462064475 wait complete. 2026-02-21T22:15:37.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607555 got 51539607555 for osd.4 2026-02-21T22:15:37.681 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:37.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.5 2026-02-21T22:15:37.688 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:38.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.397+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ffa7812f020 con 0x7ffa780909a0 2026-02-21T22:15:38.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.397+0000 7ffa74ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.397+0000 7ffa74ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:38.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa7e3ba640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-21T22:15:38.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa7e3ba640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa7e3ba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffa78095010 con 0x7ffa780909a0 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa74ff9640 1 -- 10.20.193.102:0/4004724774 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 847495621) 0x7ffa64017870 con 0x7ffa780909a0 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa74ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.481+0000 7ffa74ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:38.489 INFO:teuthology.orchestra.run.trial102.stdout:47244640259 2026-02-21T22:15:38.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 >> v1:10.20.193.178:6800/2549949439 conn(0x7ffa4403d300 legacy=0x7ffa4405d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 10 monclient: shutdown 2026-02-21T22:15:38.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 >> v1:10.20.193.102:6790/0 conn(0x7ffa780909a0 legacy=0x7ffa7812b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 shutdown_connections 2026-02-21T22:15:38.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 >> 10.20.193.102:0/4004724774 conn(0x7ffa7805e660 msgr2=0x7ffa7808ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 shutdown_connections 2026-02-21T22:15:38.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.485+0000 7ffa7e3ba640 1 -- 10.20.193.102:0/4004724774 wait complete. 2026-02-21T22:15:38.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.0 2026-02-21T22:15:38.500 DEBUG:teuthology.parallel:result is None 2026-02-21T22:15:38.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T22:15:38.500 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:38.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:38.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: get_monmap_and_config 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: monmap: 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:38.568012+0000 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:38.568012+0000 2026-02-21T22:15:38.568 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.569 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.569 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.569 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:38.569 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:38.569 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 1 Processor -- start 2026-02-21T22:15:38.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 1 -- start start 2026-02-21T22:15:38.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: init 2026-02-21T22:15:38.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient(hunting): picked mon.noname-b con 0x7feac80584c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient(hunting): picked mon.noname-a con 0x7feac80588a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient(hunting): picked mon.noname-c con 0x7feac81435f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac80902a0 con 0x7feac80588a0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac80921d0 con 0x7feac81435f0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac80574a0 con 0x7feac80584c0 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.561+0000 7feace76a640 10 monclient(hunting): authenticate will time out at 1147.803931s 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac77fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7feac80588a0 0x7feac8183d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58570/0 (socket says 10.20.193.102:58570) 2026-02-21T22:15:38.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac77fe640 1 -- 10.20.193.102:0/705304335 learned_addr learned my addr 10.20.193.102:0/705304335 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac77fe640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.102:6789/0 conn(0x7feac80588a0 legacy=0x7feac8183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac7fff640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.102:6790/0 conn(0x7feac80584c0 legacy=0x7feac8142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feaccce0640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.178:6789/0 conn(0x7feac81435f0 legacy=0x7feac81a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198109639 0 0) 0x7feab8002c60 con 0x7feac80588a0 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): my global_id is 4305 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaac003620 con 0x7feac80588a0 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894053505 0 0) 0x7feac80574a0 con 0x7feac80584c0 2026-02-21T22:15:38.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): my global_id is 4304 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab8002c60 con 0x7feac80584c0 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3623583225 0 0) 0x7feac80921d0 con 0x7feac81435f0 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient(hunting): my global_id is 4282 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac80574a0 con 0x7feac81435f0 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 642659839 0 0) 0x7feaac003620 con 0x7feac80588a0 2026-02-21T22:15:38.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.102:6790/0 conn(0x7feac80584c0 legacy=0x7feac8142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.178:6789/0 conn(0x7feac81435f0 legacy=0x7feac81a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 monclient: found mon.noname-a 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac8056640 con 0x7feac80588a0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7feab80032e0 con 0x7feac80588a0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: dump: 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 5 monclient: authenticate success, global_id 4305 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feab80049a0 con 0x7feac80588a0 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 1 -- 10.20.193.102:0/705304335 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7feab8004d30 con 0x7feac80588a0 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feac6ffd640 10 monclient: dump: 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.582 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 4 set_mon_vals no callback set 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: shutdown 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- 10.20.193.102:0/705304335 >> v1:10.20.193.102:6789/0 conn(0x7feac80588a0 legacy=0x7feac8183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- 10.20.193.102:0/705304335 shutdown_connections 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- 10.20.193.102:0/705304335 >> 10.20.193.102:0/705304335 conn(0x7feac805e700 msgr2=0x7feac8095ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- 10.20.193.102:0/705304335 shutdown_connections 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- 10.20.193.102:0/705304335 wait complete. 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: monmap: 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:38.584 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 Processor -- start 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- start start 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: init 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient(hunting): picked mon.noname-b con 0x7feac80588a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient(hunting): picked mon.noname-c con 0x7feac81435f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient(hunting): picked mon.noname-a con 0x7feac8130070 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac8150c00 con 0x7feac8130070 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac8059020 con 0x7feac80588a0 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac8130970 con 0x7feac81435f0 2026-02-21T22:15:38.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.565+0000 7feace76a640 10 monclient(hunting): authenticate will time out at 1147.807997s 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feaccce0640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7feac8130070 0x7feac8195cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58572/0 (socket says 10.20.193.102:58572) 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feaccce0640 1 -- 10.20.193.102:0/3730555840 learned_addr learned my addr 10.20.193.102:0/3730555840 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feaccce0640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.102:6789/0 conn(0x7feac8130070 legacy=0x7feac8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac77fe640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.102:6790/0 conn(0x7feac81435f0 legacy=0x7feac81945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749035154 0 0) 0x7feac8130970 con 0x7feac81435f0 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): my global_id is 4307 2026-02-21T22:15:38.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea98003620 con 0x7feac81435f0 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497152041 0 0) 0x7feac80574a0 con 0x7feac8130070 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): my global_id is 4308 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac8130970 con 0x7feac8130070 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac7fff640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.178:6789/0 conn(0x7feac80588a0 legacy=0x7feac813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4261453754 0 0) 0x7fea98003620 con 0x7feac81435f0 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feac80574a0 con 0x7feac81435f0 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7feab8002b50 con 0x7feac81435f0 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:38.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:38.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1171531184 0 0) 0x7feac8130970 con 0x7feac8130070 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fea98003620 con 0x7feac8130070 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7feabc0049a0 con 0x7feac8130070 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.589 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 145461098 0 0) 0x7feac80574a0 con 0x7feac81435f0 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.178:6789/0 conn(0x7feac80588a0 legacy=0x7feac813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.102:6789/0 conn(0x7feac8130070 legacy=0x7feac8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 monclient: found mon.c 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac812f510 con 0x7feac81435f0 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 5 monclient: authenticate success, global_id 4307 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _renew_subs 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feac8131870 con 0x7feac81435f0 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _renew_subs 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7feac8150190 con 0x7feac81435f0 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac80008e0 con 0x7feac81435f0 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7feab80049a0 con 0x7feac81435f0 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7feab8005060 con 0x7feac81435f0 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 10 monclient: dump: 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.592 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac67fc640 4 set_mon_vals no callback set 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7feab80116f0 con 0x7feac81435f0 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7feab804c420 con 0x7feac81435f0 2026-02-21T22:15:38.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.569+0000 7feac7fff640 10 monclient: get_auth_request con 0x7fea9803d720 auth_method 0 2026-02-21T22:15:38.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.573+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7feac80902a0 con 0x7feac81435f0 2026-02-21T22:15:38.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.573+0000 7feac4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.573+0000 7feac4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:38.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feac8130490 con 0x7fea9803d720 2026-02-21T22:15:38.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feac4ff9640 1 -- 10.20.193.102:0/3730555840 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34187 (unknown 2965378022 0 146340977) 0x7feac8130490 con 0x7fea9803d720 2026-02-21T22:15:38.664 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.178:6800/2549949439 conn(0x7fea9803d720 legacy=0x7fea9805dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 10 monclient: shutdown 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 >> v1:10.20.193.102:6790/0 conn(0x7feac81435f0 legacy=0x7feac81945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 shutdown_connections 2026-02-21T22:15:38.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 >> 10.20.193.102:0/3730555840 conn(0x7feac805e700 msgr2=0x7feac8092fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 shutdown_connections 2026-02-21T22:15:38.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.657+0000 7feace76a640 1 -- 10.20.193.102:0/3730555840 wait complete. 2026-02-21T22:15:38.674 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-21T22:15:37.487922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":162832,"kb_used_data":1072,"kb_used_omap":48,"kb_used_meta":161615,"kb_avail":566068208,"statfs":{"total":579820584960,"available":579653844992,"internally_reserved":0,"allocated":1097728,"data_stored":1484939,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":50058,"internal_metadata":165493878},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540434+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:15:36.540434+0000","last_peered":"2026-02-21T22:15:36.540434+0000","last_clean":"2026-02-21T22:15:36.540434+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:15:36.540434+0000","last_undegraded":"2026-02-21T22:15:36.540434+0000","last_fullsized":"2026-02-21T22:15:36.540434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537959+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537959+0000","last_peered":"2026-02-21T22:15:36.537959+0000","last_clean":"2026-02-21T22:15:36.537959+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:15:36.537959+0000","last_undegraded":"2026-02-21T22:15:36.537959+0000","last_fullsized":"2026-02-21T22:15:36.537959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540562+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:15:36.540562+0000","last_peered":"2026-02-21T22:15:36.540562+0000","last_clean":"2026-02-21T22:15:36.540562+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:15:36.540562+0000","last_undegraded":"2026-02-21T22:15:36.540562+0000","last_fullsized":"2026-02-21T22:15:36.540562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537987+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537987+0000","last_peered":"2026-02-21T22:15:36.537987+0000","last_clean":"2026-02-21T22:15:36.537987+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:15:36.537987+0000","last_undegraded":"2026-02-21T22:15:36.537987+0000","last_fullsized":"2026-02-21T22:15:36.537987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.552137+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:15:36.552137+0000","last_peered":"2026-02-21T22:15:36.552137+0000","last_clean":"2026-02-21T22:15:36.552137+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:15:36.552137+0000","last_undegraded":"2026-02-21T22:15:36.552137+0000","last_fullsized":"2026-02-21T22:15:36.552137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.538007+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:15:36.538007+0000","last_peered":"2026-02-21T22:15:36.538007+0000","last_clean":"2026-02-21T22:15:36.538007+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:15:36.538007+0000","last_undegraded":"2026-02-21T22:15:36.538007+0000","last_fullsized":"2026-02-21T22:15:36.538007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540688+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:15:36.540688+0000","last_peered":"2026-02-21T22:15:36.540688+0000","last_clean":"2026-02-21T22:15:36.540688+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:15:36.540688+0000","last_undegraded":"2026-02-21T22:15:36.540688+0000","last_fullsized":"2026-02-21T22:15:36.540688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.542334+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:15:36.542334+0000","last_peered":"2026-02-21T22:15:36.542334+0000","last_clean":"2026-02-21T22:15:36.542334+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:15:36.542334+0000","last_undegraded":"2026-02-21T22:15:36.542334+0000","last_fullsized":"2026-02-21T22:15:36.542334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'112","reported_seq":157,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540168+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:15:36.540168+0000","last_peered":"2026-02-21T22:15:36.540168+0000","last_clean":"2026-02-21T22:15:36.540168+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:15:36.540168+0000","last_undegraded":"2026-02-21T22:15:36.540168+0000","last_fullsized":"2026-02-21T22:15:36.540168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8772,"internal_metadata":27581884},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":156,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":159744,"data_stored":48640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8786,"internal_metadata":27581870},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:38.675 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:38.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.729+0000 7f769ad99640 10 monclient: get_monmap_and_config 2026-02-21T22:15:38.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.729+0000 7f769ad99640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.729+0000 7f769ad99640 10 monclient: monmap: 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:38.736422+0000 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:38.736422+0000 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:38.737 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.729+0000 7f769ad99640 1 Processor -- start 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- start start 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient: init 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-b con 0x7f7694091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-a con 0x7f76940909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-c con 0x7f76941854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7694057a20 con 0x7f76940909a0 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7694091d20 con 0x7f76941854b0 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7694057740 con 0x7f7694091240 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient(hunting): authenticate will time out at 1147.972328s 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 0x7f7694183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58574/0 (socket says 10.20.193.102:58574) 2026-02-21T22:15:38.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768bfff640 1 -- 10.20.193.102:0/196741784 learned_addr learned my addr 10.20.193.102:0/196741784 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768bfff640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 legacy=0x7f7694183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f7698b0e640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.102:6790/0 conn(0x7f7694091240 legacy=0x7f7694091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890605709 0 0) 0x7f7694057a20 con 0x7f76940909a0 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): my global_id is 4311 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769930f640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.178:6789/0 conn(0x7f76941854b0 legacy=0x7f76941a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7680003620 con 0x7f76940909a0 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755916236 0 0) 0x7f7694057740 con 0x7f7694091240 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient(hunting): my global_id is 4310 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7694057a20 con 0x7f7694091240 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2746613718 0 0) 0x7f7694057a20 con 0x7f7694091240 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.178:6789/0 conn(0x7f76941854b0 legacy=0x7f76941a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 legacy=0x7f7694183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 monclient: found mon.noname-b 2026-02-21T22:15:38.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7694090dc0 con 0x7f7694091240 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76840030e0 con 0x7f7694091240 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: dump: 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.746 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 5 monclient: authenticate success, global_id 4310 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f76840033b0 con 0x7f7694091240 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 1 -- 10.20.193.102:0/196741784 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76840038c0 con 0x7f7694091240 2026-02-21T22:15:38.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f768b7fe640 10 monclient: dump: 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 4 set_mon_vals no callback set 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 10 monclient: shutdown 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- 10.20.193.102:0/196741784 >> v1:10.20.193.102:6790/0 conn(0x7f7694091240 legacy=0x7f7694091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- 10.20.193.102:0/196741784 shutdown_connections 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- 10.20.193.102:0/196741784 >> 10.20.193.102:0/196741784 conn(0x7f769405e660 msgr2=0x7f7694095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- 10.20.193.102:0/196741784 shutdown_connections 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.733+0000 7f769ad99640 1 -- 10.20.193.102:0/196741784 wait complete. 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient: monmap: 2026-02-21T22:15:38.749 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 1 Processor -- start 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 1 -- start start 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient: init 2026-02-21T22:15:38.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-a con 0x7f76940909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-b con 0x7f7694091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient(hunting): picked mon.noname-c con 0x7f76941854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769412bbf0 con 0x7f76940909a0 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7694112e40 con 0x7f7694091240 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7694111fd0 con 0x7f76941854b0 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769ad99640 10 monclient(hunting): authenticate will time out at 1147.976651s 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f7698b0e640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 0x7f769412b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58580/0 (socket says 10.20.193.102:58580) 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f7698b0e640 1 -- 10.20.193.102:0/3863985333 learned_addr learned my addr 10.20.193.102:0/3863985333 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f7698b0e640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 legacy=0x7f769412b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f769930f640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.102:6790/0 conn(0x7f76941854b0 legacy=0x7f7694173fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f768bfff640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.178:6789/0 conn(0x7f7694091240 legacy=0x7f76941728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.737+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1889386068 0 0) 0x7f769412bbf0 con 0x7f76940909a0 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): my global_id is 4314 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7664003620 con 0x7f76940909a0 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337333601 0 0) 0x7f7694112e40 con 0x7f7694091240 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): my global_id is 4291 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f769412bbf0 con 0x7f7694091240 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362307141 0 0) 0x7f7694111fd0 con 0x7f76941854b0 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): my global_id is 4313 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7694112e40 con 0x7f76941854b0 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1374851121 0 0) 0x7f7664003620 con 0x7f76940909a0 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7694111fd0 con 0x7f76940909a0 2026-02-21T22:15:38.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7684003710 con 0x7f76940909a0 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): dump: 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.758 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2849865093 0 0) 0x7f7694112e40 con 0x7f76941854b0 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7664003620 con 0x7f76941854b0 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7690004890 con 0x7f76941854b0 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): dump: 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.759 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3202691521 0 0) 0x7f769412bbf0 con 0x7f7694091240 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7694112e40 con 0x7f7694091240 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f767c0048e0 con 0x7f7694091240 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient(hunting): dump: 2026-02-21T22:15:38.760 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2676255934 0 0) 0x7f7694111fd0 con 0x7f76940909a0 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.102:6790/0 conn(0x7f76941854b0 legacy=0x7f7694173fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.178:6789/0 conn(0x7f7694091240 legacy=0x7f76941728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 monclient: found mon.a 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7694110ae0 con 0x7f76940909a0 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 5 monclient: authenticate success, global_id 4314 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _renew_subs 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:38.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f76941125a0 con 0x7f76940909a0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _renew_subs 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7694111810 con 0x7f76940909a0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7684002d10 con 0x7f76940909a0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7684003ee0 con 0x7f76940909a0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 10 monclient: dump: 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.763 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f768affd640 4 set_mon_vals no callback set 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7694110530 con 0x7f76940909a0 2026-02-21T22:15:38.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f76840041b0 con 0x7f76940909a0 2026-02-21T22:15:38.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f768404c4f0 con 0x7f76940909a0 2026-02-21T22:15:38.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.741+0000 7f768bfff640 10 monclient: get_auth_request con 0x7f766403da20 auth_method 0 2026-02-21T22:15:38.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.745+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f7684017660 con 0x7f76940909a0 2026-02-21T22:15:38.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.745+0000 7f76897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.745+0000 7f76897fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:38.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f769405eb70 con 0x7f766403da20 2026-02-21T22:15:38.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f76897fa640 1 -- 10.20.193.102:0/3863985333 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34187 (unknown 2965378022 0 146340977) 0x7f769405eb70 con 0x7f766403da20 2026-02-21T22:15:38.836 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:38.837 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:15:38.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.178:6800/2549949439 conn(0x7f766403da20 legacy=0x7f766405ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 10 monclient: shutdown 2026-02-21T22:15:38.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 >> v1:10.20.193.102:6789/0 conn(0x7f76940909a0 legacy=0x7f769412b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 shutdown_connections 2026-02-21T22:15:38.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 >> 10.20.193.102:0/3863985333 conn(0x7f769405e660 msgr2=0x7f7694089c70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 shutdown_connections 2026-02-21T22:15:38.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.829+0000 7f769ad99640 1 -- 10.20.193.102:0/3863985333 wait complete. 2026-02-21T22:15:38.844 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-21T22:15:37.487922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":162832,"kb_used_data":1072,"kb_used_omap":48,"kb_used_meta":161615,"kb_avail":566068208,"statfs":{"total":579820584960,"available":579653844992,"internally_reserved":0,"allocated":1097728,"data_stored":1484939,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":50058,"internal_metadata":165493878},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000431"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540434+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:15:36.540434+0000","last_peered":"2026-02-21T22:15:36.540434+0000","last_clean":"2026-02-21T22:15:36.540434+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:15:36.540434+0000","last_undegraded":"2026-02-21T22:15:36.540434+0000","last_fullsized":"2026-02-21T22:15:36.540434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537959+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537959+0000","last_peered":"2026-02-21T22:15:36.537959+0000","last_clean":"2026-02-21T22:15:36.537959+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:15:36.537959+0000","last_undegraded":"2026-02-21T22:15:36.537959+0000","last_fullsized":"2026-02-21T22:15:36.537959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540562+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:15:36.540562+0000","last_peered":"2026-02-21T22:15:36.540562+0000","last_clean":"2026-02-21T22:15:36.540562+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:15:36.540562+0000","last_undegraded":"2026-02-21T22:15:36.540562+0000","last_fullsized":"2026-02-21T22:15:36.540562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537987+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537987+0000","last_peered":"2026-02-21T22:15:36.537987+0000","last_clean":"2026-02-21T22:15:36.537987+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:15:36.537987+0000","last_undegraded":"2026-02-21T22:15:36.537987+0000","last_fullsized":"2026-02-21T22:15:36.537987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.552137+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:15:36.552137+0000","last_peered":"2026-02-21T22:15:36.552137+0000","last_clean":"2026-02-21T22:15:36.552137+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:15:36.552137+0000","last_undegraded":"2026-02-21T22:15:36.552137+0000","last_fullsized":"2026-02-21T22:15:36.552137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.538007+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:15:36.538007+0000","last_peered":"2026-02-21T22:15:36.538007+0000","last_clean":"2026-02-21T22:15:36.538007+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:15:36.538007+0000","last_undegraded":"2026-02-21T22:15:36.538007+0000","last_fullsized":"2026-02-21T22:15:36.538007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540688+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:15:36.540688+0000","last_peered":"2026-02-21T22:15:36.540688+0000","last_clean":"2026-02-21T22:15:36.540688+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:15:36.540688+0000","last_undegraded":"2026-02-21T22:15:36.540688+0000","last_fullsized":"2026-02-21T22:15:36.540688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.542334+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:15:36.542334+0000","last_peered":"2026-02-21T22:15:36.542334+0000","last_clean":"2026-02-21T22:15:36.542334+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:15:36.542334+0000","last_undegraded":"2026-02-21T22:15:36.542334+0000","last_fullsized":"2026-02-21T22:15:36.542334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'112","reported_seq":157,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540168+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:15:36.540168+0000","last_peered":"2026-02-21T22:15:36.540168+0000","last_clean":"2026-02-21T22:15:36.540168+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:15:36.540168+0000","last_undegraded":"2026-02-21T22:15:36.540168+0000","last_fullsized":"2026-02-21T22:15:36.540168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640259,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8772,"internal_metadata":27581884},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640259,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":156,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":159744,"data_stored":48640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8786,"internal_metadata":27581870},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:38.845 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T22:15:38.845 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T22:15:38.845 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T22:15:38.845 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: get_monmap_and_config 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: monmap: 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:38.911375+0000 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:38.911375+0000 2026-02-21T22:15:38.911 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.912 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.912 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.912 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:38.912 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:38.912 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 1 Processor -- start 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 1 -- start start 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: init 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient(hunting): picked mon.noname-b con 0x7f745c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient(hunting): picked mon.noname-a con 0x7f745c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient(hunting): picked mon.noname-c con 0x7f745c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c086a10 con 0x7f745c08b7f0 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c08b130 con 0x7f745c08d8e0 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c08bd60 con 0x7f745c08ce00 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7463184640 10 monclient(hunting): authenticate will time out at 1148.147548s 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7453fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f745c08b7f0 0x7f745c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58590/0 (socket says 10.20.193.102:58590) 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7460ef9640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f745c08ce00 0x7f745c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60388/0 (socket says 10.20.193.102:60388) 2026-02-21T22:15:38.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.905+0000 7f7453fff640 1 -- 10.20.193.102:0/3698723226 learned_addr learned my addr 10.20.193.102:0/3698723226 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7460ef9640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.102:6790/0 conn(0x7f745c08ce00 legacy=0x7f745c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7453fff640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.102:6789/0 conn(0x7f745c08b7f0 legacy=0x7f745c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557775509 0 0) 0x7f745c08bd60 con 0x7f745c08ce00 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): my global_id is 4316 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7448003620 con 0x7f745c08ce00 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120404980 0 0) 0x7f745c086a10 con 0x7f745c08b7f0 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient(hunting): my global_id is 4317 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f745c08bd60 con 0x7f745c08b7f0 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74616fa640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.178:6789/0 conn(0x7f745c08d8e0 legacy=0x7f745c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3658054794 0 0) 0x7f7448003620 con 0x7f745c08ce00 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.178:6789/0 conn(0x7f745c08d8e0 legacy=0x7f745c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.102:6789/0 conn(0x7f745c08b7f0 legacy=0x7f745c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 monclient: found mon.noname-b 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c08b370 con 0x7f745c08ce00 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f744c003190 con 0x7f745c08ce00 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 5 monclient: authenticate success, global_id 4316 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: dump: 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:38.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f744c003420 con 0x7f745c08ce00 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 1 -- 10.20.193.102:0/3698723226 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f744c004a90 con 0x7f745c08ce00 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f74537fe640 10 monclient: dump: 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 4 set_mon_vals no callback set 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 10 monclient: shutdown 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- 10.20.193.102:0/3698723226 >> v1:10.20.193.102:6790/0 conn(0x7f745c08ce00 legacy=0x7f745c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- 10.20.193.102:0/3698723226 shutdown_connections 2026-02-21T22:15:38.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- 10.20.193.102:0/3698723226 >> 10.20.193.102:0/3698723226 conn(0x7f745c091a50 msgr2=0x7f745c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:38.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- 10.20.193.102:0/3698723226 shutdown_connections 2026-02-21T22:15:38.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- 10.20.193.102:0/3698723226 wait complete. 2026-02-21T22:15:38.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 10 monclient: build_initial_monmap 2026-02-21T22:15:38.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 10 monclient: monmap: 2026-02-21T22:15:38.921 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:38.921 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 Processor -- start 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 1 -- start start 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.909+0000 7f7463184640 10 monclient: init 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:38.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient(hunting): picked mon.noname-c con 0x7f745c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient(hunting): picked mon.noname-b con 0x7f745c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient(hunting): picked mon.noname-a con 0x7f745c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c05e150 con 0x7f745c08d8e0 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c07ee10 con 0x7f745c08ce00 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745c081170 con 0x7f745c08b7f0 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient(hunting): authenticate will time out at 1148.152153s 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7460ef9640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f745c08b7f0 0x7f745c0846c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60392/0 (socket says 10.20.193.102:60392) 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7460ef9640 1 -- 10.20.193.102:0/1246236646 learned_addr learned my addr 10.20.193.102:0/1246236646 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:38.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7460ef9640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.102:6790/0 conn(0x7f745c08b7f0 legacy=0x7f745c0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74616fa640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.102:6789/0 conn(0x7f745c08d8e0 legacy=0x7f745c193a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313928418 0 0) 0x7f745c081170 con 0x7f745c08b7f0 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): my global_id is 4319 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f742c003620 con 0x7f745c08b7f0 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141299416 0 0) 0x7f745c05e150 con 0x7f745c08d8e0 2026-02-21T22:15:38.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): my global_id is 4320 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f745c081170 con 0x7f745c08d8e0 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7453fff640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.178:6789/0 conn(0x7f745c08ce00 legacy=0x7f745c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1326093359 0 0) 0x7f742c003620 con 0x7f745c08b7f0 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f745c05e150 con 0x7f745c08b7f0 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f744c004a90 con 0x7f745c08b7f0 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): dump: 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.926 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 724377489 0 0) 0x7f745c081170 con 0x7f745c08d8e0 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f742c003620 con 0x7f745c08d8e0 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7458004890 con 0x7f745c08d8e0 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient(hunting): dump: 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.927 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2484726124 0 0) 0x7f745c05e150 con 0x7f745c08b7f0 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.178:6789/0 conn(0x7f745c08ce00 legacy=0x7f745c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.102:6789/0 conn(0x7f745c08d8e0 legacy=0x7f745c193a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 monclient: found mon.c 2026-02-21T22:15:38.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c0808d0 con 0x7f745c08b7f0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 5 monclient: authenticate success, global_id 4319 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _renew_subs 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f745c084dd0 con 0x7f745c08b7f0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _renew_subs 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f745c07fb40 con 0x7f745c08b7f0 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f744c0033b0 con 0x7f745c08b7f0 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7452ffd640 4 set_mon_vals no callback set 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f744c005260 con 0x7f745c08b7f0 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 10 monclient: dump: 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:38.930 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0008e0 con 0x7f745c08b7f0 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f744c005480 con 0x7f745c08b7f0 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f744c04c700 con 0x7f745c08b7f0 2026-02-21T22:15:38.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.913+0000 7f7453fff640 10 monclient: get_auth_request con 0x7f742c03d610 auth_method 0 2026-02-21T22:15:38.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.917+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f744c017810 con 0x7f745c08b7f0 2026-02-21T22:15:38.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.917+0000 7f74517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:38.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:38.917+0000 7f74517fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f7463184640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f7463184640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f7463184640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f745c090810 con 0x7f745c08b7f0 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f74517fa640 1 -- 10.20.193.102:0/1246236646 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f744c01f0b0 con 0x7f745c08b7f0 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f74517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.021+0000 7f74517fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:39.028 INFO:teuthology.orchestra.run.trial102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.178:6800/2549949439 conn(0x7f742c03d610 legacy=0x7f742c05dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 10 monclient: shutdown 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 >> v1:10.20.193.102:6790/0 conn(0x7f745c08b7f0 legacy=0x7f745c0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 shutdown_connections 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 >> 10.20.193.102:0/1246236646 conn(0x7f745c091a50 msgr2=0x7f745c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 shutdown_connections 2026-02-21T22:15:39.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.025+0000 7f7463184640 1 -- 10.20.193.102:0/1246236646 wait complete. 2026-02-21T22:15:39.038 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T22:15:39.038 INFO:teuthology.run_tasks:Running task mon_thrash... 2026-02-21T22:15:39.052 INFO:tasks.mon_thrash:Beginning mon_thrash... 2026-02-21T22:15:39.052 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:15:39.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: monmap: 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.114292+0000 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.114292+0000 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.116 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 1 Processor -- start 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 1 -- start start 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: init 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-b con 0x7f9a4805d590 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-a con 0x7f9a48140b40 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-c con 0x7f9a48183360 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a48140fa0 con 0x7f9a48140b40 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4805ae70 con 0x7f9a48183360 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a48059930 con 0x7f9a4805d590 2026-02-21T22:15:39.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4f737640 10 monclient(hunting): authenticate will time out at 1148.350221s 2026-02-21T22:15:39.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4ccab640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9a48140b40 0x7f9a48181c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58600/0 (socket says 10.20.193.102:58600) 2026-02-21T22:15:39.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.109+0000 7f9a4ccab640 1 -- 10.20.193.102:0/2668842226 learned_addr learned my addr 10.20.193.102:0/2668842226 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4ccab640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.102:6789/0 conn(0x7f9a48140b40 legacy=0x7f9a48181c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4d4ac640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.102:6790/0 conn(0x7f9a4805d590 legacy=0x7f9a4805d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4dcad640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.178:6789/0 conn(0x7f9a48183360 legacy=0x7f9a481a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388233219 0 0) 0x7f9a48140fa0 con 0x7f9a48140b40 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): my global_id is 4323 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a30003620 con 0x7f9a48140b40 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392863345 0 0) 0x7f9a48059930 con 0x7f9a4805d590 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): my global_id is 4322 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a48140fa0 con 0x7f9a4805d590 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343510738 0 0) 0x7f9a4805ae70 con 0x7f9a48183360 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient(hunting): my global_id is 4300 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a48059930 con 0x7f9a48183360 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087034983 0 0) 0x7f9a30003620 con 0x7f9a48140b40 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.102:6790/0 conn(0x7f9a4805d590 legacy=0x7f9a4805d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.178:6789/0 conn(0x7f9a48183360 legacy=0x7f9a481a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 monclient: found mon.noname-a 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4805a8b0 con 0x7f9a48140b40 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 5 monclient: authenticate success, global_id 4323 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9a3c003110 con 0x7f9a48140b40 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: dump: 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.128 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9a3c0033e0 con 0x7f9a48140b40 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 1 -- 10.20.193.102:0/2668842226 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9a3c004b50 con 0x7f9a48140b40 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a37fff640 10 monclient: dump: 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.130 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 4 set_mon_vals no callback set 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: shutdown 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- 10.20.193.102:0/2668842226 >> v1:10.20.193.102:6789/0 conn(0x7f9a48140b40 legacy=0x7f9a48181c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- 10.20.193.102:0/2668842226 shutdown_connections 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- 10.20.193.102:0/2668842226 >> 10.20.193.102:0/2668842226 conn(0x7f9a48056b30 msgr2=0x7f9a48093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- 10.20.193.102:0/2668842226 shutdown_connections 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- 10.20.193.102:0/2668842226 wait complete. 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: monmap: 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.131 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 Processor -- start 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- start start 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: init 2026-02-21T22:15:39.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-a con 0x7f9a4805d590 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-c con 0x7f9a48140b40 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient(hunting): picked mon.noname-b con 0x7f9a48183360 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4812b5e0 con 0x7f9a4805d590 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4805d150 con 0x7f9a48183360 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4805ea30 con 0x7f9a48140b40 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4f737640 10 monclient(hunting): authenticate will time out at 1148.354853s 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4d4ac640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9a4805d590 0x7f9a4819dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58602/0 (socket says 10.20.193.102:58602) 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.113+0000 7f9a4d4ac640 1 -- 10.20.193.102:0/1712883896 learned_addr learned my addr 10.20.193.102:0/1712883896 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4d4ac640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.102:6789/0 conn(0x7f9a4805d590 legacy=0x7f9a4819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4ccab640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.102:6790/0 conn(0x7f9a48140b40 legacy=0x7f9a48181980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499916268 0 0) 0x7f9a4812b5e0 con 0x7f9a4805d590 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): my global_id is 4326 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a18003620 con 0x7f9a4805d590 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920707372 0 0) 0x7f9a4805ea30 con 0x7f9a48140b40 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): my global_id is 4325 2026-02-21T22:15:39.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a4812b5e0 con 0x7f9a48140b40 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4dcad640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.178:6789/0 conn(0x7f9a48183360 legacy=0x7f9a4812fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 794564446 0 0) 0x7f9a18003620 con 0x7f9a4805d590 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a4805ea30 con 0x7f9a4805d590 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9a38003710 con 0x7f9a4805d590 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): dump: 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.135 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2819282707 0 0) 0x7f9a4812b5e0 con 0x7f9a48140b40 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a18003620 con 0x7f9a48140b40 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9a3c0048e0 con 0x7f9a48140b40 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): dump: 2026-02-21T22:15:39.136 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839466876 0 0) 0x7f9a4805d150 con 0x7f9a48183360 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient(hunting): my global_id is 4303 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a4812b5e0 con 0x7f9a48183360 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4246473431 0 0) 0x7f9a4805ea30 con 0x7f9a4805d590 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.102:6790/0 conn(0x7f9a48140b40 legacy=0x7f9a48181980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.178:6789/0 conn(0x7f9a48183360 legacy=0x7f9a4812fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 monclient: found mon.a 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a481a3dc0 con 0x7f9a4805d590 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 5 monclient: authenticate success, global_id 4326 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _renew_subs 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4812b300 con 0x7f9a4805d590 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _renew_subs 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9a481a4920 con 0x7f9a4805d590 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9a38003c90 con 0x7f9a4805d590 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9a38003ff0 con 0x7f9a4805d590 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 10 monclient: dump: 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a377fe640 4 set_mon_vals no callback set 2026-02-21T22:15:39.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9a380041d0 con 0x7f9a4805d590 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4808da00 con 0x7f9a4805d590 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f9a3804c560 con 0x7f9a4805d590 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.117+0000 7f9a4ccab640 10 monclient: get_auth_request con 0x7f9a1803d820 auth_method 0 2026-02-21T22:15:39.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.121+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9a38017750 con 0x7f9a4805d590 2026-02-21T22:15:39.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.121+0000 7f9a35ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.121+0000 7f9a35ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a48062ce0 con 0x7f9a4805d590 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a35ffb640 1 -- 10.20.193.102:0/1712883896 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+11009 (unknown 941590788 0 586439007) 0x7f9a38017030 con 0x7f9a4805d590 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a35ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a35ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:39.210 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":19,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:15:36.532998+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T22:15:33.157956+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9a1803d820 legacy=0x7f9a1805dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 10 monclient: shutdown 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 >> v1:10.20.193.102:6789/0 conn(0x7f9a4805d590 legacy=0x7f9a4819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 shutdown_connections 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 >> 10.20.193.102:0/1712883896 conn(0x7f9a48056b30 msgr2=0x7f9a4808d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 shutdown_connections 2026-02-21T22:15:39.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.205+0000 7f9a4f737640 1 -- 10.20.193.102:0/1712883896 wait complete. 2026-02-21T22:15:39.220 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-21T22:15:29.531133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2026-02-21T22:15:33.157956+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-21T22:15:39.220 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: monmap: 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.283559+0000 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.283559+0000 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.283 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 1 Processor -- start 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 1 -- start start 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: init 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-a con 0x7fdd70084940 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-b con 0x7fdd70086d00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-c con 0x7fdd70087140 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd70086040 con 0x7fdd70084940 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd70086320 con 0x7fdd70087140 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd70086600 con 0x7fdd70086d00 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd751dc640 10 monclient(hunting): authenticate will time out at 1148.519453s 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd6ed76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 0x7fdd70084d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58608/0 (socket says 10.20.193.102:58608) 2026-02-21T22:15:39.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.277+0000 7fdd6ed76640 1 -- 10.20.193.102:0/3082198857 learned_addr learned my addr 10.20.193.102:0/3082198857 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6ed76640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 legacy=0x7fdd70084d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6e575640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.102:6790/0 conn(0x7fdd70086d00 legacy=0x7fdd70181bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207752837 0 0) 0x7fdd70086040 con 0x7fdd70084940 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6f577640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.178:6789/0 conn(0x7fdd70087140 legacy=0x7fdd701a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): my global_id is 4329 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd5c003620 con 0x7fdd70084940 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 289821907 0 0) 0x7fdd70086600 con 0x7fdd70086d00 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient(hunting): my global_id is 4328 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd70086040 con 0x7fdd70086d00 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758280458 0 0) 0x7fdd5c003620 con 0x7fdd70084940 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.102:6790/0 conn(0x7fdd70086d00 legacy=0x7fdd70181bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.178:6789/0 conn(0x7fdd70087140 legacy=0x7fdd701a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 monclient: found mon.noname-a 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7008b1e0 con 0x7fdd70084940 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd64003110 con 0x7fdd70084940 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: dump: 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.287 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 5 monclient: authenticate success, global_id 4329 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd64003450 con 0x7fdd70084940 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 1 -- 10.20.193.102:0/3082198857 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd64004b50 con 0x7fdd70084940 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6dd74640 10 monclient: dump: 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 4 set_mon_vals no callback set 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: shutdown 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- 10.20.193.102:0/3082198857 >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 legacy=0x7fdd70084d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- 10.20.193.102:0/3082198857 shutdown_connections 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- 10.20.193.102:0/3082198857 >> 10.20.193.102:0/3082198857 conn(0x7fdd7008e470 msgr2=0x7fdd7008e870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- 10.20.193.102:0/3082198857 shutdown_connections 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- 10.20.193.102:0/3082198857 wait complete. 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: monmap: 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 Processor -- start 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- start start 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: init 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-a con 0x7fdd70084940 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-b con 0x7fdd70086d00 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient(hunting): picked mon.noname-c con 0x7fdd70087140 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd701a2cc0 con 0x7fdd70084940 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd701a2fa0 con 0x7fdd70086d00 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd701db1c0 con 0x7fdd70087140 2026-02-21T22:15:39.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd751dc640 10 monclient(hunting): authenticate will time out at 1148.523511s 2026-02-21T22:15:39.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6ed76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 0x7fdd701da3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58612/0 (socket says 10.20.193.102:58612) 2026-02-21T22:15:39.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.281+0000 7fdd6ed76640 1 -- 10.20.193.102:0/3938857899 learned_addr learned my addr 10.20.193.102:0/3938857899 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd6ed76640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 legacy=0x7fdd701da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd6f577640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.102:6790/0 conn(0x7fdd70087140 legacy=0x7fdd7023c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012968115 0 0) 0x7fdd701a2cc0 con 0x7fdd70084940 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): my global_id is 4332 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd3c003620 con 0x7fdd70084940 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 176019609 0 0) 0x7fdd701db1c0 con 0x7fdd70087140 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): my global_id is 4331 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd6e575640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.178:6789/0 conn(0x7fdd70086d00 legacy=0x7fdd701daab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd701a2cc0 con 0x7fdd70087140 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 603055919 0 0) 0x7fdd3c003620 con 0x7fdd70084940 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd701db1c0 con 0x7fdd70084940 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd64002b70 con 0x7fdd70084940 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.291 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229206965 0 0) 0x7fdd701a2fa0 con 0x7fdd70086d00 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): my global_id is 4309 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd3c003620 con 0x7fdd70086d00 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1619220788 0 0) 0x7fdd701a2cc0 con 0x7fdd70087140 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd701a2fa0 con 0x7fdd70087140 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd60004890 con 0x7fdd70087140 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 220742447 0 0) 0x7fdd701db1c0 con 0x7fdd70084940 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.102:6790/0 conn(0x7fdd70087140 legacy=0x7fdd7023c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.178:6789/0 conn(0x7fdd70086d00 legacy=0x7fdd701daab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 monclient: found mon.a 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd701a19c0 con 0x7fdd70084940 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 5 monclient: authenticate success, global_id 4332 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _renew_subs 2026-02-21T22:15:39.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdd70088ca0 con 0x7fdd70084940 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _renew_subs 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdd701a2700 con 0x7fdd70084940 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd64003380 con 0x7fdd70084940 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd640052e0 con 0x7fdd70084940 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 10 monclient: dump: 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd6d573640 4 set_mon_vals no callback set 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdd64011970 con 0x7fdd70084940 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd700008e0 con 0x7fdd70084940 2026-02-21T22:15:39.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fdd6404c530 con 0x7fdd70084940 2026-02-21T22:15:39.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.285+0000 7fdd6e575640 10 monclient: get_auth_request con 0x7fdd3c03d880 auth_method 0 2026-02-21T22:15:39.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.289+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fdd64017760 con 0x7fdd70084940 2026-02-21T22:15:39.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.289+0000 7fdd4f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.289+0000 7fdd4f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd70090810 con 0x7fdd70084940 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd4f7fe640 1 -- 10.20.193.102:0/3938857899 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (unknown 4170632370 0 2170607528) 0x7fdd64017040 con 0x7fdd70084940 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd4f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2026-02-21T22:15:39.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd4f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stdout:pg_num: 1 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.178:6800/2549949439 conn(0x7fdd3c03d880 legacy=0x7fdd3c05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 10 monclient: shutdown 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 >> v1:10.20.193.102:6789/0 conn(0x7fdd70084940 legacy=0x7fdd701da3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 shutdown_connections 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 >> 10.20.193.102:0/3938857899 conn(0x7fdd7008e470 msgr2=0x7fdd70054480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 shutdown_connections 2026-02-21T22:15:39.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.373+0000 7fdd751dc640 1 -- 10.20.193.102:0/3938857899 wait complete. 2026-02-21T22:15:39.388 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2026-02-21T22:15:39.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: monmap: 2026-02-21T22:15:39.448 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.448840+0000 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.448840+0000 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 1 Processor -- start 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 1 -- start start 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: init 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-b con 0x7f1e30084760 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-c con 0x7f1e3008c3c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-a con 0x7f1e301855f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e30089860 con 0x7f1e301855f0 2026-02-21T22:15:39.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3008a440 con 0x7f1e3008c3c0 2026-02-21T22:15:39.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3008a720 con 0x7f1e30084760 2026-02-21T22:15:39.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient(hunting): authenticate will time out at 1148.684739s 2026-02-21T22:15:39.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2ed76640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f1e30084760 0x7f1e30084b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60432/0 (socket says 10.20.193.102:60432) 2026-02-21T22:15:39.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2ed76640 1 -- 10.20.193.102:0/4286321023 learned_addr learned my addr 10.20.193.102:0/4286321023 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2ed76640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.102:6790/0 conn(0x7f1e30084760 legacy=0x7f1e30084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2f577640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.102:6789/0 conn(0x7f1e301855f0 legacy=0x7f1e301a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2e575640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.178:6789/0 conn(0x7f1e3008c3c0 legacy=0x7f1e30183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318181723 0 0) 0x7f1e3008a720 con 0x7f1e30084760 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): my global_id is 4334 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e1c003620 con 0x7f1e30084760 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3655691119 0 0) 0x7f1e30089860 con 0x7f1e301855f0 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): my global_id is 4335 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e3008a720 con 0x7f1e301855f0 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1331310652 0 0) 0x7f1e3008a440 con 0x7f1e3008c3c0 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient(hunting): my global_id is 4312 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e30089860 con 0x7f1e3008c3c0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214155858 0 0) 0x7f1e3008a720 con 0x7f1e301855f0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.102:6790/0 conn(0x7f1e30084760 legacy=0x7f1e30084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.178:6789/0 conn(0x7f1e3008c3c0 legacy=0x7f1e30183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 monclient: found mon.noname-a 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e30085240 con 0x7f1e301855f0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e200045a0 con 0x7f1e301855f0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: dump: 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 5 monclient: authenticate success, global_id 4335 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1e20004870 con 0x7f1e301855f0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 1 -- 10.20.193.102:0/4286321023 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e20004c20 con 0x7f1e301855f0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e2dd74640 10 monclient: dump: 2026-02-21T22:15:39.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.445+0000 7f1e35220640 4 set_mon_vals no callback set 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: shutdown 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/4286321023 >> v1:10.20.193.102:6789/0 conn(0x7f1e301855f0 legacy=0x7f1e301a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/4286321023 shutdown_connections 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/4286321023 >> 10.20.193.102:0/4286321023 conn(0x7f1e3008e470 msgr2=0x7f1e3008e870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/4286321023 shutdown_connections 2026-02-21T22:15:39.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/4286321023 wait complete. 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: monmap: 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 Processor -- start 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- start start 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: init 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-b con 0x7f1e30084760 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-a con 0x7f1e3008c3c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient(hunting): picked mon.noname-c con 0x7f1e301855f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e300883a0 con 0x7f1e3008c3c0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3005d1e0 con 0x7f1e30084760 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3007eab0 con 0x7f1e301855f0 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient(hunting): authenticate will time out at 1148.689008s 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2e575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1e3008c3c0 0x7f1e30172c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58638/0 (socket says 10.20.193.102:58638) 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2e575640 1 -- 10.20.193.102:0/3324295162 learned_addr learned my addr 10.20.193.102:0/3324295162 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2f577640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.102:6790/0 conn(0x7f1e301855f0 legacy=0x7f1e30174370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2e575640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.102:6789/0 conn(0x7f1e3008c3c0 legacy=0x7f1e30172c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750896752 0 0) 0x7f1e3007eab0 con 0x7f1e301855f0 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): my global_id is 4337 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2ed76640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.178:6789/0 conn(0x7f1e30084760 legacy=0x7f1e30131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dfc003620 con 0x7f1e301855f0 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123565037 0 0) 0x7f1e300883a0 con 0x7f1e3008c3c0 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): my global_id is 4338 2026-02-21T22:15:39.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e3007eab0 con 0x7f1e3008c3c0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 272138727 0 0) 0x7f1dfc003620 con 0x7f1e301855f0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e300883a0 con 0x7f1e301855f0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e20004000 con 0x7f1e301855f0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2501716384 0 0) 0x7f1e3005d1e0 con 0x7f1e30084760 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): my global_id is 4315 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dfc003620 con 0x7f1e30084760 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4225820482 0 0) 0x7f1e3007eab0 con 0x7f1e3008c3c0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e3005d1e0 con 0x7f1e3008c3c0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e18003490 con 0x7f1e3008c3c0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient(hunting): dump: 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1779903926 0 0) 0x7f1e300883a0 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.178:6789/0 conn(0x7f1e30084760 legacy=0x7f1e30131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.102:6789/0 conn(0x7f1e3008c3c0 legacy=0x7f1e30172c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 monclient: found mon.c 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e30080e10 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 5 monclient: authenticate success, global_id 4337 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _renew_subs 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1e30080b30 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _renew_subs 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f1e301326d0 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1e20004870 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e200053b0 con 0x7f1e301855f0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e0f7fe640 10 monclient: dump: 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.458 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.449+0000 7f1e2d573640 4 set_mon_vals no callback set 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e35220640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e35220640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e35220640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e300008e0 con 0x7f1e301855f0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1e20005650 con 0x7f1e301855f0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f1e2004d7c0 con 0x7f1e301855f0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e2ed76640 10 monclient: get_auth_request con 0x7f1dfc03d900 auth_method 0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f1e20018930 con 0x7f1e301855f0 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e0f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.453+0000 7f1e0f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7f1e30090810 con 0x7f1e301855f0 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e0f7fe640 1 -- 10.20.193.102:0/3324295162 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) ==== 92+0+10 (unknown 1398289234 0 265718887) 0x7f1e20020170 con 0x7f1e301855f0 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e0f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2026-02-21T22:15:39.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e0f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stdout:pg_num: 8 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.178:6800/2549949439 conn(0x7f1dfc03d900 legacy=0x7f1dfc05ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 10 monclient: shutdown 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 >> v1:10.20.193.102:6790/0 conn(0x7f1e301855f0 legacy=0x7f1e30174370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 shutdown_connections 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 >> 10.20.193.102:0/3324295162 conn(0x7f1e3008e470 msgr2=0x7f1e3005bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 shutdown_connections 2026-02-21T22:15:39.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.537+0000 7f1e35220640 1 -- 10.20.193.102:0/3324295162 wait complete. 2026-02-21T22:15:39.552 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2026-02-21T22:15:39.553 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 10 monclient: monmap: 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.620339+0000 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.620339+0000 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.620 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 1 Processor -- start 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 1 -- start start 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.613+0000 7ff223d81640 10 monclient: init 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-a con 0x7ff21c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-b con 0x7ff21c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-c con 0x7ff21c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c086a10 con 0x7ff21c08ce00 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c08b130 con 0x7ff21c08d8e0 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c08bd60 con 0x7ff21c08b7f0 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient(hunting): authenticate will time out at 1148.856306s 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff221af6640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 0x7ff21c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58654/0 (socket says 10.20.193.102:58654) 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff221af6640 1 -- 10.20.193.102:0/1049612576 learned_addr learned my addr 10.20.193.102:0/1049612576 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff2212f5640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff21c08b7f0 0x7ff21c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60436/0 (socket says 10.20.193.102:60436) 2026-02-21T22:15:39.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff221af6640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 legacy=0x7ff21c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff2212f5640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.102:6790/0 conn(0x7ff21c08b7f0 legacy=0x7ff21c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff2222f7640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.178:6789/0 conn(0x7ff21c08d8e0 legacy=0x7ff21c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996730259 0 0) 0x7ff21c086a10 con 0x7ff21c08ce00 2026-02-21T22:15:39.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): my global_id is 4341 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff204003620 con 0x7ff21c08ce00 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189079574 0 0) 0x7ff21c08bd60 con 0x7ff21c08b7f0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): my global_id is 4340 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff21c086a10 con 0x7ff21c08b7f0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2760253626 0 0) 0x7ff21c08b130 con 0x7ff21c08d8e0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient(hunting): my global_id is 4318 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff21c08bd60 con 0x7ff21c08d8e0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 66032755 0 0) 0x7ff21c086a10 con 0x7ff21c08b7f0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.178:6789/0 conn(0x7ff21c08d8e0 legacy=0x7ff21c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 legacy=0x7ff21c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 monclient: found mon.noname-b 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c08b370 con 0x7ff21c08b7f0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff210003110 con 0x7ff21c08b7f0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: dump: 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.624 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 5 monclient: authenticate success, global_id 4340 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2100033e0 con 0x7ff21c08b7f0 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 1 -- 10.20.193.102:0/1049612576 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff210004a90 con 0x7ff21c08b7f0 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff220af4640 10 monclient: dump: 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 4 set_mon_vals no callback set 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 10 monclient: shutdown 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- 10.20.193.102:0/1049612576 >> v1:10.20.193.102:6790/0 conn(0x7ff21c08b7f0 legacy=0x7ff21c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- 10.20.193.102:0/1049612576 shutdown_connections 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- 10.20.193.102:0/1049612576 >> 10.20.193.102:0/1049612576 conn(0x7ff21c091a50 msgr2=0x7ff21c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- 10.20.193.102:0/1049612576 shutdown_connections 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.617+0000 7ff223d81640 1 -- 10.20.193.102:0/1049612576 wait complete. 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: monmap: 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 Processor -- start 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- start start 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: init 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-b con 0x7ff21c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-a con 0x7ff21c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient(hunting): picked mon.noname-c con 0x7ff21c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c154220 con 0x7ff21c08ce00 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c058de0 con 0x7ff21c08b7f0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21c083250 con 0x7ff21c08d8e0 2026-02-21T22:15:39.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient(hunting): authenticate will time out at 1148.860655s 2026-02-21T22:15:39.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff2212f5640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 0x7ff21c084a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58668/0 (socket says 10.20.193.102:58668) 2026-02-21T22:15:39.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff2212f5640 1 -- 10.20.193.102:0/337644902 learned_addr learned my addr 10.20.193.102:0/337644902 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff2222f7640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.102:6790/0 conn(0x7ff21c08d8e0 legacy=0x7ff21c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff2212f5640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 legacy=0x7ff21c084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff221af6640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.178:6789/0 conn(0x7ff21c08b7f0 legacy=0x7ff21c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358928176 0 0) 0x7ff21c083250 con 0x7ff21c08d8e0 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): my global_id is 4343 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1ec003620 con 0x7ff21c08d8e0 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826241827 0 0) 0x7ff20c003400 con 0x7ff21c08b7f0 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): my global_id is 4321 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff21c083250 con 0x7ff21c08b7f0 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3541249886 0 0) 0x7ff1ec003620 con 0x7ff21c08d8e0 2026-02-21T22:15:39.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff20c003400 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2180048d0 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): dump: 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1353381052 0 0) 0x7ff20c003400 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.178:6789/0 conn(0x7ff21c08b7f0 legacy=0x7ff21c1532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.102:6789/0 conn(0x7ff21c08ce00 legacy=0x7ff21c084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 monclient: found mon.c 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c083530 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 5 monclient: authenticate success, global_id 4343 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _renew_subs 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c081d30 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _renew_subs 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff21c085170 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff218004ed0 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff218005260 con 0x7ff21c08d8e0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20a7fc640 10 monclient: dump: 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff20bfff640 4 set_mon_vals no callback set 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.621+0000 7ff223d81640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff223d81640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff223d81640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c059650 con 0x7ff21c08d8e0 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff2180054f0 con 0x7ff21c08d8e0 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7ff21804d950 con 0x7ff21c08d8e0 2026-02-21T22:15:39.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff221af6640 10 monclient: get_auth_request con 0x7ff1ec03d310 auth_method 0 2026-02-21T22:15:39.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ff218018ac0 con 0x7ff21c08d8e0 2026-02-21T22:15:39.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff20a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.625+0000 7ff20a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff21c062ce0 con 0x7ff1ec03d310 2026-02-21T22:15:39.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff20a7fc640 1 -- 10.20.193.102:0/337644902 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34187 (unknown 2965378022 0 4103109511) 0x7ff21c062ce0 con 0x7ff1ec03d310 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff1ec03d310 legacy=0x7ff1ec05d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 10 monclient: shutdown 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 >> v1:10.20.193.102:6790/0 conn(0x7ff21c08d8e0 legacy=0x7ff21c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 shutdown_connections 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 >> 10.20.193.102:0/337644902 conn(0x7ff21c091a50 msgr2=0x7ff21c05df50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 shutdown_connections 2026-02-21T22:15:39.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.709+0000 7ff223d81640 1 -- 10.20.193.102:0/337644902 wait complete. 2026-02-21T22:15:39.724 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-21T22:15:39.488151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":162848,"kb_used_data":1088,"kb_used_omap":50,"kb_used_meta":161613,"kb_avail":566068192,"statfs":{"total":579820584960,"available":579653828608,"internally_reserved":0,"allocated":1114112,"data_stored":1493206,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":51359,"internal_metadata":165492577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.960685"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540434+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:15:36.540434+0000","last_peered":"2026-02-21T22:15:36.540434+0000","last_clean":"2026-02-21T22:15:36.540434+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:15:36.540434+0000","last_undegraded":"2026-02-21T22:15:36.540434+0000","last_fullsized":"2026-02-21T22:15:36.540434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537959+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537959+0000","last_peered":"2026-02-21T22:15:36.537959+0000","last_clean":"2026-02-21T22:15:36.537959+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:15:36.537959+0000","last_undegraded":"2026-02-21T22:15:36.537959+0000","last_fullsized":"2026-02-21T22:15:36.537959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540562+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:15:36.540562+0000","last_peered":"2026-02-21T22:15:36.540562+0000","last_clean":"2026-02-21T22:15:36.540562+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:15:36.540562+0000","last_undegraded":"2026-02-21T22:15:36.540562+0000","last_fullsized":"2026-02-21T22:15:36.540562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537987+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537987+0000","last_peered":"2026-02-21T22:15:36.537987+0000","last_clean":"2026-02-21T22:15:36.537987+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:15:36.537987+0000","last_undegraded":"2026-02-21T22:15:36.537987+0000","last_fullsized":"2026-02-21T22:15:36.537987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.552137+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:15:36.552137+0000","last_peered":"2026-02-21T22:15:36.552137+0000","last_clean":"2026-02-21T22:15:36.552137+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:15:36.552137+0000","last_undegraded":"2026-02-21T22:15:36.552137+0000","last_fullsized":"2026-02-21T22:15:36.552137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.538007+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:15:36.538007+0000","last_peered":"2026-02-21T22:15:36.538007+0000","last_clean":"2026-02-21T22:15:36.538007+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:15:36.538007+0000","last_undegraded":"2026-02-21T22:15:36.538007+0000","last_fullsized":"2026-02-21T22:15:36.538007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540688+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:15:36.540688+0000","last_peered":"2026-02-21T22:15:36.540688+0000","last_clean":"2026-02-21T22:15:36.540688+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:15:36.540688+0000","last_undegraded":"2026-02-21T22:15:36.540688+0000","last_fullsized":"2026-02-21T22:15:36.540688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.542334+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:15:36.542334+0000","last_peered":"2026-02-21T22:15:36.542334+0000","last_clean":"2026-02-21T22:15:36.542334+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:15:36.542334+0000","last_undegraded":"2026-02-21T22:15:36.542334+0000","last_fullsized":"2026-02-21T22:15:36.542334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'112","reported_seq":157,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540168+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:15:36.540168+0000","last_peered":"2026-02-21T22:15:36.540168+0000","last_clean":"2026-02-21T22:15:36.540168+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:15:36.540168+0000","last_undegraded":"2026-02-21T22:15:36.540168+0000","last_fullsized":"2026-02-21T22:15:36.540168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8772,"internal_metadata":27581884},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27120,"kb_used_data":160,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344720,"statfs":{"total":96636764160,"available":96608993280,"internally_reserved":0,"allocated":163840,"data_stored":52760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9437,"internal_metadata":27581219},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:39.724 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: monmap: 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.786345+0000 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.786345+0000 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.786 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 1 Processor -- start 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 1 -- start start 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: init 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-b con 0x7ffb9008e5b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-a con 0x7ffb90059150 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-c con 0x7ffb9008f090 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb90057ed0 con 0x7ffb90059150 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900581b0 con 0x7ffb9008f090 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb90058490 con 0x7ffb9008e5b0 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb97029640 10 monclient(hunting): authenticate will time out at 1149.022263s 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb87fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ffb90059150 0x7ffb90183fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58670/0 (socket says 10.20.193.102:58670) 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb94d9e640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ffb9008e5b0 0x7ffb9008e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60456/0 (socket says 10.20.193.102:60456) 2026-02-21T22:15:39.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.781+0000 7ffb87fff640 1 -- 10.20.193.102:0/1797769125 learned_addr learned my addr 10.20.193.102:0/1797769125 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb94d9e640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.102:6790/0 conn(0x7ffb9008e5b0 legacy=0x7ffb9008e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb87fff640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.102:6789/0 conn(0x7ffb90059150 legacy=0x7ffb90183fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313769578 0 0) 0x7ffb90058490 con 0x7ffb9008e5b0 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): my global_id is 4346 2026-02-21T22:15:39.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb7c003620 con 0x7ffb9008e5b0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509867943 0 0) 0x7ffb90057ed0 con 0x7ffb90059150 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient(hunting): my global_id is 4344 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb90058490 con 0x7ffb90059150 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb9559f640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.178:6789/0 conn(0x7ffb9008f090 legacy=0x7ffb901a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1266436969 0 0) 0x7ffb7c003620 con 0x7ffb9008e5b0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.178:6789/0 conn(0x7ffb9008f090 legacy=0x7ffb901a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.102:6789/0 conn(0x7ffb90059150 legacy=0x7ffb90183fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 monclient: found mon.noname-b 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9005e000 con 0x7ffb9008e5b0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb80003110 con 0x7ffb9008e5b0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: dump: 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 5 monclient: authenticate success, global_id 4346 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb94d9e640 1 --1- 10.20.193.102:0/1797769125 >> v1:10.20.193.102:6790/0 conn(0x7ffb9008e5b0 0x7ffb9008e980 :-1 s=OPENED pgs=114 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7ffb80004a90 mon_map magic: 0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb80003450 con 0x7ffb9008e5b0 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 4 set_mon_vals no callback set 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: shutdown 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- 10.20.193.102:0/1797769125 >> v1:10.20.193.102:6790/0 conn(0x7ffb9008e5b0 legacy=0x7ffb9008e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 1 -- 10.20.193.102:0/1797769125 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb80004a90 con 0x7ffb9008e5b0 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb877fe640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- 10.20.193.102:0/1797769125 shutdown_connections 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- 10.20.193.102:0/1797769125 >> 10.20.193.102:0/1797769125 conn(0x7ffb90056d40 msgr2=0x7ffb90093bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- 10.20.193.102:0/1797769125 shutdown_connections 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- 10.20.193.102:0/1797769125 wait complete. 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: monmap: 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.791 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 Processor -- start 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- start start 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: init 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-c con 0x7ffb90059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-a con 0x7ffb9008e5b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient(hunting): picked mon.noname-b con 0x7ffb9008f090 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9012d970 con 0x7ffb9008e5b0 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9012dc50 con 0x7ffb9008f090 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9012b8f0 con 0x7ffb90059150 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb97029640 10 monclient(hunting): authenticate will time out at 1149.026129s 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb87fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ffb9008e5b0 0x7ffb90194590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58674/0 (socket says 10.20.193.102:58674) 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb94d9e640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ffb90059150 0x7ffb90153310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60462/0 (socket says 10.20.193.102:60462) 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.785+0000 7ffb87fff640 1 -- 10.20.193.102:0/1419684256 learned_addr learned my addr 10.20.193.102:0/1419684256 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb94d9e640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.102:6790/0 conn(0x7ffb90059150 legacy=0x7ffb90153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb87fff640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.102:6789/0 conn(0x7ffb9008e5b0 legacy=0x7ffb90194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971109359 0 0) 0x7ffb9012b8f0 con 0x7ffb90059150 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): my global_id is 4349 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb60003620 con 0x7ffb90059150 2026-02-21T22:15:39.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818119890 0 0) 0x7ffb9012d970 con 0x7ffb9008e5b0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): my global_id is 4347 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb9012b8f0 con 0x7ffb9008e5b0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb9559f640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.178:6789/0 conn(0x7ffb9008f090 legacy=0x7ffb90195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3623087709 0 0) 0x7ffb60003620 con 0x7ffb90059150 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb9012d970 con 0x7ffb90059150 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb800048e0 con 0x7ffb90059150 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): dump: 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3483713573 0 0) 0x7ffb9012b8f0 con 0x7ffb9008e5b0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb60003620 con 0x7ffb9008e5b0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb780048e0 con 0x7ffb9008e5b0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient(hunting): dump: 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.794 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4075203253 0 0) 0x7ffb9012d970 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.178:6789/0 conn(0x7ffb9008f090 legacy=0x7ffb90195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.102:6789/0 conn(0x7ffb9008e5b0 legacy=0x7ffb90194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 monclient: found mon.c 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9013d210 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 5 monclient: authenticate success, global_id 4349 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _renew_subs 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9012d3b0 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _renew_subs 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ffb9012c620 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb80003300 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb800052d0 con 0x7ffb90059150 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb86ffd640 4 set_mon_vals no callback set 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: dump: 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb900008e0 con 0x7ffb90059150 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ffb80011960 con 0x7ffb90059150 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7ffb8004c490 con 0x7ffb90059150 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb87fff640 10 monclient: get_auth_request con 0x7ffb6003d6a0 auth_method 0 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ffb9012c620 con 0x7ffb90059150 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.789+0000 7ffb857fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:39.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.873+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb90092c10 con 0x7ffb6003d6a0 2026-02-21T22:15:39.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.873+0000 7ffb857fa640 1 -- 10.20.193.102:0/1419684256 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34187 (unknown 2965378022 0 4103109511) 0x7ffb90092c10 con 0x7ffb6003d6a0 2026-02-21T22:15:39.880 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:15:39.880 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.178:6800/2549949439 conn(0x7ffb6003d6a0 legacy=0x7ffb6005db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 10 monclient: shutdown 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 >> v1:10.20.193.102:6790/0 conn(0x7ffb90059150 legacy=0x7ffb90153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 shutdown_connections 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 >> 10.20.193.102:0/1419684256 conn(0x7ffb90056d40 msgr2=0x7ffb90140370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 shutdown_connections 2026-02-21T22:15:39.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.877+0000 7ffb97029640 1 -- 10.20.193.102:0/1419684256 wait complete. 2026-02-21T22:15:39.890 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2026-02-21T22:15:39.488151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":162848,"kb_used_data":1088,"kb_used_omap":50,"kb_used_meta":161613,"kb_avail":566068192,"statfs":{"total":579820584960,"available":579653828608,"internally_reserved":0,"allocated":1114112,"data_stored":1493206,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":51359,"internal_metadata":165492577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.960685"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540434+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:15:36.540434+0000","last_peered":"2026-02-21T22:15:36.540434+0000","last_clean":"2026-02-21T22:15:36.540434+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:15:36.540434+0000","last_undegraded":"2026-02-21T22:15:36.540434+0000","last_fullsized":"2026-02-21T22:15:36.540434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537959+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537959+0000","last_peered":"2026-02-21T22:15:36.537959+0000","last_clean":"2026-02-21T22:15:36.537959+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:15:36.537959+0000","last_undegraded":"2026-02-21T22:15:36.537959+0000","last_fullsized":"2026-02-21T22:15:36.537959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540562+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:15:36.540562+0000","last_peered":"2026-02-21T22:15:36.540562+0000","last_clean":"2026-02-21T22:15:36.540562+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:15:36.540562+0000","last_undegraded":"2026-02-21T22:15:36.540562+0000","last_fullsized":"2026-02-21T22:15:36.540562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.537987+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:15:36.537987+0000","last_peered":"2026-02-21T22:15:36.537987+0000","last_clean":"2026-02-21T22:15:36.537987+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:15:36.537987+0000","last_undegraded":"2026-02-21T22:15:36.537987+0000","last_fullsized":"2026-02-21T22:15:36.537987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.552137+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:15:36.552137+0000","last_peered":"2026-02-21T22:15:36.552137+0000","last_clean":"2026-02-21T22:15:36.552137+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:15:36.552137+0000","last_undegraded":"2026-02-21T22:15:36.552137+0000","last_fullsized":"2026-02-21T22:15:36.552137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.538007+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:15:36.538007+0000","last_peered":"2026-02-21T22:15:36.538007+0000","last_clean":"2026-02-21T22:15:36.538007+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:15:36.538007+0000","last_undegraded":"2026-02-21T22:15:36.538007+0000","last_fullsized":"2026-02-21T22:15:36.538007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540688+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:15:36.540688+0000","last_peered":"2026-02-21T22:15:36.540688+0000","last_clean":"2026-02-21T22:15:36.540688+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:15:36.540688+0000","last_undegraded":"2026-02-21T22:15:36.540688+0000","last_fullsized":"2026-02-21T22:15:36.540688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.542334+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:15:36.542334+0000","last_peered":"2026-02-21T22:15:36.542334+0000","last_clean":"2026-02-21T22:15:36.542334+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:15:36.542334+0000","last_undegraded":"2026-02-21T22:15:36.542334+0000","last_fullsized":"2026-02-21T22:15:36.542334+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'112","reported_seq":157,"reported_epoch":19,"state":"active+clean","last_fresh":"2026-02-21T22:15:36.540168+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:15:36.540168+0000","last_peered":"2026-02-21T22:15:36.540168+0000","last_clean":"2026-02-21T22:15:36.540168+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:15:36.540168+0000","last_undegraded":"2026-02-21T22:15:36.540168+0000","last_fullsized":"2026-02-21T22:15:36.540168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27112,"kb_used_data":152,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344728,"statfs":{"total":96636764160,"available":96609001472,"internally_reserved":0,"allocated":155648,"data_stored":48632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640260,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8772,"internal_metadata":27581884},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27120,"kb_used_data":160,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344720,"statfs":{"total":96636764160,"available":96608993280,"internally_reserved":0,"allocated":163840,"data_stored":52760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":164,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":167936,"data_stored":52779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9437,"internal_metadata":27581219},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":224,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":229376,"data_stored":643128,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":8125,"internal_metadata":27582531},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:15:39.890 INFO:tasks.mon_thrash.ceph_manager:clean! 2026-02-21T22:15:39.891 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_netsplit_grace_period 30 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: get_monmap_and_config 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: monmap: 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:39.963586+0000 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:39.963586+0000 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:39.963 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 1 Processor -- start 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 1 -- start start 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: init 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-b con 0x7f9954084760 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-a con 0x7f995408c3c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-c con 0x7f99541855f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9954089860 con 0x7f995408c3c0 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995408a440 con 0x7f99541855f0 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995408a720 con 0x7f9954084760 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f995bb2c640 10 monclient(hunting): authenticate will time out at 1149.199488s 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f99598a1640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 0x7f9954084b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60474/0 (socket says 10.20.193.102:60474) 2026-02-21T22:15:39.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.957+0000 7f99598a1640 1 -- 10.20.193.102:0/2911930544 learned_addr learned my addr 10.20.193.102:0/2911930544 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.966 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f99598a1640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 legacy=0x7f9954084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.966 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f99590a0640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.102:6789/0 conn(0x7f995408c3c0 legacy=0x7f9954183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995a0a2640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.178:6789/0 conn(0x7f99541855f0 legacy=0x7f99541a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833298818 0 0) 0x7f9954089860 con 0x7f995408c3c0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): my global_id is 4350 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f993c003620 con 0x7f995408c3c0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105680580 0 0) 0x7f995408a720 con 0x7f9954084760 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): my global_id is 4352 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9954089860 con 0x7f9954084760 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320475431 0 0) 0x7f995408a440 con 0x7f99541855f0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient(hunting): my global_id is 4330 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f995408a720 con 0x7f99541855f0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652873102 0 0) 0x7f993c003620 con 0x7f995408c3c0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 legacy=0x7f9954084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.178:6789/0 conn(0x7f99541855f0 legacy=0x7f99541a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 monclient: found mon.noname-a 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9954085240 con 0x7f995408c3c0 2026-02-21T22:15:39.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9948003160 con 0x7f995408c3c0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: dump: 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 5 monclient: authenticate success, global_id 4350 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f99480034a0 con 0x7f995408c3c0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 1 -- 10.20.193.102:0/2911930544 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f99480038c0 con 0x7f995408c3c0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995889f640 10 monclient: dump: 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:39.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 4 set_mon_vals no callback set 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: shutdown 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- 10.20.193.102:0/2911930544 >> v1:10.20.193.102:6789/0 conn(0x7f995408c3c0 legacy=0x7f9954183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- 10.20.193.102:0/2911930544 shutdown_connections 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- 10.20.193.102:0/2911930544 >> 10.20.193.102:0/2911930544 conn(0x7f995408e470 msgr2=0x7f995408e870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- 10.20.193.102:0/2911930544 shutdown_connections 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- 10.20.193.102:0/2911930544 wait complete. 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: build_initial_monmap 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: monmap: 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 Processor -- start 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- start start 2026-02-21T22:15:39.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: init 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-c con 0x7f9954084760 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-a con 0x7f995408c3c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient(hunting): picked mon.noname-b con 0x7f99541855f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9954080e60 con 0x7f995408c3c0 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995405d3f0 con 0x7f99541855f0 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995407eb00 con 0x7f9954084760 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f995bb2c640 10 monclient(hunting): authenticate will time out at 1149.203743s 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f99598a1640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 0x7f9954131c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60480/0 (socket says 10.20.193.102:60480) 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.961+0000 7f99598a1640 1 -- 10.20.193.102:0/3381424644 learned_addr learned my addr 10.20.193.102:0/3381424644 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99598a1640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 legacy=0x7f9954131c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99590a0640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.102:6789/0 conn(0x7f995408c3c0 legacy=0x7f9954172e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191832333 0 0) 0x7f995407eb00 con 0x7f9954084760 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): my global_id is 4355 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9924003620 con 0x7f9954084760 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995a0a2640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.178:6789/0 conn(0x7f99541855f0 legacy=0x7f99541745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456630792 0 0) 0x7f9954080e60 con 0x7f995408c3c0 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): my global_id is 4353 2026-02-21T22:15:39.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f995407eb00 con 0x7f995408c3c0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308790189 0 0) 0x7f995405d3f0 con 0x7f99541855f0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): my global_id is 4333 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9954080e60 con 0x7f99541855f0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3119329799 0 0) 0x7f9924003620 con 0x7f9954084760 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f995405d3f0 con 0x7f9954084760 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9944003400 con 0x7f9954084760 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): dump: 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 395500805 0 0) 0x7f995407eb00 con 0x7f995408c3c0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9924003620 con 0x7f995408c3c0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9948003710 con 0x7f995408c3c0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): dump: 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.972 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2062920284 0 0) 0x7f9954080e60 con 0x7f99541855f0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f995407eb00 con 0x7f99541855f0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9950004910 con 0x7f99541855f0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient(hunting): dump: 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3437934589 0 0) 0x7f995405d3f0 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.178:6789/0 conn(0x7f99541855f0 legacy=0x7f99541745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.102:6789/0 conn(0x7f995408c3c0 legacy=0x7f9954172e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 monclient: found mon.c 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540805c0 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 5 monclient: authenticate success, global_id 4355 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _renew_subs 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f99540808a0 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _renew_subs 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f995407e0f0 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9944004b90 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9944004ef0 con 0x7f9954084760 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:39.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 10 monclient: dump: 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f9943fff640 4 set_mon_vals no callback set 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99540008e0 con 0x7f9954084760 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9944005180 con 0x7f9954084760 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f994404c500 con 0x7f9954084760 2026-02-21T22:15:39.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.965+0000 7f99590a0640 10 monclient: get_auth_request con 0x7f992403d820 auth_method 0 2026-02-21T22:15:39.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.969+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f99440175b0 con 0x7f9954084760 2026-02-21T22:15:39.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.969+0000 7f99427fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:39.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:39.969+0000 7f99427fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.053+0000 7f995bb2c640 10 start_mon_command cmd=[{"prefix": "config set", "who": "mon", "name": "mon_netsplit_grace_period", "value": "30"}] 2026-02-21T22:15:40.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.053+0000 7f995bb2c640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "mon", "name": "mon_netsplit_grace_period", "value": "30"}] 2026-02-21T22:15:40.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.053+0000 7f995bb2c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.053+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 --> v1:10.20.193.102:6790/0 -- mon_command([{prefix=config set, name=mon_netsplit_grace_period}] v 0) -- 0x7f995407e3e0 con 0x7f9954084760 2026-02-21T22:15:40.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f99427fc640 1 -- 10.20.193.102:0/3381424644 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{prefix=config set, name=mon_netsplit_grace_period}]=0 v2)=0 v2) ==== 124+0+0 (unknown 948039188 0 0) 0x7f994401c8d0 con 0x7f9954084760 2026-02-21T22:15:40.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f99427fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "mon", "name": "mon_netsplit_grace_period", "value": "30"}] 2026-02-21T22:15:40.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f99427fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.178:6800/2549949439 conn(0x7f992403d820 legacy=0x7f992405dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 10 monclient: shutdown 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 >> v1:10.20.193.102:6790/0 conn(0x7f9954084760 legacy=0x7f9954131c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 shutdown_connections 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 >> 10.20.193.102:0/3381424644 conn(0x7f995408e470 msgr2=0x7f99541a59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 shutdown_connections 2026-02-21T22:15:40.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.057+0000 7f995bb2c640 1 -- 10.20.193.102:0/3381424644 wait complete. 2026-02-21T22:15:40.074 DEBUG:tasks.mon_thrash:Yielding 2026-02-21T22:15:40.074 INFO:teuthology.run_tasks:Running task rados... 2026-02-21T22:15:40.087 INFO:tasks.rados:Beginning rados... 2026-02-21T22:15:40.087 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2026-02-21T22:15:40.087 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'mon', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 20, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore debug onode segmentation random': True, 'bluestore fsck on mount': True, 'bluestore onode segment size': '512K', 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op num shards': 1, 'osd op num threads per shard': 2, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd_mclock_skip_benchmark': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', '\\(MON_NETSPLIT)', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'distro-suffix': 'rockylinux-10'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '53f8441172c46e34523029ab88f9db7a45514e84'}} 2026-02-21T22:15:40.087 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2026-02-21T22:15:40.087 DEBUG:teuthology.run_tasks:Unwinding manager rados 2026-02-21T22:15:40.098 INFO:tasks.rados:joining rados 2026-02-21T22:15:40.098 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2026-02-21T22:15:40.098 INFO:tasks.mon_thrash.mon_thrasher:seed: 1771712139, revive delay: 90.0, thrash delay: 1.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2026-02-21T22:15:40.098 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:15:40.098 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:15:40.098 INFO:tasks.rados:clients are ['client.0'] 2026-02-21T22:15:40.098 INFO:tasks.rados:starting run 0 out of 1 2026-02-21T22:15:40.098 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-21T22:15:40.098 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.149+0000 7fef5db5a640 10 monclient: get_monmap_and_config 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.149+0000 7fef5db5a640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.149+0000 7fef5db5a640 10 monclient: monmap: 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:40.156645+0000 2026-02-21T22:15:40.157 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:40.156645+0000 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:40.158 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 Processor -- start 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- start start 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient: init 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-b con 0x7fef58091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-c con 0x7fef580909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-a con 0x7fef581854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef58057a20 con 0x7fef581854b0 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef58091d20 con 0x7fef580909a0 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef58057740 con 0x7fef58091240 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient(hunting): authenticate will time out at 1149.392787s 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef57fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fef581854b0 0x7fef581a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58714/0 (socket says 10.20.193.102:58714) 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef57fff640 1 -- 10.20.193.102:0/1592264540 learned_addr learned my addr 10.20.193.102:0/1592264540 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef57fff640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.102:6789/0 conn(0x7fef581854b0 legacy=0x7fef581a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef577fe640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.102:6790/0 conn(0x7fef58091240 legacy=0x7fef58091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef56ffd640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.178:6789/0 conn(0x7fef580909a0 legacy=0x7fef58183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335333396 0 0) 0x7fef58057a20 con 0x7fef581854b0 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): my global_id is 4356 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef3c003620 con 0x7fef581854b0 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967468116 0 0) 0x7fef58057740 con 0x7fef58091240 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): my global_id is 4358 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef58057a20 con 0x7fef58091240 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704767995 0 0) 0x7fef58091d20 con 0x7fef580909a0 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient(hunting): my global_id is 4336 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef58057740 con 0x7fef580909a0 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2509636571 0 0) 0x7fef3c003620 con 0x7fef581854b0 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.102:6790/0 conn(0x7fef58091240 legacy=0x7fef58091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.178:6789/0 conn(0x7fef580909a0 legacy=0x7fef58183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 monclient: found mon.noname-a 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef58090dc0 con 0x7fef581854b0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fef4c004620 con 0x7fef581854b0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:40.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: dump: 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.168 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 5 monclient: authenticate success, global_id 4356 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fef4c0048f0 con 0x7fef581854b0 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 1 -- 10.20.193.102:0/1592264540 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fef4c006060 con 0x7fef581854b0 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef567fc640 10 monclient: dump: 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.169 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 4 set_mon_vals no callback set 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 10 monclient: shutdown 2026-02-21T22:15:40.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- 10.20.193.102:0/1592264540 >> v1:10.20.193.102:6789/0 conn(0x7fef581854b0 legacy=0x7fef581a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- 10.20.193.102:0/1592264540 shutdown_connections 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.153+0000 7fef5db5a640 1 -- 10.20.193.102:0/1592264540 >> 10.20.193.102:0/1592264540 conn(0x7fef5805e660 msgr2=0x7fef58095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- 10.20.193.102:0/1592264540 shutdown_connections 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- 10.20.193.102:0/1592264540 wait complete. 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: monmap: 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:40.171 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: get_monmap_and_config 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: monmap: 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:40.161196+0000 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:40.161196+0000 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:40.173 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:40.174 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 Processor -- start 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- start start 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: init 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-a con 0x7fef580909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-b con 0x7fef58091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient(hunting): picked mon.noname-c con 0x7fef581854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5805d450 con 0x7fef580909a0 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef580592d0 con 0x7fef58091240 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5805c5e0 con 0x7fef581854b0 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient(hunting): authenticate will time out at 1149.396905s 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef57fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fef581854b0 0x7fef581784a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60498/0 (socket says 10.20.193.102:60498) 2026-02-21T22:15:40.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef57fff640 1 -- 10.20.193.102:0/3220388833 learned_addr learned my addr 10.20.193.102:0/3220388833 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 1 Processor -- start 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 1 -- start start 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: init 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-c con 0x7fd63c13fbf0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-b con 0x7fd63c0901f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-a con 0x7fd63c185520 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c092720 con 0x7fd63c185520 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c057480 con 0x7fd63c13fbf0 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c057760 con 0x7fd63c0901f0 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd6429c5640 10 monclient(hunting): authenticate will time out at 1149.397741s 2026-02-21T22:15:40.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd63b7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 0x7fd63c183e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60514/0 (socket says 10.20.193.102:60514) 2026-02-21T22:15:40.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd63b7fe640 1 -- 10.20.193.102:0/2672617650 learned_addr learned my addr 10.20.193.102:0/2672617650 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef57fff640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.102:6790/0 conn(0x7fef581854b0 legacy=0x7fef581784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef577fe640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.102:6789/0 conn(0x7fef580909a0 legacy=0x7fef581329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489610038 0 0) 0x7fef5805c5e0 con 0x7fef581854b0 2026-02-21T22:15:40.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): my global_id is 4361 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef28003620 con 0x7fef581854b0 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef56ffd640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.178:6789/0 conn(0x7fef58091240 legacy=0x7fef58176d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367875213 0 0) 0x7fef5805d450 con 0x7fef580909a0 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): my global_id is 4359 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef5805c5e0 con 0x7fef580909a0 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394567785 0 0) 0x7fef580592d0 con 0x7fef58091240 2026-02-21T22:15:40.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): my global_id is 4339 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef5805d450 con 0x7fef58091240 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 480589695 0 0) 0x7fef28003620 con 0x7fef581854b0 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fef580592d0 con 0x7fef581854b0 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fef4c005df0 con 0x7fef581854b0 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): dump: 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.183 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2934627387 0 0) 0x7fef5805c5e0 con 0x7fef580909a0 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fef28003620 con 0x7fef580909a0 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fef44003710 con 0x7fef580909a0 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient(hunting): dump: 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.184 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2901501855 0 0) 0x7fef580592d0 con 0x7fef581854b0 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.178:6789/0 conn(0x7fef58091240 legacy=0x7fef58176d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.102:6789/0 conn(0x7fef580909a0 legacy=0x7fef581329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 monclient: found mon.c 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5805b0f0 con 0x7fef581854b0 2026-02-21T22:15:40.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 5 monclient: authenticate success, global_id 4361 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _renew_subs 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fef5805d170 con 0x7fef581854b0 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _renew_subs 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fef581330c0 con 0x7fef581854b0 2026-02-21T22:15:40.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd63b7fe640 1 -- 10.20.193.102:0/2672617650 >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 legacy=0x7fd63c183e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fef4c006040 con 0x7fef581854b0 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fef4c006680 con 0x7fef581854b0 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fd63bfff640 1 -- 10.20.193.102:0/2672617650 >> v1:10.20.193.178:6789/0 conn(0x7fd63c13fbf0 legacy=0x7fd63c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef55ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:40.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.157+0000 7fef33fff640 10 monclient: dump: 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.188 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1402558424 0 0) 0x7fd63c057760 con 0x7fd63c0901f0 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): my global_id is 4364 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd620003620 con 0x7fd63c0901f0 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453623454 0 0) 0x7fd63c057480 con 0x7fd63c13fbf0 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient(hunting): my global_id is 4342 2026-02-21T22:15:40.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd63c057760 con 0x7fd63c13fbf0 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1211677235 0 0) 0x7fd620003620 con 0x7fd63c0901f0 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 >> v1:10.20.193.178:6789/0 conn(0x7fd63c13fbf0 legacy=0x7fd63c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef5db5a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 >> v1:10.20.193.102:6789/0 conn(0x7fd63c185520 legacy=0x7fd63c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef5db5a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 monclient: found mon.noname-b 2026-02-21T22:15:40.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef5db5a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c0584a0 con 0x7fd63c0901f0 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 5 monclient: authenticate success, global_id 4364 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd62c003110 con 0x7fd63c0901f0 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: dump: 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.192 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef580008e0 con 0x7fef581854b0 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd62c003450 con 0x7fd63c0901f0 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 1 -- 10.20.193.102:0/2672617650 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd62c004b50 con 0x7fd63c0901f0 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63affd640 10 monclient: dump: 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 4 set_mon_vals no callback set 2026-02-21T22:15:40.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fef4c012d10 con 0x7fef581854b0 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: shutdown 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- 10.20.193.102:0/2672617650 >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 legacy=0x7fd63c183e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- 10.20.193.102:0/2672617650 shutdown_connections 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- 10.20.193.102:0/2672617650 >> 10.20.193.102:0/2672617650 conn(0x7fd63c05e670 msgr2=0x7fd63c095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- 10.20.193.102:0/2672617650 shutdown_connections 2026-02-21T22:15:40.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- 10.20.193.102:0/2672617650 wait complete. 2026-02-21T22:15:40.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fef4c04d8b0 con 0x7fef581854b0 2026-02-21T22:15:40.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef577fe640 10 monclient: get_auth_request con 0x7fef2803d580 auth_method 0 2026-02-21T22:15:40.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: monmap: 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 Processor -- start 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- start start 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: init 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-c con 0x7fd63c0901f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-b con 0x7fd63c13fbf0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient(hunting): picked mon.noname-a con 0x7fd63c185520 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c111480 con 0x7fd63c185520 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c110610 con 0x7fd63c13fbf0 2026-02-21T22:15:40.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c10f120 con 0x7fd63c0901f0 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd6429c5640 10 monclient(hunting): authenticate will time out at 1149.401506s 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 0x7fd63c13df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60528/0 (socket says 10.20.193.102:60528) 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63bfff640 1 -- 10.20.193.102:0/425851846 learned_addr learned my addr 10.20.193.102:0/425851846 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fef4c018a20 con 0x7fef581854b0 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef33fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fef33fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63bfff640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 legacy=0x7fd63c13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd640f3b640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.102:6789/0 conn(0x7fd63c185520 legacy=0x7fd63c181e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517829373 0 0) 0x7fd63c10f120 con 0x7fd63c0901f0 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): my global_id is 4367 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd60c003620 con 0x7fd63c0901f0 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107459908 0 0) 0x7fd63c111480 con 0x7fd63c185520 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): my global_id is 4362 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd63c10f120 con 0x7fd63c185520 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd63b7fe640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.178:6789/0 conn(0x7fd63c13fbf0 legacy=0x7fd63c180760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1857425819 0 0) 0x7fd60c003620 con 0x7fd63c0901f0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd63c111480 con 0x7fd63c0901f0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd628003490 con 0x7fd63c0901f0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.161+0000 7fd638ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552838741 0 0) 0x7fd63c110610 con 0x7fd63c13fbf0 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): my global_id is 4345 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd60c003620 con 0x7fd63c13fbf0 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1636256192 0 0) 0x7fd63c10f120 con 0x7fd63c185520 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd63c110610 con 0x7fd63c185520 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd6300057c0 con 0x7fd63c185520 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2302487824 0 0) 0x7fd63c111480 con 0x7fd63c0901f0 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.178:6789/0 conn(0x7fd63c13fbf0 legacy=0x7fd63c180760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.102:6789/0 conn(0x7fd63c185520 legacy=0x7fd63c181e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 monclient: found mon.c 2026-02-21T22:15:40.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c110be0 con 0x7fd63c0901f0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 5 monclient: authenticate success, global_id 4367 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _renew_subs 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c10ebb0 con 0x7fd63c0901f0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _renew_subs 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd63c10fc80 con 0x7fd63c0901f0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd628003c30 con 0x7fd63c0901f0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd628003fc0 con 0x7fd63c0901f0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: dump: 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.203 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd63a7fc640 4 set_mon_vals no callback set 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c056400 con 0x7fd63c0901f0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd628004210 con 0x7fd63c0901f0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fd62804c660 con 0x7fd63c0901f0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd63b7fe640 10 monclient: get_auth_request con 0x7fd60c03d880 auth_method 0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fd6280177d0 con 0x7fd63c0901f0 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.165+0000 7fd638ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.253+0000 7fd6429c5640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-21T22:15:40.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.253+0000 7fd6429c5640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-21T22:15:40.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.253+0000 7fd6429c5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.253+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} v 0) -- 0x7fd63c089670 con 0x7fd63c0901f0 2026-02-21T22:15:40.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:15:40.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:15:40.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fef58095010 con 0x7fef581854b0 2026-02-21T22:15:40.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef33fff640 1 -- 10.20.193.102:0/3220388833 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3817998540) 0x7fef4c01dc80 con 0x7fef581854b0 2026-02-21T22:15:40.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef33fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:15:40.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef33fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:40.283 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.178:6800/2549949439 conn(0x7fef2803d580 legacy=0x7fef2805da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 10 monclient: shutdown 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 >> v1:10.20.193.102:6790/0 conn(0x7fef581854b0 legacy=0x7fef581784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 shutdown_connections 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 >> 10.20.193.102:0/3220388833 conn(0x7fef5805e660 msgr2=0x7fef581a5890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 shutdown_connections 2026-02-21T22:15:40.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.277+0000 7fef5db5a640 1 -- 10.20.193.102:0/3220388833 wait complete. 2026-02-21T22:15:40.293 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:15:40.293 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:15:40.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:15:40.294 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: get_monmap_and_config 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: monmap: 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:40.395877+0000 2026-02-21T22:15:40.396 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:40.395877+0000 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:40.397 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 1 Processor -- start 2026-02-21T22:15:40.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 1 -- start start 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: init 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-b con 0x7fe79408e6c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-c con 0x7fe794059100 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-a con 0x7fe79408f1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7940904a0 con 0x7fe79408f1a0 2026-02-21T22:15:40.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe794090260 con 0x7fe794059100 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe794058820 con 0x7fe79408e6c0 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.389+0000 7fe799e3f640 10 monclient(hunting): authenticate will time out at 1149.631933s 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7937fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 0x7fe79408ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60540/0 (socket says 10.20.193.102:60540) 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7937fe640 1 -- 10.20.193.102:0/68706681 learned_addr learned my addr 10.20.193.102:0/68706681 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7937fe640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 legacy=0x7fe79408ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe793fff640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.102:6789/0 conn(0x7fe79408f1a0 legacy=0x7fe7941a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163340734 0 0) 0x7fe794058820 con 0x7fe79408e6c0 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): my global_id is 4370 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe778003620 con 0x7fe79408e6c0 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe792ffd640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.178:6789/0 conn(0x7fe794059100 legacy=0x7fe794184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795292979 0 0) 0x7fe7940904a0 con 0x7fe79408f1a0 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient(hunting): my global_id is 4365 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe794058820 con 0x7fe79408f1a0 2026-02-21T22:15:40.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2562545502 0 0) 0x7fe778003620 con 0x7fe79408e6c0 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.178:6789/0 conn(0x7fe794059100 legacy=0x7fe794184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.102:6789/0 conn(0x7fe79408f1a0 legacy=0x7fe7941a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 monclient: found mon.noname-b 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe794057f80 con 0x7fe79408e6c0 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 5 monclient: authenticate success, global_id 4370 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe780003110 con 0x7fe79408e6c0 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: dump: 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.407 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe7800033e0 con 0x7fe79408e6c0 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 1 -- 10.20.193.102:0/68706681 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe780004b50 con 0x7fe79408e6c0 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe7927fc640 10 monclient: dump: 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.408 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 4 set_mon_vals no callback set 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 10 monclient: shutdown 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- 10.20.193.102:0/68706681 >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 legacy=0x7fe79408ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- 10.20.193.102:0/68706681 shutdown_connections 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- 10.20.193.102:0/68706681 >> 10.20.193.102:0/68706681 conn(0x7fe794056b30 msgr2=0x7fe794093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- 10.20.193.102:0/68706681 shutdown_connections 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- 10.20.193.102:0/68706681 wait complete. 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 10 monclient: monmap: 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.410 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 Processor -- start 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 1 -- start start 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.393+0000 7fe799e3f640 10 monclient: init 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-b con 0x7fe794059100 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-c con 0x7fe79408e6c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient(hunting): picked mon.noname-a con 0x7fe79408f1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79412bb60 con 0x7fe79408f1a0 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe794140130 con 0x7fe794059100 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79405ea30 con 0x7fe79408e6c0 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient(hunting): authenticate will time out at 1149.636105s 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe792ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 0x7fe7941945e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60548/0 (socket says 10.20.193.102:60548) 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe793fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fe79408f1a0 0x7fe794195d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58750/0 (socket says 10.20.193.102:58750) 2026-02-21T22:15:40.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe792ffd640 1 -- 10.20.193.102:0/2744276119 learned_addr learned my addr 10.20.193.102:0/2744276119 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe792ffd640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 legacy=0x7fe7941945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe793fff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe79408f1a0 legacy=0x7fe794195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530653280 0 0) 0x7fe79405ea30 con 0x7fe79408e6c0 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): my global_id is 4373 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe764003620 con 0x7fe79408e6c0 2026-02-21T22:15:40.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe7937fe640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.178:6789/0 conn(0x7fe794059100 legacy=0x7fe79413c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913556724 0 0) 0x7fe79412bb60 con 0x7fe79408f1a0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): my global_id is 4368 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe79405ea30 con 0x7fe79408f1a0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 393197549 0 0) 0x7fe764003620 con 0x7fe79408e6c0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe79412bb60 con 0x7fe79408e6c0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe784003400 con 0x7fe79408e6c0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:40.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:40.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010067308 0 0) 0x7fe794140130 con 0x7fe794059100 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): my global_id is 4351 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe764003620 con 0x7fe794059100 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2643273492 0 0) 0x7fe79405ea30 con 0x7fe79408f1a0 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe794140130 con 0x7fe79408f1a0 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe788004900 con 0x7fe79408f1a0 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient(hunting): dump: 2026-02-21T22:15:40.422 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 86322208 0 0) 0x7fe79412bb60 con 0x7fe79408e6c0 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.178:6789/0 conn(0x7fe794059100 legacy=0x7fe79413c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe79408f1a0 legacy=0x7fe794195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 monclient: found mon.c 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79412dec0 con 0x7fe79408e6c0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 5 monclient: authenticate success, global_id 4373 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _renew_subs 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe79412d620 con 0x7fe79408e6c0 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _renew_subs 2026-02-21T22:15:40.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe79413ce00 con 0x7fe79408e6c0 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe7840039c0 con 0x7fe79408e6c0 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe784003d20 con 0x7fe79408e6c0 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 10 monclient: dump: 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.425 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe791ffb640 4 set_mon_vals no callback set 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe7940008e0 con 0x7fe794092c10 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fe784003fb0 con 0x7fe79408e6c0 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.397+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fe78404c5a0 con 0x7fe79408e6c0 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe793fff640 10 monclient: get_auth_request con 0x7fe76403d880 auth_method 0 2026-02-21T22:15:40.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe7937fe640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe794092c10 legacy=0x7fe794089df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531215114 0 0) 0x7fe7940008e0 con 0x7fe794092c10 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient(hunting): my global_id is 4371 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe79412bb60 con 0x7fe794092c10 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1941352752 0 0) 0x7fe79412bb60 con 0x7fe794092c10 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7940008e0 con 0x7fe794092c10 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe7800048e0 con 0x7fe794092c10 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient: dump: 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 273635722 0 0) 0x7fe7940008e0 con 0x7fe794092c10 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7940949d0 con 0x7fe794092c10 2026-02-21T22:15:40.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fe780002db0 con 0x7fe794092c10 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe794092c10 legacy=0x7fe794089df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.401+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe794092c10 legacy=0x7fe794089df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.405+0000 7fe799e3f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.405+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe7941538b0 con 0x7fe7941534e0 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe7937fe640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe7941534e0 legacy=0x7fe79408a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890841500 0 0) 0x7fe7941538b0 con 0x7fe7941534e0 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient(hunting): my global_id is 4374 2026-02-21T22:15:40.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7940008e0 con 0x7fe7941534e0 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 835131879 0 0) 0x7fe7940008e0 con 0x7fe7941534e0 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7941538b0 con 0x7fe7941534e0 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe7800051b0 con 0x7fe7941534e0 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient: dump: 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.430 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3658791079 0 0) 0x7fe7941538b0 con 0x7fe7941534e0 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe79408ac10 con 0x7fe7941534e0 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4515 (unknown 0 0 67215097) 0x7fe7800052e0 con 0x7fe7941534e0 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe7941534e0 legacy=0x7fe79408a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe76ffff640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6789/0 conn(0x7fe7941534e0 legacy=0x7fe79408a500 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.431 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 8, 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 23055, 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.432 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 17, 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.433 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.434 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:15:40.435 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.436 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.437 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.438 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.439 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.440 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:15:40.441 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.442 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.443 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:15:40.443 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.178:6800/2549949439 conn(0x7fe76403d880 legacy=0x7fe76405dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 10 monclient: shutdown 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 >> v1:10.20.193.102:6790/0 conn(0x7fe79408e6c0 legacy=0x7fe7941945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe793fff640 1 -- 10.20.193.102:0/2744276119 reap_dead start 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe793fff640 1 -- 10.20.193.102:0/2744276119 reap_dead start 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 shutdown_connections 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 >> 10.20.193.102:0/2744276119 conn(0x7fe794056b30 msgr2=0x7fe794141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 shutdown_connections 2026-02-21T22:15:40.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.409+0000 7fe799e3f640 1 -- 10.20.193.102:0/2744276119 wait complete. 2026-02-21T22:15:40.445 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:15:40.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:15:40.445 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:15:40.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: get_monmap_and_config 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: monmap: 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:40.515334+0000 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:40.515334+0000 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:40.516 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 1 Processor -- start 2026-02-21T22:15:40.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 1 -- start start 2026-02-21T22:15:40.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: init 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-b con 0x7f6968091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-c con 0x7f69680909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-a con 0x7f69681854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6968057a20 con 0x7f69681854b0 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6968091d20 con 0x7f69680909a0 2026-02-21T22:15:40.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6968057740 con 0x7f6968091240 2026-02-21T22:15:40.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696ec7d640 10 monclient(hunting): authenticate will time out at 1149.751484s 2026-02-21T22:15:40.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696d1f3640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 0x7f69681a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58766/0 (socket says 10.20.193.102:58766) 2026-02-21T22:15:40.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696c9f2640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f6968091240 0x7f6968091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60552/0 (socket says 10.20.193.102:60552) 2026-02-21T22:15:40.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.509+0000 7f696d1f3640 1 -- 10.20.193.102:0/442366388 learned_addr learned my addr 10.20.193.102:0/442366388 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696d1f3640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 legacy=0x7f69681a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696c9f2640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.102:6790/0 conn(0x7f6968091240 legacy=0x7f6968091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695ffff640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.178:6789/0 conn(0x7f69680909a0 legacy=0x7f6968183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169619408 0 0) 0x7f6968057a20 con 0x7f69681854b0 2026-02-21T22:15:40.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): my global_id is 4377 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6954003620 con 0x7f69681854b0 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854595 0 0) 0x7f6968057740 con 0x7f6968091240 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): my global_id is 4376 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968057a20 con 0x7f6968091240 2026-02-21T22:15:40.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785796366 0 0) 0x7f6968091d20 con 0x7f69680909a0 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient(hunting): my global_id is 4354 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968057740 con 0x7f69680909a0 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 566863031 0 0) 0x7f6968057a20 con 0x7f6968091240 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.178:6789/0 conn(0x7f69680909a0 legacy=0x7f6968183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 legacy=0x7f69681a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 monclient: found mon.noname-b 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6968090dc0 con 0x7f6968091240 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6958003110 con 0x7f6968091240 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: dump: 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.525 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 5 monclient: authenticate success, global_id 4376 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f69580033e0 con 0x7f6968091240 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 1 -- 10.20.193.102:0/442366388 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6958004a90 con 0x7f6968091240 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f695f7fe640 10 monclient: dump: 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.526 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 4 set_mon_vals no callback set 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 10 monclient: shutdown 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- 10.20.193.102:0/442366388 >> v1:10.20.193.102:6790/0 conn(0x7f6968091240 legacy=0x7f6968091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- 10.20.193.102:0/442366388 shutdown_connections 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- 10.20.193.102:0/442366388 >> 10.20.193.102:0/442366388 conn(0x7f696805e660 msgr2=0x7f6968095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- 10.20.193.102:0/442366388 shutdown_connections 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- 10.20.193.102:0/442366388 wait complete. 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 10 monclient: monmap: 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 Processor -- start 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 1 -- start start 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.513+0000 7f696ec7d640 10 monclient: init 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-b con 0x7f69680909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-c con 0x7f6968091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient(hunting): picked mon.noname-a con 0x7f69681854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f696812be50 con 0x7f69681854b0 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69681130a0 con 0x7f69680909a0 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6968112230 con 0x7f6968091240 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient(hunting): authenticate will time out at 1149.756296s 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696d1f3640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 0x7f6968174220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58782/0 (socket says 10.20.193.102:58782) 2026-02-21T22:15:40.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696d1f3640 1 -- 10.20.193.102:0/2303673216 learned_addr learned my addr 10.20.193.102:0/2303673216 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696d1f3640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 legacy=0x7f6968174220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695ffff640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f6968091240 legacy=0x7f6968172b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696c9f2640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.178:6789/0 conn(0x7f69680909a0 legacy=0x7f696812b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397627865 0 0) 0x7f6968112230 con 0x7f6968091240 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): my global_id is 4379 2026-02-21T22:15:40.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6938003620 con 0x7f6968091240 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3994383467 0 0) 0x7f696812be50 con 0x7f69681854b0 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): my global_id is 4380 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968112230 con 0x7f69681854b0 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004040870 0 0) 0x7f69681130a0 con 0x7f69680909a0 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): my global_id is 4357 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f696812be50 con 0x7f69680909a0 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2151486574 0 0) 0x7f6938003620 con 0x7f6968091240 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f69681130a0 con 0x7f6968091240 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f69500033c0 con 0x7f6968091240 2026-02-21T22:15:40.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): dump: 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:40.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4002450169 0 0) 0x7f6968112230 con 0x7f69681854b0 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6938003620 con 0x7f69681854b0 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6964005df0 con 0x7f69681854b0 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient(hunting): dump: 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.537 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3128058268 0 0) 0x7f69681130a0 con 0x7f6968091240 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.178:6789/0 conn(0x7f69680909a0 legacy=0x7f696812b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6789/0 conn(0x7f69681854b0 legacy=0x7f6968174220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 monclient: found mon.c 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6968110d40 con 0x7f6968091240 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 5 monclient: authenticate success, global_id 4379 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _renew_subs 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6968112800 con 0x7f6968091240 2026-02-21T22:15:40.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _renew_subs 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6968111a70 con 0x7f6968091240 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f69500039c0 con 0x7f6968091240 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6950003d50 con 0x7f6968091240 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 10 monclient: dump: 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.539 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695effd640 4 set_mon_vals no callback set 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6968110740 con 0x7f696812e6e0 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.517+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f6950003fe0 con 0x7f6968091240 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f695004c510 con 0x7f6968091240 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f696d1f3640 10 monclient: get_auth_request con 0x7f693803d5c0 auth_method 0 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f696c9f2640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696812e6e0 legacy=0x7f696808d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4161024697 0 0) 0x7f6968110740 con 0x7f696812e6e0 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient(hunting): my global_id is 4382 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69681130a0 con 0x7f696812e6e0 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 621129998 0 0) 0x7f69681130a0 con 0x7f696812e6e0 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6968110740 con 0x7f696812e6e0 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6958004fb0 con 0x7f696812e6e0 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient: dump: 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.541 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3612713190 0 0) 0x7f6968110740 con 0x7f696812e6e0 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69680008e0 con 0x7f696812e6e0 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f6958005040 con 0x7f696812e6e0 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696812e6e0 legacy=0x7f696808d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.521+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696812e6e0 legacy=0x7f696808d110 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.525+0000 7f696ec7d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:15:40.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.525+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f696812eab0 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696c9f2640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696808d820 legacy=0x7f6968176950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666323617 0 0) 0x7f696812eab0 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient(hunting): my global_id is 4385 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968110740 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3101369536 0 0) 0x7f6968110740 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f696812eab0 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6958005700 con 0x7f696808d820 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient: dump: 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.543 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3666408479 0 0) 0x7f696812eab0 con 0x7f696808d820 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6968095010 con 0x7f696808d820 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4513 (unknown 0 0 3133155677) 0x7f6958005a50 con 0x7f696808d820 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696808d820 legacy=0x7f6968176950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f695d7fa640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f696808d820 legacy=0x7f6968176950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 8, 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 23159, 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:15:40.545 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 17, 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.546 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.547 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.548 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.549 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:15:40.550 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:15:40.551 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:15:40.552 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:15:40.553 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:15:40.554 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:15:40.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.178:6800/2549949439 conn(0x7f693803d5c0 legacy=0x7f693805da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 10 monclient: shutdown 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 >> v1:10.20.193.102:6790/0 conn(0x7f6968091240 legacy=0x7f6968172b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696d1f3640 1 -- 10.20.193.102:0/2303673216 reap_dead start 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696d1f3640 1 -- 10.20.193.102:0/2303673216 reap_dead start 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 shutdown_connections 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 >> 10.20.193.102:0/2303673216 conn(0x7f696805e660 msgr2=0x7f696808b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 shutdown_connections 2026-02-21T22:15:40.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.529+0000 7f696ec7d640 1 -- 10.20.193.102:0/2303673216 wait complete. 2026-02-21T22:15:40.556 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:15:40.556 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:15:40.556 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: get_monmap_and_config 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: monmap: 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:40.623835+0000 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:40.623835+0000 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.624 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:40.625 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:40.625 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 1 Processor -- start 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 1 -- start start 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: init 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-b con 0x7f24a408cd00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-c con 0x7f24a408d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-a con 0x7f24a408b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a40890e0 con 0x7f24a408b8d0 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a4088ea0 con 0x7f24a408d7e0 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a408aea0 con 0x7f24a408cd00 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24aa472640 10 monclient(hunting): authenticate will time out at 1149.859739s 2026-02-21T22:15:40.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24a3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f24a408cd00 0x7f24a408d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60592/0 (socket says 10.20.193.102:60592) 2026-02-21T22:15:40.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24a3fff640 1 -- 10.20.193.102:0/3012424129 learned_addr learned my addr 10.20.193.102:0/3012424129 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.617+0000 7f24a89e8640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f24a408b8d0 0x7f24a41a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58794/0 (socket says 10.20.193.102:58794) 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a3fff640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.102:6790/0 conn(0x7f24a408cd00 legacy=0x7f24a408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a89e8640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.102:6789/0 conn(0x7f24a408b8d0 legacy=0x7f24a41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a37fe640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.178:6789/0 conn(0x7f24a408d7e0 legacy=0x7f24a41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072223423 0 0) 0x7f24a408aea0 con 0x7f24a408cd00 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): my global_id is 4388 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2488003620 con 0x7f24a408cd00 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3763351388 0 0) 0x7f24a40890e0 con 0x7f24a408b8d0 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): my global_id is 4383 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a408aea0 con 0x7f24a408b8d0 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180061591 0 0) 0x7f24a4088ea0 con 0x7f24a408d7e0 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient(hunting): my global_id is 4360 2026-02-21T22:15:40.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a40890e0 con 0x7f24a408d7e0 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1576239880 0 0) 0x7f2488003620 con 0x7f24a408cd00 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.178:6789/0 conn(0x7f24a408d7e0 legacy=0x7f24a41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.102:6789/0 conn(0x7f24a408b8d0 legacy=0x7f24a41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 monclient: found mon.noname-b 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a408a600 con 0x7f24a408cd00 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2490003110 con 0x7f24a408cd00 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 5 monclient: authenticate success, global_id 4388 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: dump: 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f24900033e0 con 0x7f24a408cd00 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 1 -- 10.20.193.102:0/3012424129 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2490004b50 con 0x7f24a408cd00 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24a2ffd640 10 monclient: dump: 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 10 monclient: get_monmap_and_config success 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 4 set_mon_vals no callback set 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 10 monclient: shutdown 2026-02-21T22:15:40.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- 10.20.193.102:0/3012424129 >> v1:10.20.193.102:6790/0 conn(0x7f24a408cd00 legacy=0x7f24a408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- 10.20.193.102:0/3012424129 shutdown_connections 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- 10.20.193.102:0/3012424129 >> 10.20.193.102:0/3012424129 conn(0x7f24a4091a50 msgr2=0x7f24a4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- 10.20.193.102:0/3012424129 shutdown_connections 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- 10.20.193.102:0/3012424129 wait complete. 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 10 monclient: build_initial_monmap 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 10 monclient: monmap: 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:15:40.635 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 Processor -- start 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 1 -- start start 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.621+0000 7f24aa472640 10 monclient: init 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-c con 0x7f24a408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-b con 0x7f24a408cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient(hunting): picked mon.noname-a con 0x7f24a408d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a40820b0 con 0x7f24a408d7e0 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a407fd50 con 0x7f24a408cd00 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a4081810 con 0x7f24a408b8d0 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient(hunting): authenticate will time out at 1149.864404s 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a89e8640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f24a408d7e0 0x7f24a4195720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58802/0 (socket says 10.20.193.102:58802) 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a89e8640 1 -- 10.20.193.102:0/2774334312 learned_addr learned my addr 10.20.193.102:0/2774334312 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:40.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f24a408b8d0 0x7f24a41533e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60600/0 (socket says 10.20.193.102:60600) 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a89e8640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.102:6789/0 conn(0x7f24a408d7e0 legacy=0x7f24a4195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a3fff640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.102:6790/0 conn(0x7f24a408b8d0 legacy=0x7f24a41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a37fe640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a408cd00 legacy=0x7f24a4153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 400605893 0 0) 0x7f24a40820b0 con 0x7f24a408d7e0 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): my global_id is 4386 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2474003620 con 0x7f24a408d7e0 2026-02-21T22:15:40.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388769569 0 0) 0x7f24a4081810 con 0x7f24a408b8d0 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): my global_id is 4391 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a40820b0 con 0x7f24a408b8d0 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411409567 0 0) 0x7f24a407fd50 con 0x7f24a408cd00 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): my global_id is 4363 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a4081810 con 0x7f24a408cd00 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 958952181 0 0) 0x7f2474003620 con 0x7f24a408d7e0 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24a407fd50 con 0x7f24a408d7e0 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2498004450 con 0x7f24a408d7e0 2026-02-21T22:15:40.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:15:40.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1404586192 0 0) 0x7f24a40820b0 con 0x7f24a408b8d0 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2474003620 con 0x7f24a408b8d0 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f24900048e0 con 0x7f24a408b8d0 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient(hunting): dump: 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4272510461 0 0) 0x7f24a407fd50 con 0x7f24a408d7e0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.102:6790/0 conn(0x7f24a408b8d0 legacy=0x7f24a41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a408cd00 legacy=0x7f24a4153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 monclient: found mon.a 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4081af0 con 0x7f24a408d7e0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: _finish_auth 0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 5 monclient: authenticate success, global_id 4386 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _renew_subs 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4082ab0 con 0x7f24a408d7e0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _renew_subs 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:15:40.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f24a4080a80 con 0x7f24a408d7e0 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f24980060a0 con 0x7f24a408d7e0 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2498006400 con 0x7f24a408d7e0 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 10 monclient: dump: 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.645 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a27fc640 4 set_mon_vals no callback set 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f24a40008e0 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2498012a90 con 0x7f24a408d7e0 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.625+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7f249804d8f0 con 0x7f24a408d7e0 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a37fe640 10 monclient: get_auth_request con 0x7f247403d800 auth_method 0 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a3fff640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407f390 legacy=0x7f24a407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550469643 0 0) 0x7f24a40008e0 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient(hunting): my global_id is 4366 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a407fd50 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3837051986 0 0) 0x7f24a407fd50 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24a40008e0 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2490005070 con 0x7f24a407f390 2026-02-21T22:15:40.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient: dump: 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2356421519 0 0) 0x7f24a40008e0 con 0x7f24a407f390 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24a4090810 con 0x7f24a407f390 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2490005070 con 0x7f24a407f390 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:15:40.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:15:40.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407f390 legacy=0x7f24a407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.629+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407f390 legacy=0x7f24a407a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.633+0000 7f24aa472640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:15:40.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.633+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f24a40909c0 con 0x7f24a407aa80 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a3fff640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407aa80 legacy=0x7f24a4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330235224 0 0) 0x7f24a40909c0 con 0x7f24a407aa80 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient(hunting): my global_id is 4369 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a40008e0 con 0x7f24a407aa80 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2200564167 0 0) 0x7f24a40008e0 con 0x7f24a407aa80 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24a40909c0 con 0x7f24a407aa80 2026-02-21T22:15:40.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2490005750 con 0x7f24a407aa80 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient: dump: 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1784911178 0 0) 0x7f24a40909c0 con 0x7f24a407aa80 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f24a407ae50 con 0x7f24a407aa80 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4680 (unknown 0 0 1648465220) 0x7f24900059f0 con 0x7f24a407aa80 2026-02-21T22:15:40.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407aa80 legacy=0x7f24a4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a0ff9640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6789/0 conn(0x7f24a407aa80 legacy=0x7f24a4197e50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 8, 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 23243, 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:15:40.651 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 17, 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.652 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.653 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:15:40.654 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:15:40.655 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:15:40.656 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:15:40.657 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:15:40.658 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:15:40.659 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:15:40.660 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.178:6800/2549949439 conn(0x7f247403d800 legacy=0x7f247405dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 10 monclient: shutdown 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 >> v1:10.20.193.102:6789/0 conn(0x7f24a408d7e0 legacy=0x7f24a4195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24a89e8640 1 -- 10.20.193.102:0/2774334312 reap_dead start 2026-02-21T22:15:40.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 shutdown_connections 2026-02-21T22:15:40.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 >> 10.20.193.102:0/2774334312 conn(0x7f24a4091a50 msgr2=0x7f24a4085760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:15:40.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 shutdown_connections 2026-02-21T22:15:40.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:40.637+0000 7f24aa472640 1 -- 10.20.193.102:0/2774334312 wait complete. 2026-02-21T22:15:40.662 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:15:40.662 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2026-02-21T22:15:40.662 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2026-02-21T22:15:40.662 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2026-02-21T22:15:40.662 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2026-02-21T22:15:40.662 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T22:15:40.662 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:15:40.674 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T22:15:40.674 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.b 2026-02-21T22:15:40.674 INFO:tasks.mon_thrash:Sending STOP to mon b 2026-02-21T22:15:40.674 INFO:tasks.ceph.mon.b:Sent signal 19 2026-02-21T22:15:40.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:15:43.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:43.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:43.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:43.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:43.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:43.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:43.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:43.165+0000 7fd6177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:43.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:43.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:44.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:44.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:44.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:44.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:44.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:44.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:45.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:45.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:45.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:45.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:45.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:45.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:46.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:46.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:46.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:47.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:47.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:47.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:47.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:47.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:47.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:48.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:48.161+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:48.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:48.161+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:48.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:48.161+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:48.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:48.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:49.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:49.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:49.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:49.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:49.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:49.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:50.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:50.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:50.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:50.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.173+0000 7fd6397fa640 10 monclient: _renew_subs 2026-02-21T22:15:50.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.173+0000 7fd6397fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:15:50.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:50.173+0000 7fd6397fa640 1 -- 10.20.193.102:0/425851846 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=20}) -- 0x7fd6100035a0 con 0x7fd63c0901f0 2026-02-21T22:15:51.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:51.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:51.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:51.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:51.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:51.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:52.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:52.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:52.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:52.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:52.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:52.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:53.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:53.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:53.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:53.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:53.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:53.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:53.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:53.165+0000 7fd6177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:15:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:15:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:15:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:53.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:15:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:15:53.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:15:53.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:15:54.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:54.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:54.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:54.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:54.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:54.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:55.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.161+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:55.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.161+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:55.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.161+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:55.675 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.b 2026-02-21T22:15:55.676 INFO:tasks.mon_thrash:Sending CONT to mon b 2026-02-21T22:15:55.676 INFO:tasks.ceph.mon.b:Sent signal 18 2026-02-21T22:15:55.676 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:15:55.676 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:15:55.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: get_monmap_and_config 2026-02-21T22:15:55.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: build_initial_monmap 2026-02-21T22:15:55.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: monmap: 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:55.746907+0000 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:55.746907+0000 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:15:55.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 1 Processor -- start 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 1 -- start start 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: init 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: _reopen_session rank -1 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-b con 0x7fceec08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-c con 0x7fceec08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-a con 0x7fceec08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec086a10 con 0x7fceec08d8e0 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec08b130 con 0x7fceec08b7f0 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec08bd60 con 0x7fceec08ce00 2026-02-21T22:15:55.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:55.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fcef2081640 10 monclient(hunting): authenticate will time out at 1164.982858s 2026-02-21T22:15:55.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fceebfff640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:55.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fceeb7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 0x7fceec08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60202/0 (socket says 10.20.193.102:60202) 2026-02-21T22:15:55.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.741+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 learned_addr learned my addr 10.20.193.102:0/991594401 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:15:55.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.745+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fceec08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:55.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.745+0000 7fceeaffd640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.178:6789/0 conn(0x7fceec08b7f0 legacy=0x7fceec181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:55.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:55.945+0000 7fceebfff640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:56.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:56.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:56.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:56.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:56.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:56.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:56.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:56.349+0000 7fceebfff640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:57.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:57.157+0000 7fceebfff640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:57.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:57.161+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:57.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:57.161+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:57.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:57.161+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:58.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:58.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:58.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:58.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:15:58.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:15:58.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): tick 2026-02-21T22:15:58.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:15:58.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 monclient(hunting): continuing hunt 2026-02-21T22:15:58.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:15:58.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fceec08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.178:6789/0 conn(0x7fceec08b7f0 legacy=0x7fceec181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec1a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): picked mon.noname-a con 0x7fceec08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): picked mon.noname-b con 0x7fceec08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): picked mon.noname-c con 0x7fceec08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fcec80238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec08b370 con 0x7fceec08d8e0 2026-02-21T22:15:58.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec086a10 con 0x7fceec08b7f0 2026-02-21T22:15:58.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 1 -- 10.20.193.102:0/991594401 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcec806a0f0 con 0x7fceec08ce00 2026-02-21T22:15:58.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.741+0000 7fcee97fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceeaffd640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fcec80450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceeaffd640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fcec80450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceeaffd640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fcec80450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceeaffd640 1 --1- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 0x7fcec80450a0 :-1 s=OPENED pgs=147 gs=9 cs=1 l=1).handle_message read tag failed 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceeaffd640 1 --1- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 0x7fcec80450a0 :-1 s=OPENED pgs=147 gs=9 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceea7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.102:6790/0 2026-02-21T22:15:58.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.745+0000 7fceebfff640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.178:6789/0 conn(0x7fceec08b7f0 legacy=0x7fcec80668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:15:58.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:58.945+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fcec80238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:15:59.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:59.165+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:15:59.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:59.165+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:15:59.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:59.165+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:15:59.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:15:59.349+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fcec80238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.153+0000 7fceeb7fe640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fcec80238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.161+0000 7fd6177fe640 10 monclient: tick 2026-02-21T22:16:00.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.161+0000 7fd6177fe640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.161+0000 7fd6177fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887804740 0 0) 0x7fceec086a10 con 0x7fceec08b7f0 2026-02-21T22:16:00.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 10 monclient(hunting): my global_id is 4372 2026-02-21T22:16:00.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.717+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced0003620 con 0x7fceec08b7f0 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474310961 0 0) 0x7fced0003620 con 0x7fceec08b7f0 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6790/0 conn(0x7fceec08ce00 legacy=0x7fcec80450a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.102:6789/0 conn(0x7fceec08d8e0 legacy=0x7fcec80238d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 monclient: found mon.noname-c 2026-02-21T22:16:00.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec806b3f0 con 0x7fceec08b7f0 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 5 monclient: authenticate success, global_id 4372 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcee0005a80 con 0x7fceec08b7f0 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:16:00.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: dump: 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcee0005d50 con 0x7fceec08b7f0 2026-02-21T22:16:00.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 10 monclient: get_monmap_and_config success 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 4 set_mon_vals no callback set 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 1 -- 10.20.193.102:0/991594401 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcee0006160 con 0x7fceec08b7f0 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fceea7fc640 10 monclient: dump: 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.736 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 10 monclient: shutdown 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 1 -- 10.20.193.102:0/991594401 >> v1:10.20.193.178:6789/0 conn(0x7fceec08b7f0 legacy=0x7fcec80668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 1 -- 10.20.193.102:0/991594401 shutdown_connections 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 1 -- 10.20.193.102:0/991594401 >> 10.20.193.102:0/991594401 conn(0x7fceec091a50 msgr2=0x7fceec091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 1 -- 10.20.193.102:0/991594401 shutdown_connections 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 1 -- 10.20.193.102:0/991594401 wait complete. 2026-02-21T22:16:00.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 10 monclient: build_initial_monmap 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.725+0000 7fcef2081640 10 monclient: monmap: 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 1 Processor -- start 2026-02-21T22:16:00.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 1 -- start start 2026-02-21T22:16:00.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient: init 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-c con 0x7fceec08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-a con 0x7fceec08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient(hunting): picked mon.noname-b con 0x7fceec08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec088a70 con 0x7fceec08ce00 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec080f10 con 0x7fceec08d8e0 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec083270 con 0x7fceec08b7f0 2026-02-21T22:16:00.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcef2081640 10 monclient(hunting): authenticate will time out at 1169.968879s 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fceeaffd640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fceec08ce00 legacy=0x7fceec21af10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fceeb7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fceec08b7f0 0x7fceec0847a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60224/0 (socket says 10.20.193.102:60224) 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fceeb7fe640 1 -- 10.20.193.102:0/592536853 learned_addr learned my addr 10.20.193.102:0/592536853 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fceeb7fe640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.102:6790/0 conn(0x7fceec08b7f0 legacy=0x7fceec0847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fceebfff640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.178:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec23c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220019361 0 0) 0x7fceec080f10 con 0x7fceec08d8e0 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 10 monclient(hunting): my global_id is 4375 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.729+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcebc003620 con 0x7fceec08d8e0 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 857389697 0 0) 0x7fcebc003620 con 0x7fceec08d8e0 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fceec080f10 con 0x7fceec08d8e0 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcee00060b0 con 0x7fceec08d8e0 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient(hunting): dump: 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.745 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1075313546 0 0) 0x7fceec080f10 con 0x7fceec08d8e0 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.102:6790/0 conn(0x7fceec08b7f0 legacy=0x7fceec0847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.102:6789/0 conn(0x7fceec08ce00 legacy=0x7fceec21af10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 monclient: found mon.b 2026-02-21T22:16:00.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec084eb0 con 0x7fceec08d8e0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 5 monclient: authenticate success, global_id 4375 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _renew_subs 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fceec081a70 con 0x7fceec08d8e0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _renew_subs 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fceec1d9dc0 con 0x7fceec08d8e0 2026-02-21T22:16:00.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcee0005d50 con 0x7fceec08d8e0 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcee9ffb640 4 set_mon_vals no callback set 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcee00066b0 con 0x7fceec08d8e0 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 10 monclient: dump: 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.748 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec0008e0 con 0x7fceec08d8e0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fcee0012d40 con 0x7fceec08d8e0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(19..19 src has 1..19) ==== 3559+0+0 (unknown 1576237739 0 0) 0x7fcee004d840 con 0x7fceec08d8e0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.733+0000 7fceeb7fe640 10 monclient: get_auth_request con 0x7fcebc03d340 auth_method 0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.737+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fcee0018a70 con 0x7fceec08d8e0 2026-02-21T22:16:00.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.737+0000 7fcec7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.737+0000 7fcec7fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:16:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.777+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 10 ==== osd_map(20..20 src has 1..20) ==== 643+0+0 (unknown 2063888876 0 0) 0x7fd62804a550 con 0x7fd63c0901f0 2026-02-21T22:16:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.777+0000 7fd638ff9640 1 -- 10.20.193.102:0/425851846 <== mon.2 v1:10.20.193.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]=0 pool 'unique_pool_0' created v20) ==== 130+0+0 (unknown 3098031066 0 0) 0x7fd6280170b0 con 0x7fd63c0901f0 2026-02-21T22:16:00.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.777+0000 7fd638ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2026-02-21T22:16:00.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.777+0000 7fd638ff9640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' created 2026-02-21T22:16:00.783 INFO:teuthology.orchestra.run.trial102.stderr:pool 'unique_pool_0' created 2026-02-21T22:16:00.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd60c03d880 legacy=0x7fd60c05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 10 monclient: shutdown 2026-02-21T22:16:00.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 >> v1:10.20.193.102:6790/0 conn(0x7fd63c0901f0 legacy=0x7fd63c13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 shutdown_connections 2026-02-21T22:16:00.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 >> 10.20.193.102:0/425851846 conn(0x7fd63c05e670 msgr2=0x7fd63c08a360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 shutdown_connections 2026-02-21T22:16:00.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.781+0000 7fd6429c5640 1 -- 10.20.193.102:0/425851846 wait complete. 2026-02-21T22:16:00.797 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-21T22:16:00.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcef2081640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:16:00.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcef2081640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:16:00.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcef2081640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fceec085080 con 0x7fceec08d8e0 2026-02-21T22:16:00.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcec7fff640 1 -- 10.20.193.102:0/592536853 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 2252860960) 0x7fcee0018350 con 0x7fceec08d8e0 2026-02-21T22:16:00.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcec7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:16:00.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.853+0000 7fcec7fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:16:00.861 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":10,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.178:6800/2549949439 conn(0x7fcebc03d340 legacy=0x7fcebc05d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 10 monclient: shutdown 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 >> v1:10.20.193.178:6789/0 conn(0x7fceec08d8e0 legacy=0x7fceec23c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 shutdown_connections 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 >> 10.20.193.102:0/592536853 conn(0x7fceec091a50 msgr2=0x7fceec088d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 shutdown_connections 2026-02-21T22:16:00.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7fcef2081640 1 -- 10.20.193.102:0/592536853 wait complete. 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: get_monmap_and_config 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: build_initial_monmap 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: monmap: 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:16:00.862649+0000 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:16:00.862649+0000 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:00.863 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:16:00.864 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:16:00.864 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 1 Processor -- start 2026-02-21T22:16:00.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 1 -- start start 2026-02-21T22:16:00.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: init 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-b con 0x7f0fa408c1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-c con 0x7f0fa408cc80 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-a con 0x7f0fa41834f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa408d4a0 con 0x7f0fa41834f0 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa408d050 con 0x7f0fa408cc80 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0faa9df640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7f0fa41834f0 legacy=0x7f0fa41a38d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa4088730 con 0x7f0fa408c1a0 2026-02-21T22:16:00.866 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:00.866 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0fac469640 10 monclient(hunting): authenticate will time out at 1170.098977s 2026-02-21T22:16:00.866 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0faa1de640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 0x7f0fa408c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60232/0 (socket says 10.20.193.102:60232) 2026-02-21T22:16:00.866 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.857+0000 7f0faa1de640 1 -- 10.20.193.102:0/2703375146 learned_addr learned my addr 10.20.193.102:0/2703375146 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:00.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0faa1de640 1 -- 10.20.193.102:0/2703375146 >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 legacy=0x7f0fa408c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 882725231 0 0) 0x7f0fa4088730 con 0x7f0fa408c1a0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient(hunting): my global_id is 4394 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f8c003620 con 0x7f0fa408c1a0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa99dd640 1 -- 10.20.193.102:0/2703375146 >> v1:10.20.193.178:6789/0 conn(0x7f0fa408cc80 legacy=0x7f0fa4181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3760562521 0 0) 0x7f0f8c003620 con 0x7f0fa408c1a0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 >> v1:10.20.193.178:6789/0 conn(0x7f0fa408cc80 legacy=0x7f0fa4181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 >> v1:10.20.193.102:6789/0 conn(0x7f0fa41834f0 legacy=0x7f0fa41a38d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 monclient: found mon.noname-b 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa40857a0 con 0x7f0fa408c1a0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0f94003110 con 0x7f0fa408c1a0 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 5 monclient: authenticate success, global_id 4394 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: dump: 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0f94003450 con 0x7f0fa408c1a0 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 1 -- 10.20.193.102:0/2703375146 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0f94004b50 con 0x7f0fa408c1a0 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: get_monmap_and_config success 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 4 set_mon_vals no callback set 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa91dc640 10 monclient: dump: 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.871 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: shutdown 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- 10.20.193.102:0/2703375146 >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 legacy=0x7f0fa408c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- 10.20.193.102:0/2703375146 shutdown_connections 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- 10.20.193.102:0/2703375146 >> 10.20.193.102:0/2703375146 conn(0x7f0fa40917a0 msgr2=0x7f0fa4091ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- 10.20.193.102:0/2703375146 shutdown_connections 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- 10.20.193.102:0/2703375146 wait complete. 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: build_initial_monmap 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: monmap: 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:16:00.872 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 Processor -- start 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- start start 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: init 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-c con 0x7f0fa408c1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-a con 0x7f0fa408cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient(hunting): picked mon.noname-b con 0x7f0fa41834f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa407fb80 con 0x7f0fa408cc80 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa40828c0 con 0x7f0fa41834f0 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa4081640 con 0x7f0fa408c1a0 2026-02-21T22:16:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:00.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fac469640 10 monclient(hunting): authenticate will time out at 1170.102929s 2026-02-21T22:16:00.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0fa99dd640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7f0fa408cc80 legacy=0x7f0fa4181dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0faa1de640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 0x7f0fa41411c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60248/0 (socket says 10.20.193.102:60248) 2026-02-21T22:16:00.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.861+0000 7f0faa1de640 1 -- 10.20.193.102:0/1350553507 learned_addr learned my addr 10.20.193.102:0/1350553507 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0faa1de640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 legacy=0x7f0fa41411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0faa1de640 1 --1- 10.20.193.102:0/1350553507 >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 0x7f0fa41411c0 :-1 s=OPENED pgs=150 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f0fa4081640 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0faa9df640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.178:6789/0 conn(0x7f0fa41834f0 legacy=0x7f0fa4123930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195956587 0 0) 0x7f0fa40828c0 con 0x7f0fa41834f0 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): my global_id is 4381 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f74003620 con 0x7f0fa41834f0 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3434919402 0 0) 0x7f0f74003620 con 0x7f0fa41834f0 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0fa40828c0 con 0x7f0fa41834f0 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0fa0005870 con 0x7f0fa41834f0 2026-02-21T22:16:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): dump: 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 619312601 0 0) 0x7f0fa40828c0 con 0x7f0fa41834f0 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.102:6790/0 conn(0x7f0fa408c1a0 legacy=0x7f0fa41411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.102:6789/0 conn(0x7f0fa408cc80 legacy=0x7f0fa4181dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 monclient: found mon.b 2026-02-21T22:16:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa4081920 con 0x7f0fa41834f0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547436702 0 0) 0x7f0fa4081640 con 0x7f0fa408c1a0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 5 monclient: authenticate success, global_id 4381 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _renew_subs 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa4085440 con 0x7f0fa41834f0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _renew_subs 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0fa40808b0 con 0x7f0fa41834f0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa40008e0 con 0x7f0fa41834f0 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0fa0005b40 con 0x7f0fa41834f0 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0fa0005ee0 con 0x7f0fa41834f0 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 10 monclient: dump: 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0fa89db640 4 set_mon_vals no callback set 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0fa0006180 con 0x7f0fa41834f0 2026-02-21T22:16:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(20..20 src has 1..20) ==== 3886+0+0 (unknown 4169619608 0 0) 0x7f0fa004d960 con 0x7f0fa41834f0 2026-02-21T22:16:00.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.865+0000 7f0faa1de640 10 monclient: get_auth_request con 0x7f0f7403d390 auth_method 0 2026-02-21T22:16:00.880 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:16:00.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:16:00.880 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:16:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.869+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0fa0018940 con 0x7f0fa41834f0 2026-02-21T22:16:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.869+0000 7f0f8affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.869+0000 7f0f8affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:16:00.941 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: get_monmap_and_config 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: build_initial_monmap 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: monmap: 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:16:00.941588+0000 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:16:00.941588+0000 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:16:00.942 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 1 Processor -- start 2026-02-21T22:16:00.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 1 -- start start 2026-02-21T22:16:00.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: init 2026-02-21T22:16:00.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-b con 0x7fa2a8091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-c con 0x7fa2a80909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-a con 0x7fa2a81854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8057a20 con 0x7fa2a81854b0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8091d20 con 0x7fa2a80909a0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2a77fe640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fa2a81854b0 legacy=0x7fa2a81a5890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8057740 con 0x7fa2a8091240 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2ad60c640 10 monclient(hunting): authenticate will time out at 1170.178143s 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2a6ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa2a8091240 0x7fa2a8091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60252/0 (socket says 10.20.193.102:60252) 2026-02-21T22:16:00.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.937+0000 7fa2a6ffd640 1 -- 10.20.193.102:0/247820577 learned_addr learned my addr 10.20.193.102:0/247820577 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:00.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a6ffd640 1 -- 10.20.193.102:0/247820577 >> v1:10.20.193.102:6790/0 conn(0x7fa2a8091240 legacy=0x7fa2a8091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a67fc640 1 -- 10.20.193.102:0/247820577 >> v1:10.20.193.178:6789/0 conn(0x7fa2a80909a0 legacy=0x7fa2a8183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 506990711 0 0) 0x7fa2a8057740 con 0x7fa2a8091240 2026-02-21T22:16:00.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): my global_id is 4400 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa294003620 con 0x7fa2a8091240 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914218372 0 0) 0x7fa2a8091d20 con 0x7fa2a80909a0 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient(hunting): my global_id is 4384 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a8057740 con 0x7fa2a80909a0 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219466068 0 0) 0x7fa294003620 con 0x7fa2a8091240 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 >> v1:10.20.193.178:6789/0 conn(0x7fa2a80909a0 legacy=0x7fa2a8183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 >> v1:10.20.193.102:6789/0 conn(0x7fa2a81854b0 legacy=0x7fa2a81a5890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 monclient: found mon.noname-b 2026-02-21T22:16:00.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8090dc0 con 0x7fa2a8091240 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa29c003110 con 0x7fa2a8091240 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: dump: 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.949 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 5 monclient: authenticate success, global_id 4400 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa29c0033e0 con 0x7fa2a8091240 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 1 -- 10.20.193.102:0/247820577 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa29c004b50 con 0x7fa2a8091240 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a5ffb640 10 monclient: dump: 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.950 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: get_monmap_and_config success 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 4 set_mon_vals no callback set 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: shutdown 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- 10.20.193.102:0/247820577 >> v1:10.20.193.102:6790/0 conn(0x7fa2a8091240 legacy=0x7fa2a8091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- 10.20.193.102:0/247820577 shutdown_connections 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- 10.20.193.102:0/247820577 >> 10.20.193.102:0/247820577 conn(0x7fa2a805e660 msgr2=0x7fa2a8095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- 10.20.193.102:0/247820577 shutdown_connections 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- 10.20.193.102:0/247820577 wait complete. 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: build_initial_monmap 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: monmap: 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:16:00.951 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 Processor -- start 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- start start 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: init 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-c con 0x7fa2a80909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-a con 0x7fa2a8091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient(hunting): picked mon.noname-b con 0x7fa2a81854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:00.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a812bc60 con 0x7fa2a8091240 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8112eb0 con 0x7fa2a81854b0 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8112040 con 0x7fa2a80909a0 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a67fc640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fa2a8091240 legacy=0x7fa2a8172910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2ad60c640 10 monclient(hunting): authenticate will time out at 1170.182134s 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a6ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa2a80909a0 0x7fa2a812b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60258/0 (socket says 10.20.193.102:60258) 2026-02-21T22:16:00.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.941+0000 7fa2a6ffd640 1 -- 10.20.193.102:0/3000060670 learned_addr learned my addr 10.20.193.102:0/3000060670 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2a6ffd640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a80909a0 legacy=0x7fa2a812b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2a77fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.178:6789/0 conn(0x7fa2a81854b0 legacy=0x7fa2a8174030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418213287 0 0) 0x7fa2a8112040 con 0x7fa2a80909a0 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): my global_id is 4403 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa274003620 con 0x7fa2a80909a0 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2978078021 0 0) 0x7fa274003620 con 0x7fa2a80909a0 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa2a8112040 con 0x7fa2a80909a0 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227189590 0 0) 0x7fa2a8112eb0 con 0x7fa2a81854b0 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): my global_id is 4387 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa274003620 con 0x7fa2a81854b0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa29c002b70 con 0x7fa2a80909a0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): dump: 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 658720463 0 0) 0x7fa2a8112040 con 0x7fa2a80909a0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.178:6789/0 conn(0x7fa2a81854b0 legacy=0x7fa2a8174030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6789/0 conn(0x7fa2a8091240 legacy=0x7fa2a8172910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: _finish_hunting 0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 monclient: found mon.c 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8110b50 con 0x7fa2a80909a0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: _finish_auth 0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: _check_auth_tickets 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 5 monclient: authenticate success, global_id 4403 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 10 monclient: _renew_subs 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a8112610 con 0x7fa2a80909a0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 10 monclient: _renew_subs 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:00.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa2a8111880 con 0x7fa2a80909a0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa29c0033c0 con 0x7fa2a80909a0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa29c003570 con 0x7fa2a80909a0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2a57fa640 4 set_mon_vals no callback set 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: dump: 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa2a81105f0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa29c011810 con 0x7fa2a80909a0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(20..20 src has 1..20) ==== 3886+0+0 (unknown 4169619608 0 0) 0x7fa29c04c6a0 con 0x7fa2a80909a0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2a77fe640 10 monclient: get_auth_request con 0x7fa27403d490 auth_method 0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2a67fc640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a812e4f0 legacy=0x7fa2a808d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874876576 0 0) 0x7fa2a81105f0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient(hunting): my global_id is 4406 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a8112040 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1760700747 0 0) 0x7fa2a8112040 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa2a81105f0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa290003280 con 0x7fa2a812e4f0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 10 monclient: dump: 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 663990306 0 0) 0x7fa2a81105f0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.945+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2a80008e0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fa2900058c0 con 0x7fa2a812e4f0 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2877fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2877fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a812e4f0 legacy=0x7fa2a808d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a812e4f0 legacy=0x7fa2a808d240 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2ad60c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:16:00.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.949+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa2a812e8c0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2a67fc640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a808d950 legacy=0x7fa2a8176760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652984673 0 0) 0x7fa2a812e8c0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient(hunting): my global_id is 4409 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a81105f0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2210558172 0 0) 0x7fa2a81105f0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa2a812e8c0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa290003ab0 con 0x7fa2a808d950 2026-02-21T22:16:00.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient: dump: 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3486632671 0 0) 0x7fa2a812e8c0 con 0x7fa2a808d950 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa2a8095010 con 0x7fa2a808d950 2026-02-21T22:16:00.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7f0fac469640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7f0fac469640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4502 (unknown 0 0 2912889776) 0x7fa290005af0 con 0x7fa2a808d950 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7f0fac469640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) -- 0x7f0fa4090810 con 0x7f0fa41834f0 2026-02-21T22:16:00.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a808d950 legacy=0x7fa2a8176760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2877fe640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a808d950 legacy=0x7fa2a8176760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 10, 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 43582, 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:00.963 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.964 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:00.965 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:00.966 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.967 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:00.968 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:16:00.969 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa27403d490 legacy=0x7fa27405d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 10 monclient: shutdown 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 >> v1:10.20.193.102:6790/0 conn(0x7fa2a80909a0 legacy=0x7fa2a812b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2a77fe640 1 -- 10.20.193.102:0/3000060670 reap_dead start 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2a77fe640 1 -- 10.20.193.102:0/3000060670 reap_dead start 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 shutdown_connections 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 >> 10.20.193.102:0/3000060670 conn(0x7fa2a805e660 msgr2=0x7fa2a808b910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 shutdown_connections 2026-02-21T22:16:00.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:00.953+0000 7fa2ad60c640 1 -- 10.20.193.102:0/3000060670 wait complete. 2026-02-21T22:16:00.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:16:00.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-21T22:16:00.971 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 10 monclient: get_monmap_and_config 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 10 monclient: build_initial_monmap 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 10 monclient: monmap: 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:16:01.040093+0000 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:16:01.040093+0000 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:16:01.040 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 1 Processor -- start 2026-02-21T22:16:01.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 1 -- start start 2026-02-21T22:16:01.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.033+0000 7fd6ab577640 10 monclient: init 2026-02-21T22:16:01.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-c con 0x7fd6ac13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-b con 0x7fd6ac13fd90 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-a con 0x7fd6ac1401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac092490 con 0x7fd6ac1401f0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac0571f0 con 0x7fd6ac13f9c0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac0574d0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient(hunting): authenticate will time out at 1170.276194s 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6aad76640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fd6ac1401f0 legacy=0x7fd6ac1a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 0x7fd6ac183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60292/0 (socket says 10.20.193.102:60292) 2026-02-21T22:16:01.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9d74640 1 -- 10.20.193.102:0/2570969069 learned_addr learned my addr 10.20.193.102:0/2570969069 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:01.043 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9d74640 1 -- 10.20.193.102:0/2570969069 >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 legacy=0x7fd6ac183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068014495 0 0) 0x7fd6ac0574d0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient(hunting): _negotiate 2026-02-21T22:16:01.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:01.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient(hunting): my global_id is 4412 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd698003620 con 0x7fd6ac13fd90 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6aa575640 1 -- 10.20.193.102:0/2570969069 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac13f9c0 legacy=0x7fd6ac0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634570185 0 0) 0x7fd698003620 con 0x7fd6ac13fd90 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac13f9c0 legacy=0x7fd6ac0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 >> v1:10.20.193.102:6789/0 conn(0x7fd6ac1401f0 legacy=0x7fd6ac1a5470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: _finish_hunting 0 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 monclient: found mon.noname-b 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac0577b0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: _finish_auth 0 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: _check_auth_tickets 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:01.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd694003110 con 0x7fd6ac13fd90 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: dump: 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 5 monclient: authenticate success, global_id 4412 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:16:01.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6940033e0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 1 -- 10.20.193.102:0/2570969069 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd694004b50 con 0x7fd6ac13fd90 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6a9573640 10 monclient: dump: 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: get_monmap_and_config success 2026-02-21T22:16:01.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 4 set_mon_vals no callback set 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: shutdown 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- 10.20.193.102:0/2570969069 >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 legacy=0x7fd6ac183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- 10.20.193.102:0/2570969069 shutdown_connections 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- 10.20.193.102:0/2570969069 >> 10.20.193.102:0/2570969069 conn(0x7fd6ac05e3f0 msgr2=0x7fd6ac095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- 10.20.193.102:0/2570969069 shutdown_connections 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 1 -- 10.20.193.102:0/2570969069 wait complete. 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: build_initial_monmap 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.037+0000 7fd6ab577640 10 monclient: monmap: 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:16:01.049 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 Processor -- start 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- start start 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: init 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _reopen_session rank -1 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-b con 0x7fd6ac13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-c con 0x7fd6ac13fd90 addr v1:10.20.193.102:6790/0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient(hunting): picked mon.noname-a con 0x7fd6ac1401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac133520 con 0x7fd6ac1401f0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac0590d0 con 0x7fd6ac13f9c0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ac1311c0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient(hunting): _renew_subs 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient(hunting): authenticate will time out at 1170.280375s 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6aad76640 1 -- >> v1:10.20.193.102:6789/0 conn(0x7fd6ac1401f0 legacy=0x7fd6ac1956e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6789/0 2026-02-21T22:16:01.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6a9d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 0x7fd6ac193fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60298/0 (socket says 10.20.193.102:60298) 2026-02-21T22:16:01.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6a9d74640 1 -- 10.20.193.102:0/2580368424 learned_addr learned my addr 10.20.193.102:0/2580368424 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6a9d74640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 legacy=0x7fd6ac193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6aa575640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac13f9c0 legacy=0x7fd6ac13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3961804121 0 0) 0x7fd6ac1311c0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): my global_id is 4415 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd678003620 con 0x7fd6ac13fd90 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2220253351 0 0) 0x7fd678003620 con 0x7fd6ac13fd90 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6ac1311c0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd6940048e0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): dump: 2026-02-21T22:16:01.054 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829983191 0 0) 0x7fd6ac0590d0 con 0x7fd6ac13f9c0 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient(hunting): my global_id is 4393 2026-02-21T22:16:01.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd678003620 con 0x7fd6ac13f9c0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2357008446 0 0) 0x7fd6ac1311c0 con 0x7fd6ac13fd90 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac13f9c0 legacy=0x7fd6ac13eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.102:6789/0 conn(0x7fd6ac1401f0 legacy=0x7fd6ac1956e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 monclient: found mon.c 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac132c80 con 0x7fd6ac13fd90 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: _finish_auth 0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 5 monclient: authenticate success, global_id 4415 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _renew_subs 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac132420 con 0x7fd6ac13fd90 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _renew_subs 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:16:01.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd6ac13f200 con 0x7fd6ac13fd90 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd694004b80 con 0x7fd6ac13fd90 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd694005170 con 0x7fd6ac13fd90 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd692ffd640 10 monclient: dump: 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6a8d72640 4 set_mon_vals no callback set 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.041+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd6ac0008e0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd694011800 con 0x7fd6ac13fd90 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(20..20 src has 1..20) ==== 3886+0+0 (unknown 4169619608 0 0) 0x7fd69404c710 con 0x7fd6ac13fd90 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd6aad76640 10 monclient: get_auth_request con 0x7fd67803d4d0 auth_method 0 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd6aa575640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac152c60 legacy=0x7fd6ac08d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2135275450 0 0) 0x7fd6ac0008e0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient(hunting): my global_id is 4396 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6ac1311c0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1497449532 0 0) 0x7fd6ac1311c0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6ac0008e0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd6a00032a0 con 0x7fd6ac152c60 2026-02-21T22:16:01.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient: dump: 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2101978223 0 0) 0x7fd6ac0008e0 con 0x7fd6ac152c60 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6ac153030 con 0x7fd6ac152c60 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fd6a00058c0 con 0x7fd6ac152c60 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:16:01.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:16:01.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac152c60 legacy=0x7fd6ac08d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.045+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac152c60 legacy=0x7fd6ac08d3b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.049+0000 7fd6ab577640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.049+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd6ac1a6da0 con 0x7fd6ac04d280 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6aa575640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac04d280 legacy=0x7fd6ac197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865526241 0 0) 0x7fd6ac1a6da0 con 0x7fd6ac04d280 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:16:01.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient(hunting): my global_id is 4399 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6ac0008e0 con 0x7fd6ac04d280 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1712609574 0 0) 0x7fd6ac0008e0 con 0x7fd6ac04d280 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6ac1a6da0 con 0x7fd6ac04d280 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd6a0003ab0 con 0x7fd6ac04d280 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient: dump: 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:16:01.062 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2239536886 0 0) 0x7fd6ac1a6da0 con 0x7fd6ac04d280 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd6ac058320 con 0x7fd6ac04d280 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4671 (unknown 0 0 1822676838) 0x7fd6a0005b40 con 0x7fd6ac04d280 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac04d280 legacy=0x7fd6ac197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd692ffd640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6789/0 conn(0x7fd6ac04d280 legacy=0x7fd6ac197e10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:16:01.063 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 10, 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 43659, 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:01.064 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:01.065 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:16:01.066 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:16:01.067 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.068 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.069 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.070 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:01.071 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:16:01.072 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:16:01.073 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd67803d4d0 legacy=0x7fd67805d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 10 monclient: shutdown 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 >> v1:10.20.193.102:6790/0 conn(0x7fd6ac13fd90 legacy=0x7fd6ac193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6aad76640 1 -- 10.20.193.102:0/2580368424 reap_dead start 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6aad76640 1 -- 10.20.193.102:0/2580368424 reap_dead start 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 shutdown_connections 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 >> 10.20.193.102:0/2580368424 conn(0x7fd6ac05e3f0 msgr2=0x7fd6ac08aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 shutdown_connections 2026-02-21T22:16:01.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.053+0000 7fd6ab577640 1 -- 10.20.193.102:0/2580368424 wait complete. 2026-02-21T22:16:01.075 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:16:01.075 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:16:01.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0f8affd640 1 -- 10.20.193.102:0/1350553507 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]=0 enabled application 'rados' on pool 'unique_pool_0' v21) ==== 197+0+0 (unknown 555284988 0 0) 0x7f0fa001dbf0 con 0x7f0fa41834f0 2026-02-21T22:16:01.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0f8affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2026-02-21T22:16:01.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0f8affd640 10 monclient: _finish_command 2 = system:0 enabled application 'rados' on pool 'unique_pool_0' 2026-02-21T22:16:01.786 INFO:teuthology.orchestra.run.trial102.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0f7403d390 legacy=0x7f0f7405d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 10 monclient: shutdown 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 >> v1:10.20.193.178:6789/0 conn(0x7f0fa41834f0 legacy=0x7f0fa4123930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 shutdown_connections 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 >> 10.20.193.102:0/1350553507 conn(0x7f0fa40917a0 msgr2=0x7f0fa408eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 shutdown_connections 2026-02-21T22:16:01.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:16:01.780+0000 7f0fac469640 1 -- 10.20.193.102:0/1350553507 wait complete. 2026-02-21T22:16:02.802 DEBUG:teuthology.orchestra.run.trial178:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0 2026-02-21T22:16:02.829 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight read -> 100 2026-02-21T22:16:02.829 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight write -> 50 2026-02-21T22:16:02.829 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight delete -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight snap_create -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight snap_remove -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight rollback -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight copy_from -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:adding op weight write_excl -> 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:ceph version 20.3.0-5425-g7c8cf961 (7c8cf961761b37084e637a305d42f77c52437802) tentacle (dev - None) 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout:Configuration: 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout: Number of operations: 4000 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout: Number of objects: 50 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout: Max in flight operations: 16 2026-02-21T22:16:02.830 INFO:tasks.rados.rados.0.trial178.stdout: Object size (in bytes): 4000000 2026-02-21T22:16:02.831 INFO:tasks.rados.rados.0.trial178.stdout: Write stride min: 400000 2026-02-21T22:16:02.831 INFO:tasks.rados.rados.0.trial178.stdout: Write stride max: 800000 2026-02-21T22:16:03.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:03.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:03.793 INFO:tasks.rados.rados.0.trial178.stdout:1: write initial oid 1 2026-02-21T22:16:03.793 INFO:tasks.rados.rados.0.trial178.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2026-02-21T22:16:03.798 INFO:tasks.rados.rados.0.trial178.stdout:1: writing trial17818531-1 from 416807 to 892056 tid 1 2026-02-21T22:16:03.803 INFO:tasks.rados.rados.0.trial178.stdout:1: writing trial17818531-1 from 1542129 to 2016807 tid 2 2026-02-21T22:16:03.803 INFO:tasks.rados.rados.0.trial178.stdout:2: write initial oid 2 2026-02-21T22:16:03.803 INFO:tasks.rados.rados.0.trial178.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2026-02-21T22:16:03.806 INFO:tasks.rados.rados.0.trial178.stdout:2: writing trial17818531-2 from 416807 to 892056 tid 1 2026-02-21T22:16:03.811 INFO:tasks.rados.rados.0.trial178.stdout:2: writing trial17818531-2 from 1542129 to 2016807 tid 2 2026-02-21T22:16:03.811 INFO:tasks.rados.rados.0.trial178.stdout:3: write initial oid 3 2026-02-21T22:16:03.811 INFO:tasks.rados.rados.0.trial178.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2026-02-21T22:16:03.814 INFO:tasks.rados.rados.0.trial178.stdout:3: writing trial17818531-3 from 433614 to 984112 tid 1 2026-02-21T22:16:03.818 INFO:tasks.rados.rados.0.trial178.stdout:3: writing trial17818531-3 from 1600611 to 2033614 tid 2 2026-02-21T22:16:03.818 INFO:tasks.rados.rados.0.trial178.stdout:4: write initial oid 4 2026-02-21T22:16:03.818 INFO:tasks.rados.rados.0.trial178.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2026-02-21T22:16:03.822 INFO:tasks.rados.rados.0.trial178.stdout:4: writing trial17818531-4 from 450421 to 1076168 tid 1 2026-02-21T22:16:03.826 INFO:tasks.rados.rados.0.trial178.stdout:4: writing trial17818531-4 from 1659093 to 2050421 tid 2 2026-02-21T22:16:03.826 INFO:tasks.rados.rados.0.trial178.stdout:5: write initial oid 5 2026-02-21T22:16:03.826 INFO:tasks.rados.rados.0.trial178.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2026-02-21T22:16:03.830 INFO:tasks.rados.rados.0.trial178.stdout:5: writing trial17818531-5 from 467228 to 1168224 tid 1 2026-02-21T22:16:03.833 INFO:tasks.rados.rados.0.trial178.stdout:5: writing trial17818531-5 from 1717575 to 2067228 tid 2 2026-02-21T22:16:03.833 INFO:tasks.rados.rados.0.trial178.stdout:6: write initial oid 6 2026-02-21T22:16:03.834 INFO:tasks.rados.rados.0.trial178.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2026-02-21T22:16:03.838 INFO:tasks.rados.rados.0.trial178.stdout:6: writing trial17818531-6 from 484035 to 1260280 tid 1 2026-02-21T22:16:03.841 INFO:tasks.rados.rados.0.trial178.stdout:6: writing trial17818531-6 from 2059704 to 2084035 tid 2 2026-02-21T22:16:03.841 INFO:tasks.rados.rados.0.trial178.stdout:7: write initial oid 7 2026-02-21T22:16:03.841 INFO:tasks.rados.rados.0.trial178.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2026-02-21T22:16:03.844 INFO:tasks.rados.rados.0.trial178.stdout:7: writing trial17818531-7 from 500842 to 952336 tid 1 2026-02-21T22:16:03.849 INFO:tasks.rados.rados.0.trial178.stdout:7: writing trial17818531-7 from 1718186 to 2100842 tid 2 2026-02-21T22:16:03.849 INFO:tasks.rados.rados.0.trial178.stdout:8: write initial oid 8 2026-02-21T22:16:03.849 INFO:tasks.rados.rados.0.trial178.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2026-02-21T22:16:03.853 INFO:tasks.rados.rados.0.trial178.stdout:8: writing trial17818531-8 from 517649 to 1044392 tid 1 2026-02-21T22:16:03.857 INFO:tasks.rados.rados.0.trial178.stdout:8: writing trial17818531-8 from 1776668 to 2117649 tid 2 2026-02-21T22:16:03.857 INFO:tasks.rados.rados.0.trial178.stdout:9: write initial oid 9 2026-02-21T22:16:03.857 INFO:tasks.rados.rados.0.trial178.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2026-02-21T22:16:03.861 INFO:tasks.rados.rados.0.trial178.stdout:9: writing trial17818531-9 from 534456 to 1252801 tid 1 2026-02-21T22:16:03.865 INFO:tasks.rados.rados.0.trial178.stdout:9: writing trial17818531-9 from 1951503 to 2134456 tid 2 2026-02-21T22:16:03.865 INFO:tasks.rados.rados.0.trial178.stdout:10: write initial oid 10 2026-02-21T22:16:03.865 INFO:tasks.rados.rados.0.trial178.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2026-02-21T22:16:03.869 INFO:tasks.rados.rados.0.trial178.stdout:10: writing trial17818531-10 from 551263 to 1344857 tid 1 2026-02-21T22:16:03.872 INFO:tasks.rados.rados.0.trial178.stdout:10: writing trial17818531-10 from 1893632 to 2151263 tid 2 2026-02-21T22:16:03.873 INFO:tasks.rados.rados.0.trial178.stdout:11: write initial oid 11 2026-02-21T22:16:03.873 INFO:tasks.rados.rados.0.trial178.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2026-02-21T22:16:03.876 INFO:tasks.rados.rados.0.trial178.stdout:11: writing trial17818531-11 from 568070 to 1036913 tid 1 2026-02-21T22:16:03.881 INFO:tasks.rados.rados.0.trial178.stdout:11: writing trial17818531-11 from 1552114 to 2168070 tid 2 2026-02-21T22:16:03.881 INFO:tasks.rados.rados.0.trial178.stdout:12: write initial oid 12 2026-02-21T22:16:03.881 INFO:tasks.rados.rados.0.trial178.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2026-02-21T22:16:03.885 INFO:tasks.rados.rados.0.trial178.stdout:12: writing trial17818531-12 from 584877 to 1128969 tid 1 2026-02-21T22:16:03.889 INFO:tasks.rados.rados.0.trial178.stdout:12: writing trial17818531-12 from 1610596 to 2172599 tid 2 2026-02-21T22:16:03.889 INFO:tasks.rados.rados.0.trial178.stdout:13: write initial oid 13 2026-02-21T22:16:03.889 INFO:tasks.rados.rados.0.trial178.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2026-02-21T22:16:03.893 INFO:tasks.rados.rados.0.trial178.stdout:13: writing trial17818531-13 from 601684 to 1221025 tid 1 2026-02-21T22:16:03.897 INFO:tasks.rados.rados.0.trial178.stdout:13: writing trial17818531-13 from 1669078 to 2201684 tid 2 2026-02-21T22:16:03.897 INFO:tasks.rados.rados.0.trial178.stdout:14: write initial oid 14 2026-02-21T22:16:03.897 INFO:tasks.rados.rados.0.trial178.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2026-02-21T22:16:03.902 INFO:tasks.rados.rados.0.trial178.stdout:14: writing trial17818531-14 from 618491 to 1313081 tid 1 2026-02-21T22:16:03.905 INFO:tasks.rados.rados.0.trial178.stdout:14: writing trial17818531-14 from 2011207 to 2218491 tid 2 2026-02-21T22:16:03.906 INFO:tasks.rados.rados.0.trial178.stdout:15: write initial oid 15 2026-02-21T22:16:03.906 INFO:tasks.rados.rados.0.trial178.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2026-02-21T22:16:03.910 INFO:tasks.rados.rados.0.trial178.stdout:15: writing trial17818531-15 from 635298 to 1405137 tid 1 2026-02-21T22:16:03.913 INFO:tasks.rados.rados.0.trial178.stdout:15: writing trial17818531-15 from 2069689 to 2235298 tid 2 2026-02-21T22:16:03.914 INFO:tasks.rados.rados.0.trial178.stdout:16: write initial oid 16 2026-02-21T22:16:03.914 INFO:tasks.rados.rados.0.trial178.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2026-02-21T22:16:03.917 INFO:tasks.rados.rados.0.trial178.stdout:16: writing trial17818531-16 from 652105 to 1097193 tid 1 2026-02-21T22:16:03.922 INFO:tasks.rados.rados.0.trial178.stdout:16: writing trial17818531-16 from 1728171 to 2181159 tid 2 2026-02-21T22:16:03.922 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.922 INFO:tasks.rados.rados.0.trial178.stdout:1: finishing write tid 1 to trial17818531-1 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: finishing write tid 2 to trial17818531-1 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: finishing write tid 3 to trial17818531-1 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: finishing write tid 4 to trial17818531-1 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: oid 1 updating version 0 to 2 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: oid 1 version 2 is already newer than 1 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: oid 1 updating version 2 to 3 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:2: finishing write tid 1 to trial17818531-2 2026-02-21T22:16:03.923 INFO:tasks.rados.rados.0.trial178.stdout:2: finishing write tid 2 to trial17818531-2 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: finishing write tid 3 to trial17818531-2 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: finishing write tid 4 to trial17818531-2 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: oid 2 updating version 0 to 2 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: oid 2 updating version 2 to 3 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: oid 2 version 3 is already newer than 1 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:3: finishing write tid 1 to trial17818531-3 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:1: done (15 left) 2026-02-21T22:16:03.924 INFO:tasks.rados.rados.0.trial178.stdout:2: done (14 left) 2026-02-21T22:16:03.925 INFO:tasks.rados.rados.0.trial178.stdout:17: write initial oid 17 2026-02-21T22:16:03.925 INFO:tasks.rados.rados.0.trial178.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2026-02-21T22:16:03.926 INFO:tasks.rados.rados.0.trial178.stdout:17: writing trial17818531-17 from 668912 to 1305602 tid 1 2026-02-21T22:16:03.930 INFO:tasks.rados.rados.0.trial178.stdout:17: writing trial17818531-17 from 1903006 to 2268912 tid 2 2026-02-21T22:16:03.930 INFO:tasks.rados.rados.0.trial178.stdout:18: write initial oid 18 2026-02-21T22:16:03.930 INFO:tasks.rados.rados.0.trial178.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2026-02-21T22:16:03.935 INFO:tasks.rados.rados.0.trial178.stdout:18: writing trial17818531-18 from 685719 to 1397658 tid 1 2026-02-21T22:16:03.939 INFO:tasks.rados.rados.0.trial178.stdout:18: writing trial17818531-18 from 1845135 to 2285719 tid 2 2026-02-21T22:16:03.939 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.939 INFO:tasks.rados.rados.0.trial178.stdout:3: finishing write tid 2 to trial17818531-3 2026-02-21T22:16:03.939 INFO:tasks.rados.rados.0.trial178.stdout:3: finishing write tid 3 to trial17818531-3 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:3: finishing write tid 4 to trial17818531-3 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:3: oid 3 updating version 0 to 2 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:3: oid 3 updating version 2 to 3 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:3: oid 3 version 3 is already newer than 1 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:4: finishing write tid 1 to trial17818531-4 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:4: finishing write tid 2 to trial17818531-4 2026-02-21T22:16:03.940 INFO:tasks.rados.rados.0.trial178.stdout:4: finishing write tid 3 to trial17818531-4 2026-02-21T22:16:03.941 INFO:tasks.rados.rados.0.trial178.stdout:3: done (15 left) 2026-02-21T22:16:03.941 INFO:tasks.rados.rados.0.trial178.stdout:19: write initial oid 19 2026-02-21T22:16:03.941 INFO:tasks.rados.rados.0.trial178.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2026-02-21T22:16:03.944 INFO:tasks.rados.rados.0.trial178.stdout:19: writing trial17818531-19 from 702526 to 1489714 tid 1 2026-02-21T22:16:03.947 INFO:tasks.rados.rados.0.trial178.stdout:19: writing trial17818531-19 from 1903617 to 2302526 tid 2 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:4: finishing write tid 4 to trial17818531-4 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:4: oid 4 updating version 0 to 1 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:4: oid 4 updating version 1 to 3 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:4: oid 4 version 3 is already newer than 2 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:5: finishing write tid 1 to trial17818531-5 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:5: finishing write tid 2 to trial17818531-5 2026-02-21T22:16:03.948 INFO:tasks.rados.rados.0.trial178.stdout:5: finishing write tid 3 to trial17818531-5 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:5: finishing write tid 4 to trial17818531-5 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:5: oid 5 updating version 0 to 1 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:5: oid 5 updating version 1 to 2 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:5: oid 5 updating version 2 to 3 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:6: finishing write tid 1 to trial17818531-6 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:6: finishing write tid 2 to trial17818531-6 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:6: finishing write tid 3 to trial17818531-6 2026-02-21T22:16:03.949 INFO:tasks.rados.rados.0.trial178.stdout:4: done (15 left) 2026-02-21T22:16:03.950 INFO:tasks.rados.rados.0.trial178.stdout:5: done (14 left) 2026-02-21T22:16:03.950 INFO:tasks.rados.rados.0.trial178.stdout:20: write initial oid 20 2026-02-21T22:16:03.950 INFO:tasks.rados.rados.0.trial178.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2026-02-21T22:16:03.951 INFO:tasks.rados.rados.0.trial178.stdout:20: writing trial17818531-20 from 719333 to 1181770 tid 1 2026-02-21T22:16:03.956 INFO:tasks.rados.rados.0.trial178.stdout:20: writing trial17818531-20 from 1962099 to 2319333 tid 2 2026-02-21T22:16:03.956 INFO:tasks.rados.rados.0.trial178.stdout:21: write initial oid 21 2026-02-21T22:16:03.956 INFO:tasks.rados.rados.0.trial178.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2026-02-21T22:16:03.960 INFO:tasks.rados.rados.0.trial178.stdout:21: writing trial17818531-21 from 736140 to 1273826 tid 1 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:21: writing trial17818531-21 from 2020581 to 2336140 tid 2 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:6: finishing write tid 4 to trial17818531-6 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:6: oid 6 updating version 0 to 5 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:6: oid 6 version 5 is already newer than 4 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:6: oid 6 updating version 5 to 6 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2026-02-21T22:16:03.965 INFO:tasks.rados.rados.0.trial178.stdout:7: finishing write tid 1 to trial17818531-7 2026-02-21T22:16:03.966 INFO:tasks.rados.rados.0.trial178.stdout:6: done (15 left) 2026-02-21T22:16:03.966 INFO:tasks.rados.rados.0.trial178.stdout:22: write initial oid 22 2026-02-21T22:16:03.966 INFO:tasks.rados.rados.0.trial178.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2026-02-21T22:16:03.969 INFO:tasks.rados.rados.0.trial178.stdout:22: writing trial17818531-22 from 752947 to 1365882 tid 1 2026-02-21T22:16:03.973 INFO:tasks.rados.rados.0.trial178.stdout:22: writing trial17818531-22 from 1962710 to 2352947 tid 2 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: finishing write tid 2 to trial17818531-7 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: finishing write tid 3 to trial17818531-7 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: finishing write tid 4 to trial17818531-7 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: oid 7 updating version 0 to 2 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: oid 7 version 2 is already newer than 1 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: oid 7 updating version 2 to 3 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:03.974 INFO:tasks.rados.rados.0.trial178.stdout:8: finishing write tid 1 to trial17818531-8 2026-02-21T22:16:03.975 INFO:tasks.rados.rados.0.trial178.stdout:7: done (15 left) 2026-02-21T22:16:03.975 INFO:tasks.rados.rados.0.trial178.stdout:23: write initial oid 23 2026-02-21T22:16:03.975 INFO:tasks.rados.rados.0.trial178.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2026-02-21T22:16:03.978 INFO:tasks.rados.rados.0.trial178.stdout:23: writing trial17818531-23 from 769754 to 1457938 tid 1 2026-02-21T22:16:03.982 INFO:tasks.rados.rados.0.trial178.stdout:23: writing trial17818531-23 from 2021192 to 2369754 tid 2 2026-02-21T22:16:03.982 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.982 INFO:tasks.rados.rados.0.trial178.stdout:8: finishing write tid 2 to trial17818531-8 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: finishing write tid 3 to trial17818531-8 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: finishing write tid 4 to trial17818531-8 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: oid 8 updating version 0 to 5 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: oid 8 version 5 is already newer than 4 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: oid 8 updating version 5 to 6 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 6 (ObjNum 7 snap 0 seq_num 7) dirty exists 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:9: finishing write tid 1 to trial17818531-9 2026-02-21T22:16:03.983 INFO:tasks.rados.rados.0.trial178.stdout:9: finishing write tid 2 to trial17818531-9 2026-02-21T22:16:03.984 INFO:tasks.rados.rados.0.trial178.stdout:8: done (15 left) 2026-02-21T22:16:03.984 INFO:tasks.rados.rados.0.trial178.stdout:24: write initial oid 24 2026-02-21T22:16:03.984 INFO:tasks.rados.rados.0.trial178.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2026-02-21T22:16:03.986 INFO:tasks.rados.rados.0.trial178.stdout:24: writing trial17818531-24 from 786561 to 1266347 tid 1 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:24: writing trial17818531-24 from 1796027 to 2263691 tid 2 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:9: finishing write tid 3 to trial17818531-9 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:9: finishing write tid 4 to trial17818531-9 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:9: oid 9 updating version 0 to 3 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:9: oid 9 version 3 is already newer than 1 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:9: oid 9 version 3 is already newer than 2 2026-02-21T22:16:03.991 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 3 (ObjNum 8 snap 0 seq_num 8) dirty exists 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: finishing write tid 1 to trial17818531-10 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: finishing write tid 2 to trial17818531-10 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: finishing write tid 3 to trial17818531-10 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: finishing write tid 4 to trial17818531-10 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: oid 10 updating version 0 to 8 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: oid 10 updating version 8 to 9 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: oid 10 version 9 is already newer than 7 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2026-02-21T22:16:03.992 INFO:tasks.rados.rados.0.trial178.stdout:11: finishing write tid 1 to trial17818531-11 2026-02-21T22:16:03.993 INFO:tasks.rados.rados.0.trial178.stdout:9: done (15 left) 2026-02-21T22:16:03.993 INFO:tasks.rados.rados.0.trial178.stdout:10: done (14 left) 2026-02-21T22:16:03.993 INFO:tasks.rados.rados.0.trial178.stdout:25: write initial oid 25 2026-02-21T22:16:03.993 INFO:tasks.rados.rados.0.trial178.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2026-02-21T22:16:03.994 INFO:tasks.rados.rados.0.trial178.stdout:25: writing trial17818531-25 from 403368 to 958403 tid 1 2026-02-21T22:16:03.999 INFO:tasks.rados.rados.0.trial178.stdout:25: writing trial17818531-25 from 1454509 to 2182184 tid 2 2026-02-21T22:16:03.999 INFO:tasks.rados.rados.0.trial178.stdout:26: write initial oid 26 2026-02-21T22:16:03.999 INFO:tasks.rados.rados.0.trial178.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2026-02-21T22:16:04.002 INFO:tasks.rados.rados.0.trial178.stdout:26: writing trial17818531-26 from 420175 to 1050459 tid 1 2026-02-21T22:16:04.007 INFO:tasks.rados.rados.0.trial178.stdout:26: writing trial17818531-26 from 1796638 to 2267971 tid 2 2026-02-21T22:16:04.007 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: finishing write tid 2 to trial17818531-11 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: finishing write tid 3 to trial17818531-11 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: finishing write tid 4 to trial17818531-11 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: oid 11 updating version 0 to 5 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: oid 11 version 5 is already newer than 4 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: oid 11 updating version 5 to 6 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:12: finishing write tid 1 to trial17818531-12 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:12: finishing write tid 2 to trial17818531-12 2026-02-21T22:16:04.008 INFO:tasks.rados.rados.0.trial178.stdout:12: finishing write tid 3 to trial17818531-12 2026-02-21T22:16:04.009 INFO:tasks.rados.rados.0.trial178.stdout:11: done (15 left) 2026-02-21T22:16:04.009 INFO:tasks.rados.rados.0.trial178.stdout:27: write initial oid 27 2026-02-21T22:16:04.009 INFO:tasks.rados.rados.0.trial178.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2026-02-21T22:16:04.011 INFO:tasks.rados.rados.0.trial178.stdout:27: writing trial17818531-27 from 436982 to 1142515 tid 1 2026-02-21T22:16:04.016 INFO:tasks.rados.rados.0.trial178.stdout:27: writing trial17818531-27 from 1855120 to 2436982 tid 2 2026-02-21T22:16:04.016 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: finishing write tid 4 to trial17818531-12 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: oid 12 updating version 0 to 5 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: oid 12 updating version 5 to 6 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: oid 12 version 6 is already newer than 4 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:13: finishing write tid 1 to trial17818531-13 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:13: finishing write tid 2 to trial17818531-13 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:12: done (15 left) 2026-02-21T22:16:04.017 INFO:tasks.rados.rados.0.trial178.stdout:28: write initial oid 28 2026-02-21T22:16:04.018 INFO:tasks.rados.rados.0.trial178.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2026-02-21T22:16:04.020 INFO:tasks.rados.rados.0.trial178.stdout:28: writing trial17818531-28 from 453789 to 1234571 tid 1 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:28: writing trial17818531-28 from 1913602 to 2388604 tid 2 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: finishing write tid 1 to trial17818531-14 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: finishing write tid 2 to trial17818531-14 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: finishing write tid 3 to trial17818531-14 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: finishing write tid 4 to trial17818531-14 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: oid 14 updating version 0 to 5 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: oid 14 updating version 5 to 6 2026-02-21T22:16:04.026 INFO:tasks.rados.rados.0.trial178.stdout:14: oid 14 version 6 is already newer than 4 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: finishing write tid 3 to trial17818531-13 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: finishing write tid 4 to trial17818531-13 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: oid 13 updating version 0 to 8 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: oid 13 updating version 8 to 9 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: oid 13 version 9 is already newer than 7 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2026-02-21T22:16:04.027 INFO:tasks.rados.rados.0.trial178.stdout:15: finishing write tid 1 to trial17818531-15 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: finishing write tid 2 to trial17818531-15 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: finishing write tid 3 to trial17818531-15 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: finishing write tid 4 to trial17818531-15 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: oid 15 updating version 0 to 3 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: oid 15 version 3 is already newer than 1 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: oid 15 version 3 is already newer than 2 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:16: finishing write tid 1 to trial17818531-16 2026-02-21T22:16:04.028 INFO:tasks.rados.rados.0.trial178.stdout:13: done (15 left) 2026-02-21T22:16:04.029 INFO:tasks.rados.rados.0.trial178.stdout:14: done (14 left) 2026-02-21T22:16:04.029 INFO:tasks.rados.rados.0.trial178.stdout:15: done (13 left) 2026-02-21T22:16:04.029 INFO:tasks.rados.rados.0.trial178.stdout:29: write initial oid 29 2026-02-21T22:16:04.029 INFO:tasks.rados.rados.0.trial178.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2026-02-21T22:16:04.029 INFO:tasks.rados.rados.0.trial178.stdout:29: writing trial17818531-29 from 470596 to 926627 tid 1 2026-02-21T22:16:04.033 INFO:tasks.rados.rados.0.trial178.stdout:29: writing trial17818531-29 from 1572084 to 2190744 tid 2 2026-02-21T22:16:04.033 INFO:tasks.rados.rados.0.trial178.stdout:30: write initial oid 30 2026-02-21T22:16:04.033 INFO:tasks.rados.rados.0.trial178.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2026-02-21T22:16:04.037 INFO:tasks.rados.rados.0.trial178.stdout:30: writing trial17818531-30 from 487403 to 1018683 tid 1 2026-02-21T22:16:04.041 INFO:tasks.rados.rados.0.trial178.stdout:30: writing trial17818531-30 from 1514213 to 1992884 tid 2 2026-02-21T22:16:04.041 INFO:tasks.rados.rados.0.trial178.stdout:31: write initial oid 31 2026-02-21T22:16:04.041 INFO:tasks.rados.rados.0.trial178.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2026-02-21T22:16:04.045 INFO:tasks.rados.rados.0.trial178.stdout:31: writing trial17818531-31 from 504210 to 1110739 tid 1 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:31: writing trial17818531-31 from 1572695 to 2195024 tid 2 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: finishing write tid 2 to trial17818531-16 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: finishing write tid 3 to trial17818531-16 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: finishing write tid 4 to trial17818531-16 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: oid 16 updating version 0 to 9 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: oid 16 version 9 is already newer than 8 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:16: oid 16 version 9 is already newer than 7 2026-02-21T22:16:04.050 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 9 (ObjNum 15 snap 0 seq_num 15) dirty exists 2026-02-21T22:16:04.051 INFO:tasks.rados.rados.0.trial178.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:04.051 INFO:tasks.rados.rados.0.trial178.stdout:17: finishing write tid 1 to trial17818531-17 2026-02-21T22:16:04.051 INFO:tasks.rados.rados.0.trial178.stdout:16: done (15 left) 2026-02-21T22:16:04.051 INFO:tasks.rados.rados.0.trial178.stdout:32: write initial oid 32 2026-02-21T22:16:04.051 INFO:tasks.rados.rados.0.trial178.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2026-02-21T22:16:04.054 INFO:tasks.rados.rados.0.trial178.stdout:32: writing trial17818531-32 from 521017 to 1319148 tid 1 2026-02-21T22:16:04.058 INFO:tasks.rados.rados.0.trial178.stdout:32: writing trial17818531-32 from 1747530 to 2229870 tid 2 2026-02-21T22:16:04.058 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.058 INFO:tasks.rados.rados.0.trial178.stdout:17: finishing write tid 2 to trial17818531-17 2026-02-21T22:16:04.058 INFO:tasks.rados.rados.0.trial178.stdout:17: finishing write tid 3 to trial17818531-17 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: finishing write tid 4 to trial17818531-17 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: oid 17 updating version 0 to 6 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: oid 17 version 6 is already newer than 5 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: oid 17 version 6 is already newer than 4 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:18: finishing write tid 1 to trial17818531-18 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:18: finishing write tid 2 to trial17818531-18 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:18: finishing write tid 3 to trial17818531-18 2026-02-21T22:16:04.059 INFO:tasks.rados.rados.0.trial178.stdout:17: done (15 left) 2026-02-21T22:16:04.060 INFO:tasks.rados.rados.0.trial178.stdout:33: write initial oid 33 2026-02-21T22:16:04.060 INFO:tasks.rados.rados.0.trial178.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2026-02-21T22:16:04.062 INFO:tasks.rados.rados.0.trial178.stdout:33: writing trial17818531-33 from 537824 to 1011204 tid 1 2026-02-21T22:16:04.067 INFO:tasks.rados.rados.0.trial178.stdout:33: writing trial17818531-33 from 1806012 to 2432010 tid 2 2026-02-21T22:16:04.067 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: finishing write tid 4 to trial17818531-18 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: oid 18 updating version 0 to 4 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: oid 18 updating version 4 to 5 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: oid 18 updating version 5 to 6 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 6 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:19: finishing write tid 1 to trial17818531-19 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:18: done (15 left) 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:34: write initial oid 34 2026-02-21T22:16:04.068 INFO:tasks.rados.rados.0.trial178.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2026-02-21T22:16:04.071 INFO:tasks.rados.rados.0.trial178.stdout:34: writing trial17818531-34 from 554631 to 1103260 tid 1 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:34: writing trial17818531-34 from 1748141 to 2234150 tid 2 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: finishing write tid 2 to trial17818531-19 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: finishing write tid 3 to trial17818531-19 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: finishing write tid 4 to trial17818531-19 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: oid 19 updating version 0 to 9 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: oid 19 version 9 is already newer than 8 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: oid 19 version 9 is already newer than 7 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:20: finishing write tid 1 to trial17818531-20 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:20: finishing write tid 2 to trial17818531-20 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:20: finishing write tid 3 to trial17818531-20 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:20: finishing write tid 4 to trial17818531-20 2026-02-21T22:16:04.076 INFO:tasks.rados.rados.0.trial178.stdout:20: oid 20 updating version 0 to 8 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:20: oid 20 updating version 8 to 9 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:20: oid 20 version 9 is already newer than 7 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 9 (ObjNum 19 snap 0 seq_num 19) dirty exists 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:21: finishing write tid 1 to trial17818531-21 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:19: done (15 left) 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:20: done (14 left) 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:35: write initial oid 35 2026-02-21T22:16:04.077 INFO:tasks.rados.rados.0.trial178.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2026-02-21T22:16:04.080 INFO:tasks.rados.rados.0.trial178.stdout:35: writing trial17818531-35 from 571438 to 1195316 tid 1 2026-02-21T22:16:04.085 INFO:tasks.rados.rados.0.trial178.stdout:35: writing trial17818531-35 from 1806623 to 2436290 tid 2 2026-02-21T22:16:04.085 INFO:tasks.rados.rados.0.trial178.stdout:36: write initial oid 36 2026-02-21T22:16:04.085 INFO:tasks.rados.rados.0.trial178.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2026-02-21T22:16:04.090 INFO:tasks.rados.rados.0.trial178.stdout:36: writing trial17818531-36 from 588245 to 1287372 tid 1 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout:36: writing trial17818531-36 from 1865105 to 2354783 tid 2 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout:21: finishing write tid 2 to trial17818531-21 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout:21: finishing write tid 3 to trial17818531-21 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout:21: finishing write tid 4 to trial17818531-21 2026-02-21T22:16:04.094 INFO:tasks.rados.rados.0.trial178.stdout:21: oid 21 updating version 0 to 11 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:21: oid 21 updating version 11 to 12 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:21: oid 21 version 12 is already newer than 10 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 12 (ObjNum 20 snap 0 seq_num 20) dirty exists 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:22: finishing write tid 1 to trial17818531-22 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:22: finishing write tid 2 to trial17818531-22 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:21: done (15 left) 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:37: write initial oid 37 2026-02-21T22:16:04.095 INFO:tasks.rados.rados.0.trial178.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2026-02-21T22:16:04.099 INFO:tasks.rados.rados.0.trial178.stdout:37: writing trial17818531-37 from 605052 to 1379428 tid 1 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:37: writing trial17818531-37 from 1923587 to 2556923 tid 2 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: finishing write tid 3 to trial17818531-22 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: finishing write tid 4 to trial17818531-22 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: oid 22 updating version 0 to 2 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: oid 22 version 2 is already newer than 1 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: oid 22 updating version 2 to 3 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 3 (ObjNum 21 snap 0 seq_num 21) dirty exists 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: finishing write tid 1 to trial17818531-23 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: finishing write tid 2 to trial17818531-23 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: finishing write tid 3 to trial17818531-23 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: finishing write tid 4 to trial17818531-23 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: oid 23 updating version 0 to 2 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: oid 23 version 2 is already newer than 1 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: oid 23 updating version 2 to 3 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2026-02-21T22:16:04.104 INFO:tasks.rados.rados.0.trial178.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: finishing write tid 1 to trial17818531-24 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: finishing write tid 2 to trial17818531-24 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: finishing write tid 3 to trial17818531-24 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: finishing write tid 4 to trial17818531-24 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: oid 24 updating version 0 to 14 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: oid 24 updating version 14 to 15 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: oid 24 version 15 is already newer than 13 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 15 (ObjNum 23 snap 0 seq_num 23) dirty exists 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: finishing write tid 1 to trial17818531-25 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: finishing write tid 2 to trial17818531-25 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: finishing write tid 3 to trial17818531-25 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: finishing write tid 4 to trial17818531-25 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: oid 25 updating version 0 to 2 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: oid 25 version 2 is already newer than 1 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: oid 25 updating version 2 to 3 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 3 (ObjNum 24 snap 0 seq_num 24) dirty exists 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:26: finishing write tid 1 to trial17818531-26 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:22: done (15 left) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:23: done (14 left) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:24: done (13 left) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:25: done (12 left) 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:38: write initial oid 38 2026-02-21T22:16:04.105 INFO:tasks.rados.rados.0.trial178.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2026-02-21T22:16:04.108 INFO:tasks.rados.rados.0.trial178.stdout:38: writing trial17818531-38 from 621859 to 1071484 tid 1 2026-02-21T22:16:04.114 INFO:tasks.rados.rados.0.trial178.stdout:38: writing trial17818531-38 from 1865716 to 2621859 tid 2 2026-02-21T22:16:04.114 INFO:tasks.rados.rados.0.trial178.stdout:39: write initial oid 39 2026-02-21T22:16:04.114 INFO:tasks.rados.rados.0.trial178.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2026-02-21T22:16:04.118 INFO:tasks.rados.rados.0.trial178.stdout:39: writing trial17818531-39 from 638666 to 1163540 tid 1 2026-02-21T22:16:04.124 INFO:tasks.rados.rados.0.trial178.stdout:39: writing trial17818531-39 from 1924198 to 2561203 tid 2 2026-02-21T22:16:04.124 INFO:tasks.rados.rados.0.trial178.stdout:40: write initial oid 40 2026-02-21T22:16:04.124 INFO:tasks.rados.rados.0.trial178.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2026-02-21T22:16:04.129 INFO:tasks.rados.rados.0.trial178.stdout:40: writing trial17818531-40 from 655473 to 1371949 tid 1 2026-02-21T22:16:04.133 INFO:tasks.rados.rados.0.trial178.stdout:40: writing trial17818531-40 from 2099033 to 2655473 tid 2 2026-02-21T22:16:04.133 INFO:tasks.rados.rados.0.trial178.stdout:41: write initial oid 41 2026-02-21T22:16:04.133 INFO:tasks.rados.rados.0.trial178.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2026-02-21T22:16:04.139 INFO:tasks.rados.rados.0.trial178.stdout:41: writing trial17818531-41 from 672280 to 1464005 tid 1 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:41: writing trial17818531-41 from 2157515 to 2672280 tid 2 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: finishing write tid 2 to trial17818531-26 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: finishing write tid 3 to trial17818531-26 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: finishing write tid 4 to trial17818531-26 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: oid 26 updating version 0 to 10 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: oid 26 updating version 10 to 12 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: oid 26 version 12 is already newer than 11 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 12 (ObjNum 25 snap 0 seq_num 25) dirty exists 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: finishing write tid 1 to trial17818531-27 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: finishing write tid 2 to trial17818531-27 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: finishing write tid 3 to trial17818531-27 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: finishing write tid 4 to trial17818531-27 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: oid 27 updating version 0 to 8 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: oid 27 version 8 is already newer than 7 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: oid 27 updating version 8 to 9 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 9 (ObjNum 26 snap 0 seq_num 26) dirty exists 2026-02-21T22:16:04.144 INFO:tasks.rados.rados.0.trial178.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2026-02-21T22:16:04.145 INFO:tasks.rados.rados.0.trial178.stdout:26: done (15 left) 2026-02-21T22:16:04.145 INFO:tasks.rados.rados.0.trial178.stdout:27: done (14 left) 2026-02-21T22:16:04.145 INFO:tasks.rados.rados.0.trial178.stdout:42: write initial oid 42 2026-02-21T22:16:04.145 INFO:tasks.rados.rados.0.trial178.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2026-02-21T22:16:04.148 INFO:tasks.rados.rados.0.trial178.stdout:42: writing trial17818531-42 from 689087 to 1156061 tid 1 2026-02-21T22:16:04.154 INFO:tasks.rados.rados.0.trial178.stdout:42: writing trial17818531-42 from 1699644 to 2483976 tid 2 2026-02-21T22:16:04.154 INFO:tasks.rados.rados.0.trial178.stdout:43: write initial oid 43 2026-02-21T22:16:04.154 INFO:tasks.rados.rados.0.trial178.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2026-02-21T22:16:04.158 INFO:tasks.rados.rados.0.trial178.stdout:43: writing trial17818531-43 from 705894 to 1248117 tid 1 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:43: writing trial17818531-43 from 1758126 to 2402469 tid 2 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: finishing write tid 1 to trial17818531-29 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: finishing write tid 2 to trial17818531-29 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: finishing write tid 3 to trial17818531-29 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: finishing write tid 4 to trial17818531-29 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: oid 29 updating version 0 to 3 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: oid 29 version 3 is already newer than 1 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: oid 29 version 3 is already newer than 2 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:28: finishing write tid 1 to trial17818531-28 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:28: finishing write tid 2 to trial17818531-28 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:28: finishing write tid 3 to trial17818531-28 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:29: done (15 left) 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:44: write initial oid 44 2026-02-21T22:16:04.163 INFO:tasks.rados.rados.0.trial178.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2026-02-21T22:16:04.168 INFO:tasks.rados.rados.0.trial178.stdout:44: writing trial17818531-44 from 722701 to 1340173 tid 1 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:44: writing trial17818531-44 from 1816608 to 2604609 tid 2 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:28: finishing write tid 4 to trial17818531-28 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:28: oid 28 updating version 0 to 2 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:28: oid 28 version 2 is already newer than 1 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:28: oid 28 updating version 2 to 3 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 3 (ObjNum 27 snap 0 seq_num 27) dirty exists 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: finishing write tid 1 to trial17818531-30 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: finishing write tid 2 to trial17818531-30 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: finishing write tid 3 to trial17818531-30 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: finishing write tid 4 to trial17818531-30 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: oid 30 updating version 0 to 8 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: oid 30 version 8 is already newer than 7 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: oid 30 updating version 8 to 9 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: finishing write tid 1 to trial17818531-31 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: finishing write tid 2 to trial17818531-31 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: finishing write tid 3 to trial17818531-31 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: finishing write tid 4 to trial17818531-31 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: oid 31 updating version 0 to 5 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: oid 31 updating version 5 to 6 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: oid 31 version 6 is already newer than 4 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: finishing write tid 1 to trial17818531-32 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: finishing write tid 2 to trial17818531-32 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: finishing write tid 3 to trial17818531-32 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: finishing write tid 4 to trial17818531-32 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: oid 32 updating version 0 to 11 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: oid 32 version 11 is already newer than 10 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: oid 32 updating version 11 to 12 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 12 (ObjNum 31 snap 0 seq_num 31) dirty exists 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:33: finishing write tid 1 to trial17818531-33 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:33: finishing write tid 2 to trial17818531-33 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:33: finishing write tid 3 to trial17818531-33 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:33: finishing write tid 4 to trial17818531-33 2026-02-21T22:16:04.173 INFO:tasks.rados.rados.0.trial178.stdout:33: oid 33 updating version 0 to 6 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:33: oid 33 version 6 is already newer than 4 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:33: oid 33 version 6 is already newer than 5 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:34: finishing write tid 1 to trial17818531-34 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:28: done (15 left) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:30: done (14 left) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:31: done (13 left) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:32: done (12 left) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:33: done (11 left) 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:45: write initial oid 45 2026-02-21T22:16:04.174 INFO:tasks.rados.rados.0.trial178.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2026-02-21T22:16:04.178 INFO:tasks.rados.rados.0.trial178.stdout:45: writing trial17818531-45 from 739508 to 1432229 tid 1 2026-02-21T22:16:04.182 INFO:tasks.rados.rados.0.trial178.stdout:45: writing trial17818531-45 from 1875090 to 2523102 tid 2 2026-02-21T22:16:04.182 INFO:tasks.rados.rados.0.trial178.stdout:46: write initial oid 46 2026-02-21T22:16:04.182 INFO:tasks.rados.rados.0.trial178.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2026-02-21T22:16:04.187 INFO:tasks.rados.rados.0.trial178.stdout:46: writing trial17818531-46 from 756315 to 1524285 tid 1 2026-02-21T22:16:04.192 INFO:tasks.rados.rados.0.trial178.stdout:46: writing trial17818531-46 from 1933572 to 2725242 tid 2 2026-02-21T22:16:04.192 INFO:tasks.rados.rados.0.trial178.stdout:47: write initial oid 47 2026-02-21T22:16:04.192 INFO:tasks.rados.rados.0.trial178.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2026-02-21T22:16:04.197 INFO:tasks.rados.rados.0.trial178.stdout:47: writing trial17818531-47 from 773122 to 1332694 tid 1 2026-02-21T22:16:04.202 INFO:tasks.rados.rados.0.trial178.stdout:47: writing trial17818531-47 from 1992054 to 2643735 tid 2 2026-02-21T22:16:04.202 INFO:tasks.rados.rados.0.trial178.stdout:48: write initial oid 48 2026-02-21T22:16:04.202 INFO:tasks.rados.rados.0.trial178.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2026-02-21T22:16:04.207 INFO:tasks.rados.rados.0.trial178.stdout:48: writing trial17818531-48 from 789929 to 1424750 tid 1 2026-02-21T22:16:04.212 INFO:tasks.rados.rados.0.trial178.stdout:48: writing trial17818531-48 from 2050536 to 2789929 tid 2 2026-02-21T22:16:04.212 INFO:tasks.rados.rados.0.trial178.stdout:49: write initial oid 49 2026-02-21T22:16:04.212 INFO:tasks.rados.rados.0.trial178.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2026-02-21T22:16:04.216 INFO:tasks.rados.rados.0.trial178.stdout:49: writing trial17818531-49 from 406736 to 1116806 tid 1 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:49: writing trial17818531-49 from 1709018 to 2364368 tid 2 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: finishing write tid 2 to trial17818531-34 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: finishing write tid 3 to trial17818531-34 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: finishing write tid 4 to trial17818531-34 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: oid 34 updating version 0 to 6 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: oid 34 version 6 is already newer than 4 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: oid 34 version 6 is already newer than 5 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: finishing write tid 1 to trial17818531-35 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: finishing write tid 2 to trial17818531-35 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: finishing write tid 3 to trial17818531-35 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: finishing write tid 4 to trial17818531-35 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: oid 35 updating version 0 to 15 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: oid 35 version 15 is already newer than 13 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: oid 35 version 15 is already newer than 14 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 15 (ObjNum 34 snap 0 seq_num 34) dirty exists 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:36: finishing write tid 1 to trial17818531-36 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:34: done (15 left) 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:35: done (14 left) 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:50: write initial oid 50 2026-02-21T22:16:04.221 INFO:tasks.rados.rados.0.trial178.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2026-02-21T22:16:04.225 INFO:tasks.rados.rados.0.trial178.stdout:50: writing trial17818531-50 from 423543 to 1208862 tid 1 2026-02-21T22:16:04.230 INFO:tasks.rados.rados.0.trial178.stdout:50: writing trial17818531-50 from 1767500 to 2566508 tid 2 2026-02-21T22:16:04.230 INFO:tasks.rados.rados.0.trial178.stdout:51: write oid 4 current snap is 0 2026-02-21T22:16:04.230 INFO:tasks.rados.rados.0.trial178.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2026-02-21T22:16:04.234 INFO:tasks.rados.rados.0.trial178.stdout:51: writing trial17818531-4 from 440350 to 900918 tid 1 2026-02-21T22:16:04.237 INFO:tasks.rados.rados.0.trial178.stdout:51: writing trial17818531-4 from 1309629 to 1852295 tid 2 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:51: writing trial17818531-4 from 2323973 to 2840350 tid 3 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: finishing write tid 2 to trial17818531-36 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: finishing write tid 3 to trial17818531-36 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: finishing write tid 4 to trial17818531-36 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: oid 36 updating version 0 to 2 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: oid 36 updating version 2 to 3 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: oid 36 version 3 is already newer than 1 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 3 (ObjNum 35 snap 0 seq_num 35) dirty exists 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: finishing write tid 1 to trial17818531-37 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: finishing write tid 2 to trial17818531-37 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:36: done (15 left) 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:52: read oid 28 snap -1 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:52: expect (ObjNum 27 snap 0 seq_num 27) 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout: waiting on 16 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: finishing write tid 3 to trial17818531-37 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: finishing write tid 4 to trial17818531-37 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: oid 37 updating version 0 to 5 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: oid 37 version 5 is already newer than 4 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: oid 37 updating version 5 to 6 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: finishing write tid 1 to trial17818531-38 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: finishing write tid 2 to trial17818531-38 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: finishing write tid 3 to trial17818531-38 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: finishing write tid 4 to trial17818531-38 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: oid 38 updating version 0 to 7 2026-02-21T22:16:04.242 INFO:tasks.rados.rados.0.trial178.stdout:38: oid 38 updating version 7 to 8 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:38: oid 38 updating version 8 to 9 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: finishing write tid 1 to trial17818531-39 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: finishing write tid 2 to trial17818531-39 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: finishing write tid 3 to trial17818531-39 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: finishing write tid 4 to trial17818531-39 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: oid 39 updating version 0 to 11 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: oid 39 updating version 11 to 12 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: oid 39 version 12 is already newer than 10 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: finishing write tid 1 to trial17818531-40 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: finishing write tid 2 to trial17818531-40 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: finishing write tid 3 to trial17818531-40 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: finishing write tid 4 to trial17818531-40 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: oid 40 updating version 0 to 6 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: oid 40 version 6 is already newer than 4 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: oid 40 version 6 is already newer than 5 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: finishing write tid 1 to trial17818531-41 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:37: done (15 left) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:38: done (14 left) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:39: done (13 left) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:40: done (12 left) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:53: read oid 23 snap -1 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:53: expect (ObjNum 22 snap 0 seq_num 22) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:54: read oid 31 snap -1 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:54: expect (ObjNum 30 snap 0 seq_num 30) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: finishing write tid 2 to trial17818531-41 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: finishing write tid 3 to trial17818531-41 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: finishing write tid 4 to trial17818531-41 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: oid 41 updating version 0 to 11 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: oid 41 version 11 is already newer than 10 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: oid 41 updating version 11 to 12 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 12 (ObjNum 40 snap 0 seq_num 40) dirty exists 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: finishing write tid 1 to trial17818531-42 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: finishing write tid 2 to trial17818531-42 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: finishing write tid 3 to trial17818531-42 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: finishing write tid 4 to trial17818531-42 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: oid 42 updating version 0 to 18 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: oid 42 version 18 is already newer than 16 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: oid 42 version 18 is already newer than 17 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 18 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-21T22:16:04.243 INFO:tasks.rados.rados.0.trial178.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: finishing write tid 1 to trial17818531-43 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: finishing write tid 2 to trial17818531-43 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:41: done (13 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:42: done (12 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:55: copy_from oid 5 from oid 29 current snap is 0 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: finishing write tid 3 to trial17818531-43 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: finishing write tid 4 to trial17818531-43 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: oid 43 updating version 0 to 2 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: oid 43 updating version 2 to 3 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: oid 43 version 3 is already newer than 1 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 3 (ObjNum 42 snap 0 seq_num 42) dirty exists 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: finishing write tid 1 to trial17818531-44 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: finishing write tid 2 to trial17818531-44 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: finishing write tid 3 to trial17818531-44 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: finishing write tid 4 to trial17818531-44 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: oid 44 updating version 0 to 10 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: oid 44 updating version 10 to 11 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: oid 44 updating version 11 to 12 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 12 (ObjNum 43 snap 0 seq_num 43) dirty exists 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: finishing write tid 1 to trial17818531-45 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: finishing write tid 2 to trial17818531-45 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: finishing write tid 3 to trial17818531-45 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: finishing write tid 4 to trial17818531-45 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: oid 45 updating version 0 to 9 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: oid 45 version 9 is already newer than 8 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: oid 45 version 9 is already newer than 7 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 9 (ObjNum 44 snap 0 seq_num 44) dirty exists 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: finishing write tid 1 to trial17818531-46 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: finishing write tid 2 to trial17818531-46 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: finishing write tid 3 to trial17818531-46 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: finishing write tid 4 to trial17818531-46 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: oid 46 updating version 0 to 4 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: oid 46 updating version 4 to 6 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: oid 46 version 6 is already newer than 5 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:47: finishing write tid 1 to trial17818531-47 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:47: finishing write tid 2 to trial17818531-47 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:43: done (12 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:44: done (11 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:45: done (10 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:46: done (9 left) 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:56: rollback oid 6 current snap is 0 2026-02-21T22:16:04.244 INFO:tasks.rados.rados.0.trial178.stdout:56: done (9 left) 2026-02-21T22:16:04.245 INFO:tasks.rados.rados.0.trial178.stdout:57: write oid 9 current snap is 0 2026-02-21T22:16:04.245 INFO:tasks.rados.rados.0.trial178.stdout:57: seq_num 51 ranges {457157=535817,1768111=402677} 2026-02-21T22:16:04.245 INFO:tasks.rados.rados.0.trial178.stdout:57: writing trial17818531-9 from 457157 to 992974 tid 1 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:57: writing trial17818531-9 from 1768111 to 2170788 tid 2 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:58: read oid 36 snap -1 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:58: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:59: snap_create 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: finishing write tid 3 to trial17818531-47 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: finishing write tid 4 to trial17818531-47 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: oid 47 updating version 0 to 20 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: oid 47 version 20 is already newer than 19 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: oid 47 updating version 20 to 21 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 21 (ObjNum 46 snap 0 seq_num 46) dirty exists 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: finishing write tid 1 to trial17818531-48 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: finishing write tid 2 to trial17818531-48 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: finishing write tid 3 to trial17818531-48 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: finishing write tid 4 to trial17818531-48 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: oid 48 updating version 0 to 12 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: oid 48 version 12 is already newer than 11 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: oid 48 version 12 is already newer than 10 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 12 (ObjNum 47 snap 0 seq_num 47) dirty exists 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: finishing write tid 1 to trial17818531-49 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: finishing write tid 2 to trial17818531-49 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: finishing write tid 3 to trial17818531-49 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: finishing write tid 4 to trial17818531-49 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: oid 49 updating version 0 to 8 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: oid 49 version 8 is already newer than 7 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: oid 49 updating version 8 to 9 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:50: finishing write tid 1 to trial17818531-50 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:50: finishing write tid 2 to trial17818531-50 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:50: finishing write tid 3 to trial17818531-50 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:50: finishing write tid 4 to trial17818531-50 2026-02-21T22:16:04.250 INFO:tasks.rados.rados.0.trial178.stdout:50: oid 50 updating version 0 to 9 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:50: oid 50 version 9 is already newer than 7 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:50: oid 50 version 9 is already newer than 8 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:51: finishing write tid 1 to trial17818531-4 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:51: finishing write tid 2 to trial17818531-4 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:52: expect (ObjNum 27 snap 0 seq_num 27) 2026-02-21T22:16:04.251 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:04.286 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 671328 2026-02-21T22:16:04.316 INFO:tasks.rados.rados.0.trial178.stdout:53: expect (ObjNum 22 snap 0 seq_num 22) 2026-02-21T22:16:04.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 566393 2026-02-21T22:16:04.352 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1644021 2026-02-21T22:16:04.360 INFO:tasks.rados.rados.0.trial178.stdout:51: finishing write tid 3 to trial17818531-4 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: finishing write tid 4 to trial17818531-4 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: finishing write tid 5 to trial17818531-4 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: oid 4 updating version 0 to 13 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: oid 4 updating version 13 to 15 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: oid 4 version 15 is already newer than 14 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: oid 4 updating version 15 to 16 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 16 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:51: left oid 4 (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:54: expect (ObjNum 30 snap 0 seq_num 30) 2026-02-21T22:16:04.361 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:04.379 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:04.400 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1562224 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: finishing write tid 1 to trial17818531-9 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: finishing write tid 2 to trial17818531-9 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: finishing write tid 3 to trial17818531-9 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: finishing write tid 4 to trial17818531-9 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: oid 9 updating version 0 to 14 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: oid 9 updating version 14 to 15 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: oid 9 version 15 is already newer than 13 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 15 (ObjNum 51 snap 0 seq_num 51) dirty exists 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:57: left oid 9 (ObjNum 51 snap 0 seq_num 51) 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:58: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:16:04.410 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:04.460 INFO:tasks.rados.rados.0.trial178.stdout:55: finishing copy_from racing read to trial17818531-5 2026-02-21T22:16:04.460 INFO:tasks.rados.rados.0.trial178.stdout:55: finishing copy_from to trial17818531-5 2026-02-21T22:16:04.460 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-21T22:16:04.799 INFO:tasks.rados.rados.0.trial178.stdout:47: done (11 left) 2026-02-21T22:16:04.799 INFO:tasks.rados.rados.0.trial178.stdout:48: done (10 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:49: done (9 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:50: done (8 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:51: done (7 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:52: done (6 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:53: done (5 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:54: done (4 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:55: done (3 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:57: done (2 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:58: done (1 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:59: done (0 left) 2026-02-21T22:16:04.800 INFO:tasks.rados.rados.0.trial178.stdout:60: delete oid 47 current snap is 1 2026-02-21T22:16:04.801 INFO:tasks.rados.rados.0.trial178.stdout:60: done (0 left) 2026-02-21T22:16:04.801 INFO:tasks.rados.rados.0.trial178.stdout:61: snap_remove snap 0 2026-02-21T22:16:06.670 INFO:tasks.rados.rados.0.trial178.stdout:61: done (0 left) 2026-02-21T22:16:06.670 INFO:tasks.rados.rados.0.trial178.stdout:62: write oid 26 current snap is 1 2026-02-21T22:16:06.670 INFO:tasks.rados.rados.0.trial178.stdout:62: seq_num 53 ranges {490771=686315,1885075=406346} 2026-02-21T22:16:06.674 INFO:tasks.rados.rados.0.trial178.stdout:62: writing trial17818531-26 from 490771 to 1177086 tid 1 2026-02-21T22:16:06.679 INFO:tasks.rados.rados.0.trial178.stdout:62: writing trial17818531-26 from 1885075 to 2291421 tid 2 2026-02-21T22:16:06.679 INFO:tasks.rados.rados.0.trial178.stdout:63: copy_from oid 45 from oid 33 current snap is 1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:64: read oid 47 snap -1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:64: expect deleted 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:65: read oid 35 snap -1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:65: expect (ObjNum 34 snap 0 seq_num 34) 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:66: read oid 50 snap -1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:66: expect (ObjNum 49 snap 0 seq_num 49) 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:67: rollback oid 46 current snap is 1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:67: done (5 left) 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:68: read oid 4 snap -1 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:68: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:06.680 INFO:tasks.rados.rados.0.trial178.stdout:69: rollback oid 25 current snap is 1 2026-02-21T22:16:06.681 INFO:tasks.rados.rados.0.trial178.stdout:69: done (6 left) 2026-02-21T22:16:06.681 INFO:tasks.rados.rados.0.trial178.stdout:70: snap_create 2026-02-21T22:16:06.681 INFO:tasks.rados.rados.0.trial178.stdout:62: finishing write tid 1 to trial17818531-26 2026-02-21T22:16:06.684 INFO:tasks.rados.rados.0.trial178.stdout:68: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:06.734 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:06.789 INFO:tasks.rados.rados.0.trial178.stdout:65: expect (ObjNum 34 snap 0 seq_num 34) 2026-02-21T22:16:06.789 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 568607 2026-02-21T22:16:06.833 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2246379 2026-02-21T22:16:06.835 INFO:tasks.rados.rados.0.trial178.stdout:62: finishing write tid 2 to trial17818531-26 2026-02-21T22:16:06.835 INFO:tasks.rados.rados.0.trial178.stdout:62: finishing write tid 3 to trial17818531-26 2026-02-21T22:16:06.835 INFO:tasks.rados.rados.0.trial178.stdout:62: finishing write tid 4 to trial17818531-26 2026-02-21T22:16:06.835 INFO:tasks.rados.rados.0.trial178.stdout:62: oid 26 updating version 0 to 13 2026-02-21T22:16:06.836 INFO:tasks.rados.rados.0.trial178.stdout:62: oid 26 updating version 13 to 15 2026-02-21T22:16:06.836 INFO:tasks.rados.rados.0.trial178.stdout:62: oid 26 version 15 is already newer than 14 2026-02-21T22:16:06.836 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 15 (ObjNum 53 snap 1 seq_num 53) dirty exists 2026-02-21T22:16:06.836 INFO:tasks.rados.rados.0.trial178.stdout:62: left oid 26 (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:06.836 INFO:tasks.rados.rados.0.trial178.stdout:66: expect (ObjNum 49 snap 0 seq_num 49) 2026-02-21T22:16:06.886 INFO:tasks.rados.rados.0.trial178.stdout:63: finishing copy_from racing read to trial17818531-45 2026-02-21T22:16:06.887 INFO:tasks.rados.rados.0.trial178.stdout:63: finishing copy_from to trial17818531-45 2026-02-21T22:16:06.887 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:62: done (6 left) 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:63: done (5 left) 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:64: done (4 left) 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:65: done (3 left) 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:66: done (2 left) 2026-02-21T22:16:07.553 INFO:tasks.rados.rados.0.trial178.stdout:68: done (1 left) 2026-02-21T22:16:07.554 INFO:tasks.rados.rados.0.trial178.stdout:70: done (0 left) 2026-02-21T22:16:07.554 INFO:tasks.rados.rados.0.trial178.stdout:71: write oid 27 current snap is 2 2026-02-21T22:16:07.554 INFO:tasks.rados.rados.0.trial178.stdout:71: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2026-02-21T22:16:07.557 INFO:tasks.rados.rados.0.trial178.stdout:71: writing trial17818531-27 from 507578 to 985495 tid 1 2026-02-21T22:16:07.561 INFO:tasks.rados.rados.0.trial178.stdout:71: writing trial17818531-27 from 1543557 to 2093561 tid 2 2026-02-21T22:16:07.564 INFO:tasks.rados.rados.0.trial178.stdout:71: writing trial17818531-27 from 2833665 to 2907578 tid 3 2026-02-21T22:16:07.564 INFO:tasks.rados.rados.0.trial178.stdout:72: write oid 38 current snap is 2 2026-02-21T22:16:07.564 INFO:tasks.rados.rados.0.trial178.stdout:72: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2026-02-21T22:16:07.568 INFO:tasks.rados.rados.0.trial178.stdout:72: writing trial17818531-38 from 524385 to 1077551 tid 1 2026-02-21T22:16:07.571 INFO:tasks.rados.rados.0.trial178.stdout:72: writing trial17818531-38 from 1602039 to 2012054 tid 2 2026-02-21T22:16:07.575 INFO:tasks.rados.rados.0.trial178.stdout:72: writing trial17818531-38 from 2577441 to 2924385 tid 3 2026-02-21T22:16:07.575 INFO:tasks.rados.rados.0.trial178.stdout:73: delete oid 45 current snap is 2 2026-02-21T22:16:07.575 INFO:tasks.rados.rados.0.trial178.stdout:71: finishing write tid 1 to trial17818531-27 2026-02-21T22:16:07.575 INFO:tasks.rados.rados.0.trial178.stdout:71: finishing write tid 2 to trial17818531-27 2026-02-21T22:16:07.575 INFO:tasks.rados.rados.0.trial178.stdout:71: finishing write tid 3 to trial17818531-27 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: finishing write tid 4 to trial17818531-27 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: finishing write tid 5 to trial17818531-27 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: oid 27 updating version 0 to 20 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: oid 27 version 20 is already newer than 18 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: oid 27 version 20 is already newer than 19 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: oid 27 version 20 is already newer than 17 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 20 (ObjNum 54 snap 2 seq_num 54) dirty exists 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:71: left oid 27 (ObjNum 54 snap 2 seq_num 54) 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:72: finishing write tid 1 to trial17818531-38 2026-02-21T22:16:07.576 INFO:tasks.rados.rados.0.trial178.stdout:72: finishing write tid 2 to trial17818531-38 2026-02-21T22:16:07.577 INFO:tasks.rados.rados.0.trial178.stdout:72: finishing write tid 3 to trial17818531-38 2026-02-21T22:16:07.577 INFO:tasks.rados.rados.0.trial178.stdout:72: finishing write tid 4 to trial17818531-38 2026-02-21T22:16:07.577 INFO:tasks.rados.rados.0.trial178.stdout:72: finishing write tid 5 to trial17818531-38 2026-02-21T22:16:07.577 INFO:tasks.rados.rados.0.trial178.stdout:72: oid 38 updating version 0 to 11 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:72: oid 38 updating version 11 to 13 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:72: oid 38 version 13 is already newer than 12 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:72: oid 38 updating version 13 to 14 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 14 (ObjNum 55 snap 2 seq_num 55) dirty exists 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:72: left oid 38 (ObjNum 55 snap 2 seq_num 55) 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:71: done (2 left) 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:72: done (1 left) 2026-02-21T22:16:07.578 INFO:tasks.rados.rados.0.trial178.stdout:73: done (0 left) 2026-02-21T22:16:07.579 INFO:tasks.rados.rados.0.trial178.stdout:74: snap_remove snap 1 2026-02-21T22:16:08.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:08.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:08.554 INFO:tasks.rados.rados.0.trial178.stdout:74: done (0 left) 2026-02-21T22:16:08.555 INFO:tasks.rados.rados.0.trial178.stdout:75: snap_create 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:75: done (0 left) 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:76: read oid 20 snap -1 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:76: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:77: rollback oid 45 current snap is 3 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 2 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:78: rollback oid 27 current snap is 3 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 2 2026-02-21T22:16:09.559 INFO:tasks.rados.rados.0.trial178.stdout:79: rollback oid 29 current snap is 3 2026-02-21T22:16:09.560 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 2 2026-02-21T22:16:09.560 INFO:tasks.rados.rados.0.trial178.stdout:80: snap_create 2026-02-21T22:16:09.560 INFO:tasks.rados.rados.0.trial178.stdout:77: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:16:09.560 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 0 (ObjNum 129 snap 0 seq_num 101622456) dirty dne 2026-02-21T22:16:09.564 INFO:tasks.rados.rados.0.trial178.stdout:78: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:16:09.565 INFO:tasks.rados.rados.0.trial178.stdout:78: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:16:09.565 INFO:tasks.rados.rados.0.trial178.stdout:78: finishing rollback tid 2 to trial17818531-27 2026-02-21T22:16:09.565 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 26 (ObjNum 54 snap 2 seq_num 54) dirty exists 2026-02-21T22:16:09.566 INFO:tasks.rados.rados.0.trial178.stdout:76: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-21T22:16:09.566 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:09.597 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:09.622 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 449479 2026-02-21T22:16:09.638 INFO:tasks.rados.rados.0.trial178.stdout:79: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:16:09.638 INFO:tasks.rados.rados.0.trial178.stdout:79: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:16:09.639 INFO:tasks.rados.rados.0.trial178.stdout:79: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:16:09.639 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 10 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:76: done (4 left) 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:77: done (3 left) 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:78: done (2 left) 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:79: done (1 left) 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:80: done (0 left) 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:81: rollback oid 18 current snap is 4 2026-02-21T22:16:10.560 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 2 2026-02-21T22:16:10.561 INFO:tasks.rados.rados.0.trial178.stdout:82: write (excl) oid 50 current snap is 4 2026-02-21T22:16:10.561 INFO:tasks.rados.rados.0.trial178.stdout:82: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2026-02-21T22:16:10.568 INFO:tasks.rados.rados.0.trial178.stdout:82: writing trial17818531-50 from 557999 to 1261663 tid 1 2026-02-21T22:16:10.574 INFO:tasks.rados.rados.0.trial178.stdout:82: writing trial17818531-50 from 1719003 to 2132687 tid 2 2026-02-21T22:16:10.580 INFO:tasks.rados.rados.0.trial178.stdout:82: writing trial17818531-50 from 2632287 to 2957999 tid 3 2026-02-21T22:16:10.580 INFO:tasks.rados.rados.0.trial178.stdout:83: read oid 23 snap -1 2026-02-21T22:16:10.580 INFO:tasks.rados.rados.0.trial178.stdout:83: expect (ObjNum 22 snap 0 seq_num 22) 2026-02-21T22:16:10.580 INFO:tasks.rados.rados.0.trial178.stdout:84: read oid 4 snap -1 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:84: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:85: rollback oid 11 current snap is 4 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 2 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:81: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:81: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:81: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 14 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:82: finishing write tid 1 to trial17818531-50 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:82: finishing write tid 2 to trial17818531-50 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:81: done (4 left) 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:86: read oid 7 snap -1 2026-02-21T22:16:10.581 INFO:tasks.rados.rados.0.trial178.stdout:86: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:10.582 INFO:tasks.rados.rados.0.trial178.stdout:87: write oid 13 current snap is 4 2026-02-21T22:16:10.582 INFO:tasks.rados.rados.0.trial178.stdout:87: seq_num 58 ranges {574806=778913,2061132=557342} 2026-02-21T22:16:10.588 INFO:tasks.rados.rados.0.trial178.stdout:87: writing trial17818531-13 from 574806 to 1353719 tid 1 2026-02-21T22:16:10.597 INFO:tasks.rados.rados.0.trial178.stdout:87: writing trial17818531-13 from 2061132 to 2618474 tid 2 2026-02-21T22:16:10.597 INFO:tasks.rados.rados.0.trial178.stdout:88: write (excl) oid 40 current snap is 4 2026-02-21T22:16:10.597 INFO:tasks.rados.rados.0.trial178.stdout:88: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2026-02-21T22:16:10.604 INFO:tasks.rados.rados.0.trial178.stdout:88: writing trial17818531-40 from 591613 to 1045775 tid 1 2026-02-21T22:16:10.609 INFO:tasks.rados.rados.0.trial178.stdout:88: writing trial17818531-40 from 1719614 to 2136967 tid 2 2026-02-21T22:16:10.612 INFO:tasks.rados.rados.0.trial178.stdout:88: writing trial17818531-40 from 2570780 to 2991613 tid 3 2026-02-21T22:16:10.612 INFO:tasks.rados.rados.0.trial178.stdout:89: read oid 44 snap -1 2026-02-21T22:16:10.612 INFO:tasks.rados.rados.0.trial178.stdout:89: expect (ObjNum 43 snap 0 seq_num 43) 2026-02-21T22:16:10.613 INFO:tasks.rados.rados.0.trial178.stdout:90: write oid 28 current snap is 4 2026-02-21T22:16:10.613 INFO:tasks.rados.rados.0.trial178.stdout:90: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2026-02-21T22:16:10.616 INFO:tasks.rados.rados.0.trial178.stdout:90: writing trial17818531-28 from 608420 to 1137831 tid 1 2026-02-21T22:16:10.621 INFO:tasks.rados.rados.0.trial178.stdout:90: writing trial17818531-28 from 1778096 to 2339107 tid 2 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:90: writing trial17818531-28 from 2881850 to 3008420 tid 3 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:91: read oid 19 snap -1 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:91: expect (ObjNum 18 snap 0 seq_num 18) 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:92: delete oid 35 current snap is 4 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:82: finishing write tid 3 to trial17818531-50 2026-02-21T22:16:10.624 INFO:tasks.rados.rados.0.trial178.stdout:83: expect (ObjNum 22 snap 0 seq_num 22) 2026-02-21T22:16:10.657 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1743985 2026-02-21T22:16:10.688 INFO:tasks.rados.rados.0.trial178.stdout:82: finishing write tid 4 to trial17818531-50 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: finishing write tid 5 to trial17818531-50 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: oid 50 updating version 0 to 11 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: oid 50 updating version 11 to 13 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: oid 50 version 13 is already newer than 12 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: oid 50 updating version 13 to 14 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 14 (ObjNum 57 snap 4 seq_num 57) dirty exists 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:82: left oid 50 (ObjNum 57 snap 4 seq_num 57) 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:85: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:85: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:16:10.689 INFO:tasks.rados.rados.0.trial178.stdout:85: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:16:10.690 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 19 (ObjNum 10 snap 0 seq_num 10) dirty exists 2026-02-21T22:16:10.690 INFO:tasks.rados.rados.0.trial178.stdout:84: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:10.690 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:10.767 INFO:tasks.rados.rados.0.trial178.stdout:86: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:10.805 INFO:tasks.rados.rados.0.trial178.stdout:87: finishing write tid 1 to trial17818531-13 2026-02-21T22:16:10.805 INFO:tasks.rados.rados.0.trial178.stdout:87: finishing write tid 2 to trial17818531-13 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: finishing write tid 3 to trial17818531-13 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: finishing write tid 4 to trial17818531-13 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: oid 13 updating version 0 to 22 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: oid 13 version 22 is already newer than 21 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: oid 13 updating version 22 to 23 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 23 (ObjNum 58 snap 4 seq_num 58) dirty exists 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:87: left oid 13 (ObjNum 58 snap 4 seq_num 58) 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:88: finishing write tid 1 to trial17818531-40 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:88: finishing write tid 2 to trial17818531-40 2026-02-21T22:16:10.806 INFO:tasks.rados.rados.0.trial178.stdout:88: finishing write tid 3 to trial17818531-40 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:82: done (10 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:83: done (9 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:84: done (8 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:85: done (7 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:86: done (6 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:87: done (5 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:92: done (4 left) 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:93: rollback oid 6 current snap is 4 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 2 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:88: finishing write tid 4 to trial17818531-40 2026-02-21T22:16:10.807 INFO:tasks.rados.rados.0.trial178.stdout:88: finishing write tid 5 to trial17818531-40 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:88: oid 40 updating version 0 to 17 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:88: oid 40 updating version 17 to 19 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:88: oid 40 version 19 is already newer than 18 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:88: oid 40 version 19 is already newer than 16 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 19 (ObjNum 59 snap 4 seq_num 59) dirty exists 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:88: left oid 40 (ObjNum 59 snap 4 seq_num 59) 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:89: expect (ObjNum 43 snap 0 seq_num 43) 2026-02-21T22:16:10.808 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 700120 2026-02-21T22:16:10.860 INFO:tasks.rados.rados.0.trial178.stdout:90: finishing write tid 1 to trial17818531-28 2026-02-21T22:16:10.860 INFO:tasks.rados.rados.0.trial178.stdout:90: finishing write tid 2 to trial17818531-28 2026-02-21T22:16:10.860 INFO:tasks.rados.rados.0.trial178.stdout:90: finishing write tid 3 to trial17818531-28 2026-02-21T22:16:10.860 INFO:tasks.rados.rados.0.trial178.stdout:90: finishing write tid 4 to trial17818531-28 2026-02-21T22:16:10.860 INFO:tasks.rados.rados.0.trial178.stdout:91: expect (ObjNum 18 snap 0 seq_num 18) 2026-02-21T22:16:10.893 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: finishing write tid 5 to trial17818531-28 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: oid 28 updating version 0 to 18 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: oid 28 updating version 18 to 19 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: oid 28 updating version 19 to 20 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: oid 28 updating version 20 to 21 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 21 (ObjNum 60 snap 4 seq_num 60) dirty exists 2026-02-21T22:16:10.911 INFO:tasks.rados.rados.0.trial178.stdout:90: left oid 28 (ObjNum 60 snap 4 seq_num 60) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:93: finishing rollback tid 0 to trial17818531-6 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:93: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:93: finishing rollback tid 2 to trial17818531-6 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 19 (ObjNum 5 snap 0 seq_num 5) dirty exists 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:88: done (4 left) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:89: done (3 left) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:90: done (2 left) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:91: done (1 left) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:93: done (0 left) 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:94: read oid 5 snap -1 2026-02-21T22:16:10.912 INFO:tasks.rados.rados.0.trial178.stdout:94: expect (ObjNum 28 snap 0 seq_num 28) 2026-02-21T22:16:10.913 INFO:tasks.rados.rados.0.trial178.stdout:95: write oid 31 current snap is 4 2026-02-21T22:16:10.913 INFO:tasks.rados.rados.0.trial178.stdout:95: seq_num 62 ranges {642034=796262,1895060=564680} 2026-02-21T22:16:10.920 INFO:tasks.rados.rados.0.trial178.stdout:95: writing trial17818531-31 from 642034 to 1438296 tid 1 2026-02-21T22:16:10.927 INFO:tasks.rados.rados.0.trial178.stdout:95: writing trial17818531-31 from 1895060 to 2459740 tid 2 2026-02-21T22:16:10.927 INFO:tasks.rados.rados.0.trial178.stdout:96: write oid 2 current snap is 4 2026-02-21T22:16:10.927 INFO:tasks.rados.rados.0.trial178.stdout:96: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2026-02-21T22:16:10.934 INFO:tasks.rados.rados.0.trial178.stdout:96: writing trial17818531-2 from 658841 to 1130352 tid 1 2026-02-21T22:16:10.942 INFO:tasks.rados.rados.0.trial178.stdout:96: writing trial17818531-2 from 1553542 to 2261880 tid 2 2026-02-21T22:16:10.947 INFO:tasks.rados.rados.0.trial178.stdout:96: writing trial17818531-2 from 2964119 to 3058841 tid 3 2026-02-21T22:16:10.947 INFO:tasks.rados.rados.0.trial178.stdout:97: write oid 3 current snap is 4 2026-02-21T22:16:10.947 INFO:tasks.rados.rados.0.trial178.stdout:97: seq_num 64 ranges {675648=546760,2012024=568349} 2026-02-21T22:16:10.955 INFO:tasks.rados.rados.0.trial178.stdout:97: writing trial17818531-3 from 675648 to 1222408 tid 1 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:97: writing trial17818531-3 from 2012024 to 2580373 tid 2 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:98: read oid 11 snap -1 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:98: expect (ObjNum 10 snap 0 seq_num 10) 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:99: read oid 9 snap -1 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:99: expect (ObjNum 51 snap 0 seq_num 51) 2026-02-21T22:16:10.962 INFO:tasks.rados.rados.0.trial178.stdout:100: snap_remove snap 2 2026-02-21T22:16:11.567 INFO:tasks.rados.rados.0.trial178.stdout:100: done (6 left) 2026-02-21T22:16:11.567 INFO:tasks.rados.rados.0.trial178.stdout:101: write (excl) oid 22 current snap is 4 2026-02-21T22:16:11.567 INFO:tasks.rados.rados.0.trial178.stdout:101: seq_num 65 ranges {692455=622009,2070506=712007} 2026-02-21T22:16:11.572 INFO:tasks.rados.rados.0.trial178.stdout:101: writing trial17818531-22 from 692455 to 1314464 tid 1 2026-02-21T22:16:11.578 INFO:tasks.rados.rados.0.trial178.stdout:101: writing trial17818531-22 from 2070506 to 2782513 tid 2 2026-02-21T22:16:11.578 INFO:tasks.rados.rados.0.trial178.stdout:102: snap_create 2026-02-21T22:16:11.578 INFO:tasks.rados.rados.0.trial178.stdout:94: expect (ObjNum 28 snap 0 seq_num 28) 2026-02-21T22:16:11.645 INFO:tasks.rados.rados.0.trial178.stdout:95: finishing write tid 1 to trial17818531-31 2026-02-21T22:16:11.645 INFO:tasks.rados.rados.0.trial178.stdout:95: finishing write tid 2 to trial17818531-31 2026-02-21T22:16:11.645 INFO:tasks.rados.rados.0.trial178.stdout:95: finishing write tid 3 to trial17818531-31 2026-02-21T22:16:11.645 INFO:tasks.rados.rados.0.trial178.stdout:95: finishing write tid 4 to trial17818531-31 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:95: oid 31 updating version 0 to 11 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:95: oid 31 updating version 11 to 13 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:95: oid 31 version 13 is already newer than 12 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 13 (ObjNum 62 snap 4 seq_num 62) dirty exists 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:95: left oid 31 (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: finishing write tid 1 to trial17818531-2 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: finishing write tid 2 to trial17818531-2 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: finishing write tid 3 to trial17818531-2 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: finishing write tid 4 to trial17818531-2 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: finishing write tid 5 to trial17818531-2 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: oid 2 updating version 0 to 22 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: oid 2 updating version 22 to 24 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: oid 2 version 24 is already newer than 23 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: oid 2 version 24 is already newer than 21 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 24 (ObjNum 63 snap 4 seq_num 63) dirty exists 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:96: left oid 2 (ObjNum 63 snap 4 seq_num 63) 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:97: finishing write tid 1 to trial17818531-3 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:99: expect (ObjNum 51 snap 0 seq_num 51) 2026-02-21T22:16:11.646 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:11.700 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 833081 2026-02-21T22:16:11.723 INFO:tasks.rados.rados.0.trial178.stdout:97: finishing write tid 2 to trial17818531-3 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: finishing write tid 3 to trial17818531-3 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: finishing write tid 4 to trial17818531-3 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: oid 3 updating version 0 to 29 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: oid 3 version 29 is already newer than 27 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: oid 3 version 29 is already newer than 28 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 29 (ObjNum 64 snap 4 seq_num 64) dirty exists 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:97: left oid 3 (ObjNum 64 snap 4 seq_num 64) 2026-02-21T22:16:11.724 INFO:tasks.rados.rados.0.trial178.stdout:98: expect (ObjNum 10 snap 0 seq_num 10) 2026-02-21T22:16:11.735 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:11.752 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 659083 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: finishing write tid 1 to trial17818531-22 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: finishing write tid 2 to trial17818531-22 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: finishing write tid 3 to trial17818531-22 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: finishing write tid 4 to trial17818531-22 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: oid 22 updating version 0 to 27 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: oid 22 version 27 is already newer than 26 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: oid 22 version 27 is already newer than 25 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 27 (ObjNum 65 snap 4 seq_num 65) dirty exists 2026-02-21T22:16:11.763 INFO:tasks.rados.rados.0.trial178.stdout:101: left oid 22 (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:12.571 INFO:tasks.rados.rados.0.trial178.stdout:94: done (7 left) 2026-02-21T22:16:12.571 INFO:tasks.rados.rados.0.trial178.stdout:95: done (6 left) 2026-02-21T22:16:12.571 INFO:tasks.rados.rados.0.trial178.stdout:96: done (5 left) 2026-02-21T22:16:12.571 INFO:tasks.rados.rados.0.trial178.stdout:97: done (4 left) 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:98: done (3 left) 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:99: done (2 left) 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:101: done (1 left) 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:102: done (0 left) 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:103: copy_from oid 41 from oid 36 current snap is 5 2026-02-21T22:16:12.572 INFO:tasks.rados.rados.0.trial178.stdout:104: delete oid 50 current snap is 5 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:104: done (1 left) 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:105: read oid 20 snap -1 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:105: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:106: read oid 26 snap -1 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:106: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:12.573 INFO:tasks.rados.rados.0.trial178.stdout:107: delete oid 25 current snap is 5 2026-02-21T22:16:12.574 INFO:tasks.rados.rados.0.trial178.stdout:105: expect (ObjNum 19 snap 0 seq_num 19) 2026-02-21T22:16:12.600 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2064416 2026-02-21T22:16:12.619 INFO:tasks.rados.rados.0.trial178.stdout:106: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:12.706 INFO:tasks.rados.rados.0.trial178.stdout:103: finishing copy_from racing read to trial17818531-41 2026-02-21T22:16:12.706 INFO:tasks.rados.rados.0.trial178.stdout:103: finishing copy_from to trial17818531-41 2026-02-21T22:16:12.706 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 3 (ObjNum 35 snap 0 seq_num 35) dirty exists 2026-02-21T22:16:12.706 INFO:tasks.rados.rados.0.trial178.stdout:103: done (3 left) 2026-02-21T22:16:12.707 INFO:tasks.rados.rados.0.trial178.stdout:105: done (2 left) 2026-02-21T22:16:12.707 INFO:tasks.rados.rados.0.trial178.stdout:106: done (1 left) 2026-02-21T22:16:12.707 INFO:tasks.rados.rados.0.trial178.stdout:107: done (0 left) 2026-02-21T22:16:12.707 INFO:tasks.rados.rados.0.trial178.stdout:108: snap_remove snap 4 2026-02-21T22:16:13.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:13.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:13.573 INFO:tasks.rados.rados.0.trial178.stdout:108: done (0 left) 2026-02-21T22:16:13.573 INFO:tasks.rados.rados.0.trial178.stdout:109: snap_create 2026-02-21T22:16:14.574 INFO:tasks.rados.rados.0.trial178.stdout:109: done (0 left) 2026-02-21T22:16:14.575 INFO:tasks.rados.rados.0.trial178.stdout:110: snap_create 2026-02-21T22:16:15.575 INFO:tasks.rados.rados.0.trial178.stdout:110: done (0 left) 2026-02-21T22:16:15.576 INFO:tasks.rados.rados.0.trial178.stdout:111: snap_remove snap 3 2026-02-21T22:16:15.682 INFO:tasks.rados.rados.0.trial178.stdout:111: done (0 left) 2026-02-21T22:16:15.682 INFO:tasks.rados.rados.0.trial178.stdout:112: write (excl) oid 39 current snap is 7 2026-02-21T22:16:15.682 INFO:tasks.rados.rados.0.trial178.stdout:112: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2026-02-21T22:16:15.689 INFO:tasks.rados.rados.0.trial178.stdout:112: writing trial17818531-39 from 742876 to 1190632 tid 1 2026-02-21T22:16:15.697 INFO:tasks.rados.rados.0.trial178.stdout:112: writing trial17818531-39 from 1729599 to 2305286 tid 2 2026-02-21T22:16:15.703 INFO:tasks.rados.rados.0.trial178.stdout:112: writing trial17818531-39 from 3101234 to 3142876 tid 3 2026-02-21T22:16:15.703 INFO:tasks.rados.rados.0.trial178.stdout:113: read oid 6 snap -1 2026-02-21T22:16:15.703 INFO:tasks.rados.rados.0.trial178.stdout:113: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-21T22:16:15.703 INFO:tasks.rados.rados.0.trial178.stdout:114: delete oid 8 current snap is 7 2026-02-21T22:16:15.703 INFO:tasks.rados.rados.0.trial178.stdout:112: finishing write tid 1 to trial17818531-39 2026-02-21T22:16:15.704 INFO:tasks.rados.rados.0.trial178.stdout:112: finishing write tid 2 to trial17818531-39 2026-02-21T22:16:15.704 INFO:tasks.rados.rados.0.trial178.stdout:112: finishing write tid 3 to trial17818531-39 2026-02-21T22:16:15.704 INFO:tasks.rados.rados.0.trial178.stdout:112: finishing write tid 4 to trial17818531-39 2026-02-21T22:16:15.704 INFO:tasks.rados.rados.0.trial178.stdout:112: finishing write tid 5 to trial17818531-39 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: oid 39 updating version 0 to 33 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: oid 39 version 33 is already newer than 31 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: oid 39 version 33 is already newer than 32 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: oid 39 version 33 is already newer than 30 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 33 (ObjNum 68 snap 7 seq_num 68) dirty exists 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: left oid 39 (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:16:15.705 INFO:tasks.rados.rados.0.trial178.stdout:112: done (2 left) 2026-02-21T22:16:15.706 INFO:tasks.rados.rados.0.trial178.stdout:114: done (1 left) 2026-02-21T22:16:15.706 INFO:tasks.rados.rados.0.trial178.stdout:115: snap_create 2026-02-21T22:16:15.706 INFO:tasks.rados.rados.0.trial178.stdout:113: expect (ObjNum 5 snap 0 seq_num 5) 2026-02-21T22:16:16.692 INFO:tasks.rados.rados.0.trial178.stdout:113: done (1 left) 2026-02-21T22:16:16.693 INFO:tasks.rados.rados.0.trial178.stdout:115: done (0 left) 2026-02-21T22:16:16.693 INFO:tasks.rados.rados.0.trial178.stdout:116: snap_remove snap 7 2026-02-21T22:16:17.693 INFO:tasks.rados.rados.0.trial178.stdout:116: done (0 left) 2026-02-21T22:16:17.694 INFO:tasks.rados.rados.0.trial178.stdout:117: delete oid 48 current snap is 8 2026-02-21T22:16:17.697 INFO:tasks.rados.rados.0.trial178.stdout:117: done (0 left) 2026-02-21T22:16:17.697 INFO:tasks.rados.rados.0.trial178.stdout:118: read oid 25 snap -1 2026-02-21T22:16:17.697 INFO:tasks.rados.rados.0.trial178.stdout:118: expect deleted 2026-02-21T22:16:17.698 INFO:tasks.rados.rados.0.trial178.stdout:119: read oid 4 snap -1 2026-02-21T22:16:17.698 INFO:tasks.rados.rados.0.trial178.stdout:119: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:17.698 INFO:tasks.rados.rados.0.trial178.stdout:120: write oid 12 current snap is 8 2026-02-21T22:16:17.698 INFO:tasks.rados.rados.0.trial178.stdout:120: seq_num 71 ranges {793297=789856,2305045=723014} 2026-02-21T22:16:17.707 INFO:tasks.rados.rados.0.trial178.stdout:120: writing trial17818531-12 from 793297 to 1583153 tid 1 2026-02-21T22:16:17.717 INFO:tasks.rados.rados.0.trial178.stdout:120: writing trial17818531-12 from 2305045 to 3028059 tid 2 2026-02-21T22:16:17.718 INFO:tasks.rados.rados.0.trial178.stdout:121: snap_remove snap 6 2026-02-21T22:16:18.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:18.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:18.697 INFO:tasks.rados.rados.0.trial178.stdout:121: done (3 left) 2026-02-21T22:16:18.697 INFO:tasks.rados.rados.0.trial178.stdout:122: copy_from oid 47 from oid 2 current snap is 8 2026-02-21T22:16:18.698 INFO:tasks.rados.rados.0.trial178.stdout:119: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:18.698 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:18.770 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:18.796 INFO:tasks.rados.rados.0.trial178.stdout:120: finishing write tid 1 to trial17818531-12 2026-02-21T22:16:18.796 INFO:tasks.rados.rados.0.trial178.stdout:120: finishing write tid 2 to trial17818531-12 2026-02-21T22:16:18.796 INFO:tasks.rados.rados.0.trial178.stdout:120: finishing write tid 3 to trial17818531-12 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: finishing write tid 4 to trial17818531-12 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: oid 12 updating version 0 to 20 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: oid 12 version 20 is already newer than 19 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: oid 12 version 20 is already newer than 18 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 20 (ObjNum 71 snap 8 seq_num 71) dirty exists 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: left oid 12 (ObjNum 71 snap 8 seq_num 71) 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:122: finishing copy_from racing read to trial17818531-47 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:122: finishing copy_from to trial17818531-47 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 24 (ObjNum 63 snap 4 seq_num 63) dirty exists 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:118: done (3 left) 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:119: done (2 left) 2026-02-21T22:16:18.797 INFO:tasks.rados.rados.0.trial178.stdout:120: done (1 left) 2026-02-21T22:16:18.798 INFO:tasks.rados.rados.0.trial178.stdout:122: done (0 left) 2026-02-21T22:16:18.798 INFO:tasks.rados.rados.0.trial178.stdout:123: snap_create 2026-02-21T22:16:19.703 INFO:tasks.rados.rados.0.trial178.stdout:123: done (0 left) 2026-02-21T22:16:19.703 INFO:tasks.rados.rados.0.trial178.stdout:124: write (excl) oid 14 current snap is 9 2026-02-21T22:16:19.703 INFO:tasks.rados.rados.0.trial178.stdout:124: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2026-02-21T22:16:19.709 INFO:tasks.rados.rados.0.trial178.stdout:124: writing trial17818531-14 from 410104 to 875209 tid 1 2026-02-21T22:16:19.717 INFO:tasks.rados.rados.0.trial178.stdout:124: writing trial17818531-14 from 1563527 to 2146552 tid 2 2026-02-21T22:16:19.725 INFO:tasks.rados.rados.0.trial178.stdout:124: writing trial17818531-14 from 2810926 to 3210104 tid 3 2026-02-21T22:16:19.725 INFO:tasks.rados.rados.0.trial178.stdout:125: snap_remove snap 5 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:125: done (1 left) 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:126: delete oid 37 current snap is 9 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:124: finishing write tid 1 to trial17818531-14 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:124: finishing write tid 2 to trial17818531-14 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:124: finishing write tid 3 to trial17818531-14 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:124: finishing write tid 4 to trial17818531-14 2026-02-21T22:16:20.683 INFO:tasks.rados.rados.0.trial178.stdout:124: finishing write tid 5 to trial17818531-14 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:124: oid 14 updating version 0 to 40 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:124: oid 14 version 40 is already newer than 38 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:124: oid 14 version 40 is already newer than 37 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:124: oid 14 version 40 is already newer than 39 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 40 (ObjNum 72 snap 9 seq_num 72) dirty exists 2026-02-21T22:16:20.684 INFO:tasks.rados.rados.0.trial178.stdout:124: left oid 14 (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:16:20.687 INFO:tasks.rados.rados.0.trial178.stdout:124: done (1 left) 2026-02-21T22:16:20.687 INFO:tasks.rados.rados.0.trial178.stdout:126: done (0 left) 2026-02-21T22:16:20.687 INFO:tasks.rados.rados.0.trial178.stdout:127: snap_create 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:127: done (0 left) 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:128: rollback oid 17 current snap is 10 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 9 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:129: rollback oid 37 current snap is 10 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 9 2026-02-21T22:16:21.684 INFO:tasks.rados.rados.0.trial178.stdout:130: read oid 45 snap -1 2026-02-21T22:16:21.685 INFO:tasks.rados.rados.0.trial178.stdout:130: expect deleted 2026-02-21T22:16:21.685 INFO:tasks.rados.rados.0.trial178.stdout:131: read oid 10 snap -1 2026-02-21T22:16:21.685 INFO:tasks.rados.rados.0.trial178.stdout:131: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-21T22:16:21.685 INFO:tasks.rados.rados.0.trial178.stdout:132: write oid 20 current snap is 10 2026-02-21T22:16:21.685 INFO:tasks.rados.rados.0.trial178.stdout:132: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2026-02-21T22:16:21.691 INFO:tasks.rados.rados.0.trial178.stdout:132: writing trial17818531-20 from 443718 to 1059321 tid 1 2026-02-21T22:16:21.697 INFO:tasks.rados.rados.0.trial178.stdout:132: writing trial17818531-20 from 1564138 to 2034479 tid 2 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:132: writing trial17818531-20 from 2633066 to 3243718 tid 3 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:133: rollback oid 18 current snap is 10 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 8 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:128: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:128: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:16:21.706 INFO:tasks.rados.rados.0.trial178.stdout:130: done (5 left) 2026-02-21T22:16:21.707 INFO:tasks.rados.rados.0.trial178.stdout:134: write (excl) oid 29 current snap is 10 2026-02-21T22:16:21.707 INFO:tasks.rados.rados.0.trial178.stdout:134: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2026-02-21T22:16:21.713 INFO:tasks.rados.rados.0.trial178.stdout:134: writing trial17818531-29 from 460525 to 1151377 tid 1 2026-02-21T22:16:21.721 INFO:tasks.rados.rados.0.trial178.stdout:134: writing trial17818531-29 from 1622620 to 2352972 tid 2 2026-02-21T22:16:21.728 INFO:tasks.rados.rados.0.trial178.stdout:134: writing trial17818531-29 from 3060489 to 3260525 tid 3 2026-02-21T22:16:21.728 INFO:tasks.rados.rados.0.trial178.stdout:135: read oid 7 snap 8 2026-02-21T22:16:21.728 INFO:tasks.rados.rados.0.trial178.stdout:135: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:21.728 INFO:tasks.rados.rados.0.trial178.stdout:136: delete oid 21 current snap is 10 2026-02-21T22:16:21.728 INFO:tasks.rados.rados.0.trial178.stdout:128: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:16:21.729 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 22 (ObjNum 16 snap 0 seq_num 16) dirty exists 2026-02-21T22:16:21.729 INFO:tasks.rados.rados.0.trial178.stdout:129: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:16:21.729 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 23 (ObjNum 33 snap 0 seq_num 4122737488) dirty dne 2026-02-21T22:16:21.729 INFO:tasks.rados.rados.0.trial178.stdout:131: expect (ObjNum 9 snap 0 seq_num 9) 2026-02-21T22:16:21.731 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1757919 2026-02-21T22:16:21.744 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:21.772 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 157337 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: finishing write tid 1 to trial17818531-20 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: finishing write tid 2 to trial17818531-20 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:133: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: finishing write tid 3 to trial17818531-20 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: finishing write tid 4 to trial17818531-20 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: finishing write tid 5 to trial17818531-20 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: oid 20 updating version 0 to 24 2026-02-21T22:16:21.787 INFO:tasks.rados.rados.0.trial178.stdout:132: oid 20 updating version 24 to 25 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:132: oid 20 updating version 25 to 26 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:132: oid 20 updating version 26 to 27 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 27 (ObjNum 74 snap 10 seq_num 74) dirty exists 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:132: left oid 20 (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:133: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:133: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 22 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:134: finishing write tid 1 to trial17818531-29 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:134: finishing write tid 2 to trial17818531-29 2026-02-21T22:16:21.788 INFO:tasks.rados.rados.0.trial178.stdout:135: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:21.808 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1894088 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: finishing write tid 3 to trial17818531-29 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: finishing write tid 4 to trial17818531-29 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: finishing write tid 5 to trial17818531-29 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: oid 29 updating version 0 to 15 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: oid 29 updating version 15 to 17 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: oid 29 version 17 is already newer than 14 2026-02-21T22:16:21.826 INFO:tasks.rados.rados.0.trial178.stdout:134: oid 29 version 17 is already newer than 16 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 17 (ObjNum 75 snap 10 seq_num 75) dirty exists 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:134: left oid 29 (ObjNum 75 snap 10 seq_num 75) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:128: done (7 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:129: done (6 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:131: done (5 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:132: done (4 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:133: done (3 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:134: done (2 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:135: done (1 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:136: done (0 left) 2026-02-21T22:16:21.827 INFO:tasks.rados.rados.0.trial178.stdout:137: copy_from oid 36 from oid 15 current snap is 10 2026-02-21T22:16:21.828 INFO:tasks.rados.rados.0.trial178.stdout:138: delete oid 3 current snap is 10 2026-02-21T22:16:21.828 INFO:tasks.rados.rados.0.trial178.stdout:138: done (1 left) 2026-02-21T22:16:21.828 INFO:tasks.rados.rados.0.trial178.stdout:139: snap_create 2026-02-21T22:16:21.835 INFO:tasks.rados.rados.0.trial178.stdout:137: finishing copy_from racing read to trial17818531-36 2026-02-21T22:16:21.836 INFO:tasks.rados.rados.0.trial178.stdout:137: finishing copy_from to trial17818531-36 2026-02-21T22:16:21.836 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 3 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:22.708 INFO:tasks.rados.rados.0.trial178.stdout:137: done (1 left) 2026-02-21T22:16:22.708 INFO:tasks.rados.rados.0.trial178.stdout:139: done (0 left) 2026-02-21T22:16:22.708 INFO:tasks.rados.rados.0.trial178.stdout:140: snap_create 2026-02-21T22:16:23.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:16:23] 2026-02-21T22:16:23.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:16:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:16:23, mode upmap, max misplaced 0.050000 2026-02-21T22:16:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:16:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:16:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'rbd', 'unique_pool_0'] 2026-02-21T22:16:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:16:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:16:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:16:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:16:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:16:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:16:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:16:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:16:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:16:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:16:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:16:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:16:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:16:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:16:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:16:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:16:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "42", 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:16:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:16:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:16:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:16:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 42, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 12, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:16:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:16:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:16:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:16:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:16:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:16:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00021124239321108218 of space, bias 1.0, pg target 0.06337271796332465 quantized to 32 (current 16) 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:16:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:16:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:16:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:16:23.709 INFO:tasks.rados.rados.0.trial178.stdout:140: done (0 left) 2026-02-21T22:16:23.709 INFO:tasks.rados.rados.0.trial178.stdout:141: rollback oid 23 current snap is 12 2026-02-21T22:16:23.709 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 10 2026-02-21T22:16:23.709 INFO:tasks.rados.rados.0.trial178.stdout:142: read oid 4 snap -1 2026-02-21T22:16:23.709 INFO:tasks.rados.rados.0.trial178.stdout:142: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:23.710 INFO:tasks.rados.rados.0.trial178.stdout:143: snap_remove snap 9 2026-02-21T22:16:24.711 INFO:tasks.rados.rados.0.trial178.stdout:143: done (2 left) 2026-02-21T22:16:24.711 INFO:tasks.rados.rados.0.trial178.stdout:144: read oid 16 snap -1 2026-02-21T22:16:24.711 INFO:tasks.rados.rados.0.trial178.stdout:144: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:24.711 INFO:tasks.rados.rados.0.trial178.stdout:145: read oid 32 snap -1 2026-02-21T22:16:24.711 INFO:tasks.rados.rados.0.trial178.stdout:145: expect (ObjNum 31 snap 0 seq_num 31) 2026-02-21T22:16:24.712 INFO:tasks.rados.rados.0.trial178.stdout:146: snap_remove snap 11 2026-02-21T22:16:25.684 INFO:tasks.rados.rados.0.trial178.stdout:146: done (4 left) 2026-02-21T22:16:25.684 INFO:tasks.rados.rados.0.trial178.stdout:147: copy_from oid 6 from oid 17 current snap is 12 2026-02-21T22:16:25.684 INFO:tasks.rados.rados.0.trial178.stdout:142: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:25.730 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 651477 2026-02-21T22:16:25.752 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:25.779 INFO:tasks.rados.rados.0.trial178.stdout:141: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:141: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:142: done (4 left) 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:148: delete oid 43 current snap is 12 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:141: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 10 (ObjNum 22 snap 0 seq_num 22) dirty exists 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:144: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:25.780 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:25.797 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:25.832 INFO:tasks.rados.rados.0.trial178.stdout:145: expect (ObjNum 31 snap 0 seq_num 31) 2026-02-21T22:16:25.832 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:25.872 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2294947 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:147: finishing copy_from racing read to trial17818531-6 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:147: finishing copy_from to trial17818531-6 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 22 (ObjNum 16 snap 0 seq_num 16) dirty exists 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:141: done (4 left) 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:144: done (3 left) 2026-02-21T22:16:25.877 INFO:tasks.rados.rados.0.trial178.stdout:145: done (2 left) 2026-02-21T22:16:25.878 INFO:tasks.rados.rados.0.trial178.stdout:147: done (1 left) 2026-02-21T22:16:25.878 INFO:tasks.rados.rados.0.trial178.stdout:148: done (0 left) 2026-02-21T22:16:25.878 INFO:tasks.rados.rados.0.trial178.stdout:149: snap_create 2026-02-21T22:16:26.717 INFO:tasks.rados.rados.0.trial178.stdout:149: done (0 left) 2026-02-21T22:16:26.717 INFO:tasks.rados.rados.0.trial178.stdout:150: write (excl) oid 28 current snap is 13 2026-02-21T22:16:26.717 INFO:tasks.rados.rados.0.trial178.stdout:150: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2026-02-21T22:16:26.725 INFO:tasks.rados.rados.0.trial178.stdout:150: writing trial17818531-28 from 527753 to 1235954 tid 1 2026-02-21T22:16:26.734 INFO:tasks.rados.rados.0.trial178.stdout:150: writing trial17818531-28 from 1856548 to 2594238 tid 2 2026-02-21T22:16:26.739 INFO:tasks.rados.rados.0.trial178.stdout:150: writing trial17818531-28 from 3286534 to 3327753 tid 3 2026-02-21T22:16:26.739 INFO:tasks.rados.rados.0.trial178.stdout:151: write (excl) oid 11 current snap is 13 2026-02-21T22:16:26.739 INFO:tasks.rados.rados.0.trial178.stdout:151: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2026-02-21T22:16:26.748 INFO:tasks.rados.rados.0.trial178.stdout:151: writing trial17818531-11 from 544560 to 1328010 tid 1 2026-02-21T22:16:26.755 INFO:tasks.rados.rados.0.trial178.stdout:151: writing trial17818531-11 from 1915030 to 2396378 tid 2 2026-02-21T22:16:26.761 INFO:tasks.rados.rados.0.trial178.stdout:151: writing trial17818531-11 from 2797604 to 3277365 tid 3 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:152: read oid 12 snap -1 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:152: expect (ObjNum 71 snap 8 seq_num 71) 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:153: read oid 34 snap 12 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:153: expect (ObjNum 33 snap 0 seq_num 33) 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:154: read oid 8 snap -1 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:154: expect deleted 2026-02-21T22:16:26.762 INFO:tasks.rados.rados.0.trial178.stdout:155: snap_remove snap 10 2026-02-21T22:16:27.735 INFO:tasks.rados.rados.0.trial178.stdout:155: done (5 left) 2026-02-21T22:16:27.735 INFO:tasks.rados.rados.0.trial178.stdout:156: read oid 20 snap -1 2026-02-21T22:16:27.735 INFO:tasks.rados.rados.0.trial178.stdout:156: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:16:27.735 INFO:tasks.rados.rados.0.trial178.stdout:157: snap_create 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: finishing write tid 1 to trial17818531-28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: finishing write tid 2 to trial17818531-28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: finishing write tid 3 to trial17818531-28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: finishing write tid 4 to trial17818531-28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: finishing write tid 5 to trial17818531-28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: oid 28 updating version 0 to 30 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: oid 28 version 30 is already newer than 27 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: oid 28 version 30 is already newer than 28 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: oid 28 version 30 is already newer than 29 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 30 (ObjNum 79 snap 13 seq_num 79) dirty exists 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:150: left oid 28 (ObjNum 79 snap 13 seq_num 79) 2026-02-21T22:16:27.736 INFO:tasks.rados.rados.0.trial178.stdout:151: finishing write tid 1 to trial17818531-11 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: finishing write tid 2 to trial17818531-11 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: finishing write tid 3 to trial17818531-11 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: finishing write tid 4 to trial17818531-11 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: finishing write tid 5 to trial17818531-11 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: oid 11 updating version 0 to 52 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: oid 11 updating version 52 to 53 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: oid 11 version 53 is already newer than 51 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: oid 11 updating version 53 to 54 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 54 (ObjNum 80 snap 13 seq_num 80) dirty exists 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:151: left oid 11 (ObjNum 80 snap 13 seq_num 80) 2026-02-21T22:16:27.737 INFO:tasks.rados.rados.0.trial178.stdout:153: expect (ObjNum 33 snap 0 seq_num 33) 2026-02-21T22:16:27.773 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 704352 2026-02-21T22:16:27.787 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 190502 2026-02-21T22:16:27.806 INFO:tasks.rados.rados.0.trial178.stdout:152: expect (ObjNum 71 snap 8 seq_num 71) 2026-02-21T22:16:27.837 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:27.869 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:27.900 INFO:tasks.rados.rados.0.trial178.stdout:156: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:16:28.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:28.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:150: done (6 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:151: done (5 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:152: done (4 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:153: done (3 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:154: done (2 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:156: done (1 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:157: done (0 left) 2026-02-21T22:16:28.734 INFO:tasks.rados.rados.0.trial178.stdout:158: delete oid 13 current snap is 14 2026-02-21T22:16:28.738 INFO:tasks.rados.rados.0.trial178.stdout:158: done (0 left) 2026-02-21T22:16:28.738 INFO:tasks.rados.rados.0.trial178.stdout:159: read oid 9 snap -1 2026-02-21T22:16:28.738 INFO:tasks.rados.rados.0.trial178.stdout:159: expect (ObjNum 51 snap 0 seq_num 51) 2026-02-21T22:16:28.738 INFO:tasks.rados.rados.0.trial178.stdout:160: write oid 4 current snap is 14 2026-02-21T22:16:28.739 INFO:tasks.rados.rados.0.trial178.stdout:160: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2026-02-21T22:16:28.746 INFO:tasks.rados.rados.0.trial178.stdout:160: writing trial17818531-4 from 578174 to 1112122 tid 1 2026-02-21T22:16:28.752 INFO:tasks.rados.rados.0.trial178.stdout:160: writing trial17818531-4 from 1515641 to 2000658 tid 2 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:160: writing trial17818531-4 from 2736097 to 3238402 tid 3 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:161: read oid 16 snap -1 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:161: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:162: read oid 6 snap -1 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:162: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:163: delete oid 35 current snap is 14 2026-02-21T22:16:28.760 INFO:tasks.rados.rados.0.trial178.stdout:159: expect (ObjNum 51 snap 0 seq_num 51) 2026-02-21T22:16:28.796 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1716985 2026-02-21T22:16:28.843 INFO:tasks.rados.rados.0.trial178.stdout:160: finishing write tid 1 to trial17818531-4 2026-02-21T22:16:28.843 INFO:tasks.rados.rados.0.trial178.stdout:160: finishing write tid 2 to trial17818531-4 2026-02-21T22:16:28.843 INFO:tasks.rados.rados.0.trial178.stdout:161: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:28.843 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:28.860 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:28.889 INFO:tasks.rados.rados.0.trial178.stdout:160: finishing write tid 3 to trial17818531-4 2026-02-21T22:16:28.889 INFO:tasks.rados.rados.0.trial178.stdout:160: finishing write tid 4 to trial17818531-4 2026-02-21T22:16:28.889 INFO:tasks.rados.rados.0.trial178.stdout:160: finishing write tid 5 to trial17818531-4 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:160: oid 4 updating version 0 to 27 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:160: oid 4 version 27 is already newer than 26 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:160: oid 4 updating version 27 to 29 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:160: oid 4 version 29 is already newer than 28 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 29 (ObjNum 82 snap 14 seq_num 82) dirty exists 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:160: left oid 4 (ObjNum 82 snap 14 seq_num 82) 2026-02-21T22:16:28.890 INFO:tasks.rados.rados.0.trial178.stdout:162: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:28.894 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1271492 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:159: done (4 left) 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:160: done (3 left) 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:161: done (2 left) 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:162: done (1 left) 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:163: done (0 left) 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:164: read oid 16 snap -1 2026-02-21T22:16:28.933 INFO:tasks.rados.rados.0.trial178.stdout:164: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:28.934 INFO:tasks.rados.rados.0.trial178.stdout:165: snap_create 2026-02-21T22:16:28.934 INFO:tasks.rados.rados.0.trial178.stdout:164: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:16:28.950 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 469448 2026-02-21T22:16:29.739 INFO:tasks.rados.rados.0.trial178.stdout:164: done (1 left) 2026-02-21T22:16:29.740 INFO:tasks.rados.rados.0.trial178.stdout:165: done (0 left) 2026-02-21T22:16:29.740 INFO:tasks.rados.rados.0.trial178.stdout:166: snap_create 2026-02-21T22:16:30.684 INFO:tasks.rados.rados.0.trial178.stdout:166: done (0 left) 2026-02-21T22:16:30.684 INFO:tasks.rados.rados.0.trial178.stdout:167: copy_from oid 49 from oid 5 current snap is 16 2026-02-21T22:16:30.685 INFO:tasks.rados.rados.0.trial178.stdout:168: write oid 48 current snap is 16 2026-02-21T22:16:30.685 INFO:tasks.rados.rados.0.trial178.stdout:168: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2026-02-21T22:16:30.691 INFO:tasks.rados.rados.0.trial178.stdout:168: writing trial17818531-48 from 611788 to 1012587 tid 1 2026-02-21T22:16:30.699 INFO:tasks.rados.rados.0.trial178.stdout:168: writing trial17818531-48 from 1748958 to 2237644 tid 2 2026-02-21T22:16:30.707 INFO:tasks.rados.rados.0.trial178.stdout:168: writing trial17818531-48 from 2907296 to 3411788 tid 3 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:169: delete oid 11 current snap is 16 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:168: finishing write tid 1 to trial17818531-48 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:167: finishing copy_from racing read to trial17818531-49 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:167: finishing copy_from to trial17818531-49 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 3 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-21T22:16:30.708 INFO:tasks.rados.rados.0.trial178.stdout:168: finishing write tid 2 to trial17818531-48 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:167: done (2 left) 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:169: done (1 left) 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:170: read oid 5 snap -1 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:170: expect (ObjNum 28 snap 0 seq_num 28) 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:171: delete oid 13 current snap is 16 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:168: finishing write tid 3 to trial17818531-48 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:168: finishing write tid 4 to trial17818531-48 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:168: finishing write tid 5 to trial17818531-48 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:168: oid 48 updating version 0 to 27 2026-02-21T22:16:30.710 INFO:tasks.rados.rados.0.trial178.stdout:168: oid 48 updating version 27 to 29 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:168: oid 48 version 29 is already newer than 28 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:168: oid 48 updating version 29 to 30 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 30 (ObjNum 84 snap 16 seq_num 84) dirty exists 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:168: left oid 48 (ObjNum 84 snap 16 seq_num 84) 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:168: done (2 left) 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:171: done (1 left) 2026-02-21T22:16:30.711 INFO:tasks.rados.rados.0.trial178.stdout:172: snap_remove snap 12 2026-02-21T22:16:31.685 INFO:tasks.rados.rados.0.trial178.stdout:172: done (1 left) 2026-02-21T22:16:31.685 INFO:tasks.rados.rados.0.trial178.stdout:173: read oid 40 snap -1 2026-02-21T22:16:31.685 INFO:tasks.rados.rados.0.trial178.stdout:173: expect (ObjNum 59 snap 4 seq_num 59) 2026-02-21T22:16:31.685 INFO:tasks.rados.rados.0.trial178.stdout:174: write (excl) oid 13 current snap is 16 2026-02-21T22:16:31.685 INFO:tasks.rados.rados.0.trial178.stdout:174: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2026-02-21T22:16:31.694 INFO:tasks.rados.rados.0.trial178.stdout:174: writing trial17818531-13 from 662209 to 1288755 tid 1 2026-02-21T22:16:31.701 INFO:tasks.rados.rados.0.trial178.stdout:174: writing trial17818531-13 from 1808051 to 2444064 tid 2 2026-02-21T22:16:31.708 INFO:tasks.rados.rados.0.trial178.stdout:174: writing trial17818531-13 from 2873212 to 3462209 tid 3 2026-02-21T22:16:31.709 INFO:tasks.rados.rados.0.trial178.stdout:175: snap_remove snap 14 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:175: done (3 left) 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:176: read oid 31 snap 8 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:176: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:177: rollback oid 4 current snap is 16 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 8 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:170: expect (ObjNum 28 snap 0 seq_num 28) 2026-02-21T22:16:32.689 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:32.754 INFO:tasks.rados.rados.0.trial178.stdout:173: expect (ObjNum 59 snap 4 seq_num 59) 2026-02-21T22:16:32.757 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 769652 2026-02-21T22:16:32.786 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1612662 2026-02-21T22:16:32.822 INFO:tasks.rados.rados.0.trial178.stdout:174: finishing write tid 1 to trial17818531-13 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: finishing write tid 2 to trial17818531-13 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: finishing write tid 3 to trial17818531-13 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: finishing write tid 4 to trial17818531-13 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: finishing write tid 5 to trial17818531-13 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: oid 13 updating version 0 to 68 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: oid 13 updating version 68 to 69 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: oid 13 version 69 is already newer than 67 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: oid 13 updating version 69 to 70 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 70 (ObjNum 87 snap 16 seq_num 87) dirty exists 2026-02-21T22:16:32.823 INFO:tasks.rados.rados.0.trial178.stdout:174: left oid 13 (ObjNum 87 snap 16 seq_num 87) 2026-02-21T22:16:32.824 INFO:tasks.rados.rados.0.trial178.stdout:177: finishing rollback tid 0 to trial17818531-4 2026-02-21T22:16:32.824 INFO:tasks.rados.rados.0.trial178.stdout:177: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:16:32.824 INFO:tasks.rados.rados.0.trial178.stdout:177: finishing rollback tid 2 to trial17818531-4 2026-02-21T22:16:32.824 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 35 (ObjNum 50 snap 0 seq_num 50) dirty exists 2026-02-21T22:16:32.824 INFO:tasks.rados.rados.0.trial178.stdout:176: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:16:32.836 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2194494 2026-02-21T22:16:32.846 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:32.875 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:32.905 INFO:tasks.rados.rados.0.trial178.stdout:170: done (4 left) 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:173: done (3 left) 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:174: done (2 left) 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:176: done (1 left) 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:177: done (0 left) 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:178: copy_from oid 49 from oid 42 current snap is 16 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:179: copy_from oid 36 from oid 27 current snap is 16 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:180: copy_from oid 8 from oid 7 current snap is 16 2026-02-21T22:16:32.906 INFO:tasks.rados.rados.0.trial178.stdout:181: snap_remove snap 8 2026-02-21T22:16:33.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:33.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:34.576 INFO:tasks.rados.rados.0.trial178.stdout:181: done (3 left) 2026-02-21T22:16:34.576 INFO:tasks.rados.rados.0.trial178.stdout:182: snap_create 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:178: finishing copy_from racing read to trial17818531-49 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:180: finishing copy_from racing read to trial17818531-8 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:178: finishing copy_from to trial17818531-49 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 18 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:180: finishing copy_from to trial17818531-8 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:179: finishing copy_from racing read to trial17818531-36 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:179: finishing copy_from to trial17818531-36 2026-02-21T22:16:34.577 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 26 (ObjNum 54 snap 2 seq_num 54) dirty exists 2026-02-21T22:16:35.578 INFO:tasks.rados.rados.0.trial178.stdout:178: done (3 left) 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:179: done (2 left) 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:180: done (1 left) 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:182: done (0 left) 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:183: read oid 18 snap -1 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:183: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:184: read oid 50 snap -1 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:184: expect deleted 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:185: write (excl) oid 43 current snap is 17 2026-02-21T22:16:35.579 INFO:tasks.rados.rados.0.trial178.stdout:185: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2026-02-21T22:16:35.588 INFO:tasks.rados.rados.0.trial178.stdout:185: writing trial17818531-43 from 679016 to 1380811 tid 1 2026-02-21T22:16:35.594 INFO:tasks.rados.rados.0.trial178.stdout:185: writing trial17818531-43 from 1866533 to 2362557 tid 2 2026-02-21T22:16:35.600 INFO:tasks.rados.rados.0.trial178.stdout:185: writing trial17818531-43 from 2900635 to 3303278 tid 3 2026-02-21T22:16:35.601 INFO:tasks.rados.rados.0.trial178.stdout:186: snap_remove snap 15 2026-02-21T22:16:35.685 INFO:tasks.rados.rados.0.trial178.stdout:186: done (3 left) 2026-02-21T22:16:35.685 INFO:tasks.rados.rados.0.trial178.stdout:187: write oid 10 current snap is 17 2026-02-21T22:16:35.685 INFO:tasks.rados.rados.0.trial178.stdout:187: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2026-02-21T22:16:35.694 INFO:tasks.rados.rados.0.trial178.stdout:187: writing trial17818531-10 from 695823 to 1472867 tid 1 2026-02-21T22:16:35.702 INFO:tasks.rados.rados.0.trial178.stdout:187: writing trial17818531-10 from 1925015 to 2564697 tid 2 2026-02-21T22:16:35.708 INFO:tasks.rados.rados.0.trial178.stdout:187: writing trial17818531-10 from 3328058 to 3495823 tid 3 2026-02-21T22:16:35.708 INFO:tasks.rados.rados.0.trial178.stdout:188: rollback oid 15 current snap is 17 2026-02-21T22:16:35.708 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 16 2026-02-21T22:16:35.709 INFO:tasks.rados.rados.0.trial178.stdout:183: expect (ObjNum 17 snap 0 seq_num 17) 2026-02-21T22:16:35.723 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2123527 2026-02-21T22:16:35.724 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:35.756 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 807713 2026-02-21T22:16:35.766 INFO:tasks.rados.rados.0.trial178.stdout:185: finishing write tid 1 to trial17818531-43 2026-02-21T22:16:35.766 INFO:tasks.rados.rados.0.trial178.stdout:185: finishing write tid 2 to trial17818531-43 2026-02-21T22:16:35.766 INFO:tasks.rados.rados.0.trial178.stdout:185: finishing write tid 3 to trial17818531-43 2026-02-21T22:16:35.766 INFO:tasks.rados.rados.0.trial178.stdout:185: finishing write tid 4 to trial17818531-43 2026-02-21T22:16:35.766 INFO:tasks.rados.rados.0.trial178.stdout:185: finishing write tid 5 to trial17818531-43 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:185: oid 43 updating version 0 to 11 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:185: oid 43 updating version 11 to 12 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:185: oid 43 updating version 12 to 13 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:185: oid 43 version 13 is already newer than 10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 13 (ObjNum 88 snap 17 seq_num 88) dirty exists 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:185: left oid 43 (ObjNum 88 snap 17 seq_num 88) 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: finishing write tid 1 to trial17818531-10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: finishing write tid 2 to trial17818531-10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: finishing write tid 3 to trial17818531-10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: finishing write tid 4 to trial17818531-10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: finishing write tid 5 to trial17818531-10 2026-02-21T22:16:35.767 INFO:tasks.rados.rados.0.trial178.stdout:187: oid 10 updating version 0 to 41 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:187: oid 10 updating version 41 to 42 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:187: oid 10 version 42 is already newer than 40 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:187: oid 10 updating version 42 to 43 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 43 (ObjNum 89 snap 17 seq_num 89) dirty exists 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:187: left oid 10 (ObjNum 89 snap 17 seq_num 89) 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:188: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:188: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:188: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 39 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:183: done (4 left) 2026-02-21T22:16:35.768 INFO:tasks.rados.rados.0.trial178.stdout:184: done (3 left) 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:185: done (2 left) 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:187: done (1 left) 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:188: done (0 left) 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:189: delete oid 33 current snap is 17 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:189: done (0 left) 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:190: write oid 32 current snap is 17 2026-02-21T22:16:35.769 INFO:tasks.rados.rados.0.trial178.stdout:190: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2026-02-21T22:16:35.776 INFO:tasks.rados.rados.0.trial178.stdout:190: writing trial17818531-32 from 729437 to 1256979 tid 1 2026-02-21T22:16:35.785 INFO:tasks.rados.rados.0.trial178.stdout:190: writing trial17818531-32 from 1925626 to 2568977 tid 2 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:190: writing trial17818531-32 from 3266551 to 3529437 tid 3 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:191: copy_from oid 48 from oid 14 current snap is 17 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:190: finishing write tid 1 to trial17818531-32 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:190: finishing write tid 2 to trial17818531-32 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:192: read oid 31 snap -1 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:192: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:16:35.792 INFO:tasks.rados.rados.0.trial178.stdout:193: snap_create 2026-02-21T22:16:35.794 INFO:tasks.rados.rados.0.trial178.stdout:190: finishing write tid 3 to trial17818531-32 2026-02-21T22:16:35.794 INFO:tasks.rados.rados.0.trial178.stdout:190: finishing write tid 4 to trial17818531-32 2026-02-21T22:16:35.794 INFO:tasks.rados.rados.0.trial178.stdout:190: finishing write tid 5 to trial17818531-32 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:190: oid 32 updating version 0 to 85 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:190: oid 32 version 85 is already newer than 84 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:190: oid 32 version 85 is already newer than 83 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:190: oid 32 version 85 is already newer than 82 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 85 (ObjNum 91 snap 17 seq_num 91) dirty exists 2026-02-21T22:16:35.795 INFO:tasks.rados.rados.0.trial178.stdout:190: left oid 32 (ObjNum 91 snap 17 seq_num 91) 2026-02-21T22:16:35.796 INFO:tasks.rados.rados.0.trial178.stdout:192: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:16:35.842 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:35.901 INFO:tasks.rados.rados.0.trial178.stdout:191: finishing copy_from racing read to trial17818531-48 2026-02-21T22:16:35.901 INFO:tasks.rados.rados.0.trial178.stdout:191: finishing copy_from to trial17818531-48 2026-02-21T22:16:35.901 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 40 (ObjNum 72 snap 9 seq_num 72) dirty exists 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:190: done (3 left) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:191: done (2 left) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:192: done (1 left) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:193: done (0 left) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:194: read oid 43 snap -1 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:194: expect (ObjNum 88 snap 17 seq_num 88) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:195: read oid 22 snap 13 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:195: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:196: copy_from oid 8 from oid 26 current snap is 18 2026-02-21T22:16:36.688 INFO:tasks.rados.rados.0.trial178.stdout:197: copy_from oid 20 from oid 25 current snap is 18 2026-02-21T22:16:36.689 INFO:tasks.rados.rados.0.trial178.stdout:198: delete oid 12 current snap is 18 2026-02-21T22:16:36.694 INFO:tasks.rados.rados.0.trial178.stdout:197: finishing copy_from to trial17818531-20 2026-02-21T22:16:36.694 INFO:tasks.rados.rados.0.trial178.stdout:197: got expected ENOENT (src dne) 2026-02-21T22:16:36.695 INFO:tasks.rados.rados.0.trial178.stdout:194: expect (ObjNum 88 snap 17 seq_num 88) 2026-02-21T22:16:36.737 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 714043 2026-02-21T22:16:36.756 INFO:tasks.rados.rados.0.trial178.stdout:197: finishing copy_from racing read to trial17818531-20 2026-02-21T22:16:36.756 INFO:tasks.rados.rados.0.trial178.stdout:195: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:36.763 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1554415 2026-02-21T22:16:36.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:196: finishing copy_from racing read to trial17818531-8 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:196: finishing copy_from to trial17818531-8 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 15 (ObjNum 53 snap 1 seq_num 53) dirty exists 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:194: done (4 left) 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:195: done (3 left) 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:196: done (2 left) 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:197: done (1 left) 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:198: done (0 left) 2026-02-21T22:16:36.847 INFO:tasks.rados.rados.0.trial178.stdout:199: snap_create 2026-02-21T22:16:37.692 INFO:tasks.rados.rados.0.trial178.stdout:199: done (0 left) 2026-02-21T22:16:37.692 INFO:tasks.rados.rados.0.trial178.stdout:200: delete oid 33 current snap is 19 2026-02-21T22:16:37.693 INFO:tasks.rados.rados.0.trial178.stdout:200: done (0 left) 2026-02-21T22:16:37.693 INFO:tasks.rados.rados.0.trial178.stdout:201: snap_remove snap 18 2026-02-21T22:16:38.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:38.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:38.692 INFO:tasks.rados.rados.0.trial178.stdout:201: done (0 left) 2026-02-21T22:16:38.692 INFO:tasks.rados.rados.0.trial178.stdout:202: read oid 26 snap 16 2026-02-21T22:16:38.692 INFO:tasks.rados.rados.0.trial178.stdout:202: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:38.692 INFO:tasks.rados.rados.0.trial178.stdout:203: snap_remove snap 17 2026-02-21T22:16:39.693 INFO:tasks.rados.rados.0.trial178.stdout:203: done (1 left) 2026-02-21T22:16:39.693 INFO:tasks.rados.rados.0.trial178.stdout:204: rollback oid 5 current snap is 19 2026-02-21T22:16:39.693 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 13 2026-02-21T22:16:39.693 INFO:tasks.rados.rados.0.trial178.stdout:202: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:39.701 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 700086 2026-02-21T22:16:39.766 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:39.796 INFO:tasks.rados.rados.0.trial178.stdout:204: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:16:39.796 INFO:tasks.rados.rados.0.trial178.stdout:204: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:16:39.797 INFO:tasks.rados.rados.0.trial178.stdout:204: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:16:39.797 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 30 (ObjNum 28 snap 0 seq_num 28) dirty exists 2026-02-21T22:16:39.797 INFO:tasks.rados.rados.0.trial178.stdout:202: done (1 left) 2026-02-21T22:16:39.797 INFO:tasks.rados.rados.0.trial178.stdout:204: done (0 left) 2026-02-21T22:16:39.797 INFO:tasks.rados.rados.0.trial178.stdout:205: delete oid 38 current snap is 19 2026-02-21T22:16:39.800 INFO:tasks.rados.rados.0.trial178.stdout:205: done (0 left) 2026-02-21T22:16:39.800 INFO:tasks.rados.rados.0.trial178.stdout:206: rollback oid 29 current snap is 19 2026-02-21T22:16:39.800 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 13 2026-02-21T22:16:39.800 INFO:tasks.rados.rados.0.trial178.stdout:207: snap_remove snap 16 2026-02-21T22:16:40.686 INFO:tasks.rados.rados.0.trial178.stdout:207: done (1 left) 2026-02-21T22:16:40.686 INFO:tasks.rados.rados.0.trial178.stdout:208: snap_create 2026-02-21T22:16:40.686 INFO:tasks.rados.rados.0.trial178.stdout:206: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:16:40.686 INFO:tasks.rados.rados.0.trial178.stdout:206: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:16:40.686 INFO:tasks.rados.rados.0.trial178.stdout:206: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:16:40.687 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 25 (ObjNum 75 snap 10 seq_num 75) dirty exists 2026-02-21T22:16:41.687 INFO:tasks.rados.rados.0.trial178.stdout:206: done (1 left) 2026-02-21T22:16:41.687 INFO:tasks.rados.rados.0.trial178.stdout:208: done (0 left) 2026-02-21T22:16:41.687 INFO:tasks.rados.rados.0.trial178.stdout:209: read oid 22 snap 19 2026-02-21T22:16:41.687 INFO:tasks.rados.rados.0.trial178.stdout:209: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:41.687 INFO:tasks.rados.rados.0.trial178.stdout:210: snap_remove snap 13 2026-02-21T22:16:42.688 INFO:tasks.rados.rados.0.trial178.stdout:210: done (1 left) 2026-02-21T22:16:42.689 INFO:tasks.rados.rados.0.trial178.stdout:211: write oid 9 current snap is 20 2026-02-21T22:16:42.689 INFO:tasks.rados.rados.0.trial178.stdout:211: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2026-02-21T22:16:42.697 INFO:tasks.rados.rados.0.trial178.stdout:211: writing trial17818531-9 from 796665 to 1341556 tid 1 2026-02-21T22:16:42.704 INFO:tasks.rados.rados.0.trial178.stdout:211: writing trial17818531-9 from 1759554 to 2410243 tid 2 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:211: writing trial17818531-9 from 2976243 to 3596665 tid 3 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:212: read oid 44 snap -1 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:212: expect (ObjNum 43 snap 0 seq_num 43) 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:213: read oid 6 snap -1 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:213: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:214: copy_from oid 48 from oid 18 current snap is 20 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:209: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:42.713 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:42.765 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:42.820 INFO:tasks.rados.rados.0.trial178.stdout:211: finishing write tid 1 to trial17818531-9 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: finishing write tid 2 to trial17818531-9 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: finishing write tid 3 to trial17818531-9 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: finishing write tid 4 to trial17818531-9 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: finishing write tid 5 to trial17818531-9 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: oid 9 updating version 0 to 50 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: oid 9 version 50 is already newer than 49 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: oid 9 updating version 50 to 51 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: oid 9 version 51 is already newer than 48 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 51 (ObjNum 95 snap 20 seq_num 95) dirty exists 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:211: left oid 9 (ObjNum 95 snap 20 seq_num 95) 2026-02-21T22:16:42.821 INFO:tasks.rados.rados.0.trial178.stdout:212: expect (ObjNum 43 snap 0 seq_num 43) 2026-02-21T22:16:42.846 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1886643 2026-02-21T22:16:42.851 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:42.870 INFO:tasks.rados.rados.0.trial178.stdout:213: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:42.881 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:42.914 INFO:tasks.rados.rados.0.trial178.stdout:209: done (4 left) 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:211: done (3 left) 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:212: done (2 left) 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:213: done (1 left) 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:215: rollback oid 36 current snap is 20 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 19 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:216: copy_from oid 35 from oid 47 current snap is 20 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:217: read oid 39 snap 19 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:217: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:16:42.915 INFO:tasks.rados.rados.0.trial178.stdout:218: snap_create 2026-02-21T22:16:42.919 INFO:tasks.rados.rados.0.trial178.stdout:215: finishing rollback tid 0 to trial17818531-36 2026-02-21T22:16:42.919 INFO:tasks.rados.rados.0.trial178.stdout:215: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:16:42.919 INFO:tasks.rados.rados.0.trial178.stdout:215: finishing rollback tid 2 to trial17818531-36 2026-02-21T22:16:42.919 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 29 (ObjNum 54 snap 2 seq_num 54) dirty exists 2026-02-21T22:16:42.920 INFO:tasks.rados.rados.0.trial178.stdout:217: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:16:42.985 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 230503 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:214: finishing copy_from racing read to trial17818531-48 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:214: finishing copy_from to trial17818531-48 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 22 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:216: finishing copy_from racing read to trial17818531-35 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:216: finishing copy_from to trial17818531-35 2026-02-21T22:16:43.016 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 24 (ObjNum 63 snap 4 seq_num 63) dirty exists 2026-02-21T22:16:43.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:43.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:43.694 INFO:tasks.rados.rados.0.trial178.stdout:214: done (4 left) 2026-02-21T22:16:43.694 INFO:tasks.rados.rados.0.trial178.stdout:215: done (3 left) 2026-02-21T22:16:43.694 INFO:tasks.rados.rados.0.trial178.stdout:216: done (2 left) 2026-02-21T22:16:43.694 INFO:tasks.rados.rados.0.trial178.stdout:217: done (1 left) 2026-02-21T22:16:43.694 INFO:tasks.rados.rados.0.trial178.stdout:218: done (0 left) 2026-02-21T22:16:43.695 INFO:tasks.rados.rados.0.trial178.stdout:219: copy_from oid 40 from oid 15 current snap is 21 2026-02-21T22:16:43.695 INFO:tasks.rados.rados.0.trial178.stdout:220: write oid 8 current snap is 21 2026-02-21T22:16:43.695 INFO:tasks.rados.rados.0.trial178.stdout:220: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2026-02-21T22:16:43.700 INFO:tasks.rados.rados.0.trial178.stdout:220: writing trial17818531-8 from 413472 to 1033612 tid 1 2026-02-21T22:16:43.709 INFO:tasks.rados.rados.0.trial178.stdout:220: writing trial17818531-8 from 1818036 to 2328736 tid 2 2026-02-21T22:16:43.718 INFO:tasks.rados.rados.0.trial178.stdout:220: writing trial17818531-8 from 3120019 to 3613472 tid 3 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:221: snap_create 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:220: finishing write tid 1 to trial17818531-8 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:219: finishing copy_from racing read to trial17818531-40 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:219: finishing copy_from to trial17818531-40 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 39 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:43.719 INFO:tasks.rados.rados.0.trial178.stdout:220: finishing write tid 2 to trial17818531-8 2026-02-21T22:16:43.720 INFO:tasks.rados.rados.0.trial178.stdout:220: finishing write tid 3 to trial17818531-8 2026-02-21T22:16:43.720 INFO:tasks.rados.rados.0.trial178.stdout:220: finishing write tid 4 to trial17818531-8 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: finishing write tid 5 to trial17818531-8 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: oid 8 updating version 0 to 47 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: oid 8 updating version 47 to 49 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: oid 8 version 49 is already newer than 48 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: oid 8 updating version 49 to 50 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 50 (ObjNum 96 snap 21 seq_num 96) dirty exists 2026-02-21T22:16:43.721 INFO:tasks.rados.rados.0.trial178.stdout:220: left oid 8 (ObjNum 96 snap 21 seq_num 96) 2026-02-21T22:16:44.696 INFO:tasks.rados.rados.0.trial178.stdout:219: done (2 left) 2026-02-21T22:16:44.696 INFO:tasks.rados.rados.0.trial178.stdout:220: done (1 left) 2026-02-21T22:16:44.696 INFO:tasks.rados.rados.0.trial178.stdout:221: done (0 left) 2026-02-21T22:16:44.696 INFO:tasks.rados.rados.0.trial178.stdout:222: copy_from oid 23 from oid 42 current snap is 22 2026-02-21T22:16:44.696 INFO:tasks.rados.rados.0.trial178.stdout:223: snap_create 2026-02-21T22:16:44.708 INFO:tasks.rados.rados.0.trial178.stdout:222: finishing copy_from racing read to trial17818531-23 2026-02-21T22:16:44.709 INFO:tasks.rados.rados.0.trial178.stdout:222: finishing copy_from to trial17818531-23 2026-02-21T22:16:44.709 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 18 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-21T22:16:45.686 INFO:tasks.rados.rados.0.trial178.stdout:222: done (1 left) 2026-02-21T22:16:45.686 INFO:tasks.rados.rados.0.trial178.stdout:223: done (0 left) 2026-02-21T22:16:45.686 INFO:tasks.rados.rados.0.trial178.stdout:224: delete oid 11 current snap is 23 2026-02-21T22:16:45.689 INFO:tasks.rados.rados.0.trial178.stdout:224: done (0 left) 2026-02-21T22:16:45.689 INFO:tasks.rados.rados.0.trial178.stdout:225: write oid 2 current snap is 23 2026-02-21T22:16:45.689 INFO:tasks.rados.rados.0.trial178.stdout:225: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2026-02-21T22:16:45.696 INFO:tasks.rados.rados.0.trial178.stdout:225: writing trial17818531-2 from 447086 to 1217724 tid 1 2026-02-21T22:16:45.707 INFO:tasks.rados.rados.0.trial178.stdout:225: writing trial17818531-2 from 1935000 to 2733016 tid 2 2026-02-21T22:16:45.713 INFO:tasks.rados.rados.0.trial178.stdout:225: writing trial17818531-2 from 3458512 to 3647086 tid 3 2026-02-21T22:16:45.713 INFO:tasks.rados.rados.0.trial178.stdout:226: copy_from oid 26 from oid 15 current snap is 23 2026-02-21T22:16:45.713 INFO:tasks.rados.rados.0.trial178.stdout:225: finishing write tid 1 to trial17818531-2 2026-02-21T22:16:45.713 INFO:tasks.rados.rados.0.trial178.stdout:225: finishing write tid 2 to trial17818531-2 2026-02-21T22:16:45.713 INFO:tasks.rados.rados.0.trial178.stdout:227: write (excl) oid 25 current snap is 23 2026-02-21T22:16:45.714 INFO:tasks.rados.rados.0.trial178.stdout:227: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2026-02-21T22:16:45.720 INFO:tasks.rados.rados.0.trial178.stdout:227: writing trial17818531-25 from 463893 to 1026133 tid 1 2026-02-21T22:16:45.728 INFO:tasks.rados.rados.0.trial178.stdout:227: writing trial17818531-25 from 1593482 to 2251509 tid 2 2026-02-21T22:16:45.734 INFO:tasks.rados.rados.0.trial178.stdout:227: writing trial17818531-25 from 2685935 to 3245276 tid 3 2026-02-21T22:16:45.734 INFO:tasks.rados.rados.0.trial178.stdout:228: snap_create 2026-02-21T22:16:45.734 INFO:tasks.rados.rados.0.trial178.stdout:225: finishing write tid 3 to trial17818531-2 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: finishing write tid 4 to trial17818531-2 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: finishing write tid 5 to trial17818531-2 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: oid 2 updating version 0 to 49 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: oid 2 updating version 49 to 50 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: oid 2 updating version 50 to 52 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: oid 2 version 52 is already newer than 51 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 52 (ObjNum 98 snap 23 seq_num 98) dirty exists 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:225: left oid 2 (ObjNum 98 snap 23 seq_num 98) 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:226: finishing copy_from racing read to trial17818531-26 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:226: finishing copy_from to trial17818531-26 2026-02-21T22:16:45.735 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 39 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:45.736 INFO:tasks.rados.rados.0.trial178.stdout:227: finishing write tid 1 to trial17818531-25 2026-02-21T22:16:45.736 INFO:tasks.rados.rados.0.trial178.stdout:227: finishing write tid 2 to trial17818531-25 2026-02-21T22:16:45.736 INFO:tasks.rados.rados.0.trial178.stdout:227: finishing write tid 3 to trial17818531-25 2026-02-21T22:16:45.736 INFO:tasks.rados.rados.0.trial178.stdout:227: finishing write tid 4 to trial17818531-25 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: finishing write tid 5 to trial17818531-25 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: oid 25 updating version 0 to 39 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: oid 25 version 39 is already newer than 37 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: oid 25 updating version 39 to 40 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: oid 25 version 40 is already newer than 38 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 40 (ObjNum 99 snap 23 seq_num 99) dirty exists 2026-02-21T22:16:45.737 INFO:tasks.rados.rados.0.trial178.stdout:227: left oid 25 (ObjNum 99 snap 23 seq_num 99) 2026-02-21T22:16:46.699 INFO:tasks.rados.rados.0.trial178.stdout:225: done (3 left) 2026-02-21T22:16:46.700 INFO:tasks.rados.rados.0.trial178.stdout:226: done (2 left) 2026-02-21T22:16:46.700 INFO:tasks.rados.rados.0.trial178.stdout:227: done (1 left) 2026-02-21T22:16:46.700 INFO:tasks.rados.rados.0.trial178.stdout:228: done (0 left) 2026-02-21T22:16:46.700 INFO:tasks.rados.rados.0.trial178.stdout:229: delete oid 34 current snap is 24 2026-02-21T22:16:46.703 INFO:tasks.rados.rados.0.trial178.stdout:229: done (0 left) 2026-02-21T22:16:46.704 INFO:tasks.rados.rados.0.trial178.stdout:230: rollback oid 37 current snap is 24 2026-02-21T22:16:46.704 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 20 2026-02-21T22:16:46.704 INFO:tasks.rados.rados.0.trial178.stdout:231: write oid 3 current snap is 24 2026-02-21T22:16:46.704 INFO:tasks.rados.rados.0.trial178.stdout:231: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2026-02-21T22:16:46.711 INFO:tasks.rados.rados.0.trial178.stdout:231: writing trial17818531-3 from 497507 to 1210245 tid 1 2026-02-21T22:16:46.719 INFO:tasks.rados.rados.0.trial178.stdout:231: writing trial17818531-3 from 1710446 to 2372142 tid 2 2026-02-21T22:16:46.728 INFO:tasks.rados.rados.0.trial178.stdout:231: writing trial17818531-3 from 3140781 to 3697507 tid 3 2026-02-21T22:16:46.729 INFO:tasks.rados.rados.0.trial178.stdout:232: write (excl) oid 8 current snap is 24 2026-02-21T22:16:46.729 INFO:tasks.rados.rados.0.trial178.stdout:232: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2026-02-21T22:16:46.737 INFO:tasks.rados.rados.0.trial178.stdout:232: writing trial17818531-8 from 514314 to 1302301 tid 1 2026-02-21T22:16:46.743 INFO:tasks.rados.rados.0.trial178.stdout:232: writing trial17818531-8 from 1768928 to 2174282 tid 2 2026-02-21T22:16:46.750 INFO:tasks.rados.rados.0.trial178.stdout:232: writing trial17818531-8 from 2768204 to 3277714 tid 3 2026-02-21T22:16:46.750 INFO:tasks.rados.rados.0.trial178.stdout:233: snap_create 2026-02-21T22:16:46.750 INFO:tasks.rados.rados.0.trial178.stdout:230: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:16:46.750 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 0 (ObjNum -1409409824 snap 32564 seq_num 304) dirty dne 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: finishing write tid 1 to trial17818531-3 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: finishing write tid 2 to trial17818531-3 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: finishing write tid 3 to trial17818531-3 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: finishing write tid 4 to trial17818531-3 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: finishing write tid 5 to trial17818531-3 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: oid 3 updating version 0 to 98 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: oid 3 updating version 98 to 99 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: oid 3 version 99 is already newer than 96 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: oid 3 version 99 is already newer than 97 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 99 (ObjNum 101 snap 24 seq_num 101) dirty exists 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:231: left oid 3 (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:16:46.751 INFO:tasks.rados.rados.0.trial178.stdout:232: finishing write tid 1 to trial17818531-8 2026-02-21T22:16:46.752 INFO:tasks.rados.rados.0.trial178.stdout:232: finishing write tid 2 to trial17818531-8 2026-02-21T22:16:46.752 INFO:tasks.rados.rados.0.trial178.stdout:232: finishing write tid 3 to trial17818531-8 2026-02-21T22:16:46.752 INFO:tasks.rados.rados.0.trial178.stdout:232: finishing write tid 4 to trial17818531-8 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: finishing write tid 5 to trial17818531-8 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: oid 8 updating version 0 to 55 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: oid 8 version 55 is already newer than 54 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: oid 8 version 55 is already newer than 52 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: oid 8 version 55 is already newer than 53 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 55 (ObjNum 102 snap 24 seq_num 102) dirty exists 2026-02-21T22:16:46.753 INFO:tasks.rados.rados.0.trial178.stdout:232: left oid 8 (ObjNum 102 snap 24 seq_num 102) 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:230: done (3 left) 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:231: done (2 left) 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:232: done (1 left) 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:233: done (0 left) 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:234: write oid 28 current snap is 25 2026-02-21T22:16:47.705 INFO:tasks.rados.rados.0.trial178.stdout:234: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2026-02-21T22:16:47.708 INFO:tasks.rados.rados.0.trial178.stdout:234: writing trial17818531-28 from 531121 to 994357 tid 1 2026-02-21T22:16:47.714 INFO:tasks.rados.rados.0.trial178.stdout:234: writing trial17818531-28 from 1711057 to 2376422 tid 2 2026-02-21T22:16:47.719 INFO:tasks.rados.rados.0.trial178.stdout:234: writing trial17818531-28 from 3079274 to 3731121 tid 3 2026-02-21T22:16:47.719 INFO:tasks.rados.rados.0.trial178.stdout:235: read oid 36 snap 24 2026-02-21T22:16:47.719 INFO:tasks.rados.rados.0.trial178.stdout:235: expect (ObjNum 54 snap 2 seq_num 54) 2026-02-21T22:16:47.720 INFO:tasks.rados.rados.0.trial178.stdout:236: write oid 29 current snap is 25 2026-02-21T22:16:47.720 INFO:tasks.rados.rados.0.trial178.stdout:236: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2026-02-21T22:16:47.723 INFO:tasks.rados.rados.0.trial178.stdout:236: writing trial17818531-29 from 547928 to 1086413 tid 1 2026-02-21T22:16:47.728 INFO:tasks.rados.rados.0.trial178.stdout:236: writing trial17818531-29 from 1769539 to 2178562 tid 2 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:236: writing trial17818531-29 from 2706697 to 3238751 tid 3 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:237: rollback oid 30 current snap is 25 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 30 to 19 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:234: finishing write tid 1 to trial17818531-28 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:234: finishing write tid 2 to trial17818531-28 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:234: finishing write tid 3 to trial17818531-28 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:234: finishing write tid 4 to trial17818531-28 2026-02-21T22:16:47.733 INFO:tasks.rados.rados.0.trial178.stdout:234: finishing write tid 5 to trial17818531-28 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:234: oid 28 updating version 0 to 52 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:234: oid 28 updating version 52 to 54 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:234: oid 28 version 54 is already newer than 53 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:234: oid 28 version 54 is already newer than 51 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 54 (ObjNum 103 snap 25 seq_num 103) dirty exists 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:234: left oid 28 (ObjNum 103 snap 25 seq_num 103) 2026-02-21T22:16:47.734 INFO:tasks.rados.rados.0.trial178.stdout:235: expect (ObjNum 54 snap 2 seq_num 54) 2026-02-21T22:16:47.781 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 545795 2026-02-21T22:16:47.827 INFO:tasks.rados.rados.0.trial178.stdout:236: finishing write tid 1 to trial17818531-29 2026-02-21T22:16:47.827 INFO:tasks.rados.rados.0.trial178.stdout:236: finishing write tid 2 to trial17818531-29 2026-02-21T22:16:47.827 INFO:tasks.rados.rados.0.trial178.stdout:237: finishing rollback tid 0 to trial17818531-30 2026-02-21T22:16:47.827 INFO:tasks.rados.rados.0.trial178.stdout:234: done (3 left) 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:235: done (2 left) 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:238: read oid 7 snap -1 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:238: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:239: delete oid 26 current snap is 25 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:237: finishing rollback tid 1 to trial17818531-30 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:236: finishing write tid 3 to trial17818531-29 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:237: finishing rollback tid 2 to trial17818531-30 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 38 (ObjNum 29 snap 0 seq_num 29) dirty exists 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:236: finishing write tid 4 to trial17818531-29 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:236: finishing write tid 5 to trial17818531-29 2026-02-21T22:16:47.828 INFO:tasks.rados.rados.0.trial178.stdout:236: oid 29 updating version 0 to 34 2026-02-21T22:16:47.829 INFO:tasks.rados.rados.0.trial178.stdout:236: oid 29 version 34 is already newer than 33 2026-02-21T22:16:47.829 INFO:tasks.rados.rados.0.trial178.stdout:236: oid 29 updating version 34 to 36 2026-02-21T22:16:47.829 INFO:tasks.rados.rados.0.trial178.stdout:236: oid 29 version 36 is already newer than 35 2026-02-21T22:16:47.829 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:16:47.829 INFO:tasks.rados.rados.0.trial178.stdout:236: left oid 29 (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:16:47.830 INFO:tasks.rados.rados.0.trial178.stdout:236: done (3 left) 2026-02-21T22:16:47.830 INFO:tasks.rados.rados.0.trial178.stdout:237: done (2 left) 2026-02-21T22:16:47.830 INFO:tasks.rados.rados.0.trial178.stdout:239: done (1 left) 2026-02-21T22:16:47.830 INFO:tasks.rados.rados.0.trial178.stdout:240: copy_from oid 27 from oid 15 current snap is 25 2026-02-21T22:16:47.830 INFO:tasks.rados.rados.0.trial178.stdout:241: snap_create 2026-02-21T22:16:47.831 INFO:tasks.rados.rados.0.trial178.stdout:238: expect (ObjNum 6 snap 0 seq_num 6) 2026-02-21T22:16:47.860 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:47.885 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1227314 2026-02-21T22:16:47.894 INFO:tasks.rados.rados.0.trial178.stdout:240: finishing copy_from racing read to trial17818531-27 2026-02-21T22:16:47.894 INFO:tasks.rados.rados.0.trial178.stdout:240: finishing copy_from to trial17818531-27 2026-02-21T22:16:47.894 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 39 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:48.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:48.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:48.706 INFO:tasks.rados.rados.0.trial178.stdout:238: done (2 left) 2026-02-21T22:16:48.706 INFO:tasks.rados.rados.0.trial178.stdout:240: done (1 left) 2026-02-21T22:16:48.706 INFO:tasks.rados.rados.0.trial178.stdout:241: done (0 left) 2026-02-21T22:16:48.706 INFO:tasks.rados.rados.0.trial178.stdout:242: delete oid 45 current snap is 26 2026-02-21T22:16:48.709 INFO:tasks.rados.rados.0.trial178.stdout:242: done (0 left) 2026-02-21T22:16:48.709 INFO:tasks.rados.rados.0.trial178.stdout:243: write oid 11 current snap is 26 2026-02-21T22:16:48.709 INFO:tasks.rados.rados.0.trial178.stdout:243: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2026-02-21T22:16:48.716 INFO:tasks.rados.rados.0.trial178.stdout:243: writing trial17818531-11 from 598349 to 1078934 tid 1 2026-02-21T22:16:48.724 INFO:tasks.rados.rados.0.trial178.stdout:243: writing trial17818531-11 from 1544985 to 2217688 tid 2 2026-02-21T22:16:48.732 INFO:tasks.rados.rados.0.trial178.stdout:243: writing trial17818531-11 from 2905319 to 3387542 tid 3 2026-02-21T22:16:48.732 INFO:tasks.rados.rados.0.trial178.stdout:244: snap_create 2026-02-21T22:16:48.732 INFO:tasks.rados.rados.0.trial178.stdout:243: finishing write tid 1 to trial17818531-11 2026-02-21T22:16:48.732 INFO:tasks.rados.rados.0.trial178.stdout:243: finishing write tid 2 to trial17818531-11 2026-02-21T22:16:48.734 INFO:tasks.rados.rados.0.trial178.stdout:243: finishing write tid 3 to trial17818531-11 2026-02-21T22:16:48.734 INFO:tasks.rados.rados.0.trial178.stdout:243: finishing write tid 4 to trial17818531-11 2026-02-21T22:16:48.734 INFO:tasks.rados.rados.0.trial178.stdout:243: finishing write tid 5 to trial17818531-11 2026-02-21T22:16:48.734 INFO:tasks.rados.rados.0.trial178.stdout:243: oid 11 updating version 0 to 102 2026-02-21T22:16:48.734 INFO:tasks.rados.rados.0.trial178.stdout:243: oid 11 version 102 is already newer than 100 2026-02-21T22:16:48.735 INFO:tasks.rados.rados.0.trial178.stdout:243: oid 11 version 102 is already newer than 101 2026-02-21T22:16:48.735 INFO:tasks.rados.rados.0.trial178.stdout:243: oid 11 updating version 102 to 103 2026-02-21T22:16:48.735 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 103 (ObjNum 107 snap 26 seq_num 107) dirty exists 2026-02-21T22:16:48.735 INFO:tasks.rados.rados.0.trial178.stdout:243: left oid 11 (ObjNum 107 snap 26 seq_num 107) 2026-02-21T22:16:49.711 INFO:tasks.rados.rados.0.trial178.stdout:243: done (1 left) 2026-02-21T22:16:49.711 INFO:tasks.rados.rados.0.trial178.stdout:244: done (0 left) 2026-02-21T22:16:49.712 INFO:tasks.rados.rados.0.trial178.stdout:245: delete oid 2 current snap is 27 2026-02-21T22:16:49.844 INFO:tasks.rados.rados.0.trial178.stdout:245: done (0 left) 2026-02-21T22:16:49.845 INFO:tasks.rados.rados.0.trial178.stdout:246: write (excl) oid 7 current snap is 27 2026-02-21T22:16:49.845 INFO:tasks.rados.rados.0.trial178.stdout:246: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2026-02-21T22:16:49.852 INFO:tasks.rados.rados.0.trial178.stdout:246: writing trial17818531-7 from 631963 to 1263046 tid 1 2026-02-21T22:16:49.862 INFO:tasks.rados.rados.0.trial178.stdout:246: writing trial17818531-7 from 2061949 to 2621968 tid 2 2026-02-21T22:16:49.870 INFO:tasks.rados.rados.0.trial178.stdout:246: writing trial17818531-7 from 3243812 to 3748579 tid 3 2026-02-21T22:16:49.870 INFO:tasks.rados.rados.0.trial178.stdout:247: snap_remove snap 20 2026-02-21T22:16:50.687 INFO:tasks.rados.rados.0.trial178.stdout:247: done (1 left) 2026-02-21T22:16:50.688 INFO:tasks.rados.rados.0.trial178.stdout:248: write oid 35 current snap is 27 2026-02-21T22:16:50.688 INFO:tasks.rados.rados.0.trial178.stdout:248: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2026-02-21T22:16:50.696 INFO:tasks.rados.rados.0.trial178.stdout:248: writing trial17818531-35 from 648770 to 1355102 tid 1 2026-02-21T22:16:50.704 INFO:tasks.rados.rados.0.trial178.stdout:248: writing trial17818531-35 from 2120431 to 2540461 tid 2 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:248: writing trial17818531-35 from 3271235 to 3703627 tid 3 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:249: rollback oid 15 current snap is 27 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 21 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:246: finishing write tid 1 to trial17818531-7 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:246: finishing write tid 2 to trial17818531-7 2026-02-21T22:16:50.713 INFO:tasks.rados.rados.0.trial178.stdout:246: finishing write tid 3 to trial17818531-7 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: finishing write tid 4 to trial17818531-7 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: finishing write tid 5 to trial17818531-7 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: oid 7 updating version 0 to 57 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: oid 7 updating version 57 to 60 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: oid 7 version 60 is already newer than 59 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: oid 7 version 60 is already newer than 58 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 60 (ObjNum 109 snap 27 seq_num 109) dirty exists 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: left oid 7 (ObjNum 109 snap 27 seq_num 109) 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:248: finishing write tid 1 to trial17818531-35 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:248: finishing write tid 2 to trial17818531-35 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:246: done (2 left) 2026-02-21T22:16:50.714 INFO:tasks.rados.rados.0.trial178.stdout:250: write oid 24 current snap is 27 2026-02-21T22:16:50.715 INFO:tasks.rados.rados.0.trial178.stdout:250: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2026-02-21T22:16:50.722 INFO:tasks.rados.rados.0.trial178.stdout:250: writing trial17818531-24 from 665577 to 1447158 tid 1 2026-02-21T22:16:50.730 INFO:tasks.rados.rados.0.trial178.stdout:250: writing trial17818531-24 from 2062560 to 2626248 tid 2 2026-02-21T22:16:50.738 INFO:tasks.rados.rados.0.trial178.stdout:250: writing trial17818531-24 from 3182305 to 3825969 tid 3 2026-02-21T22:16:50.738 INFO:tasks.rados.rados.0.trial178.stdout:251: read oid 6 snap -1 2026-02-21T22:16:50.738 INFO:tasks.rados.rados.0.trial178.stdout:251: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:50.738 INFO:tasks.rados.rados.0.trial178.stdout:252: write oid 23 current snap is 27 2026-02-21T22:16:50.738 INFO:tasks.rados.rados.0.trial178.stdout:252: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2026-02-21T22:16:50.742 INFO:tasks.rados.rados.0.trial178.stdout:252: writing trial17818531-23 from 682384 to 1139214 tid 1 2026-02-21T22:16:50.746 INFO:tasks.rados.rados.0.trial178.stdout:252: writing trial17818531-23 from 1721042 to 2144741 tid 2 2026-02-21T22:16:50.750 INFO:tasks.rados.rados.0.trial178.stdout:252: writing trial17818531-23 from 2809728 to 3264664 tid 3 2026-02-21T22:16:50.752 INFO:tasks.rados.rados.0.trial178.stdout:252: writing trial17818531-23 from 3731357 to 3882384 tid 4 2026-02-21T22:16:50.752 INFO:tasks.rados.rados.0.trial178.stdout:253: read oid 21 snap -1 2026-02-21T22:16:50.752 INFO:tasks.rados.rados.0.trial178.stdout:253: expect deleted 2026-02-21T22:16:50.753 INFO:tasks.rados.rados.0.trial178.stdout:254: snap_remove snap 25 2026-02-21T22:16:51.715 INFO:tasks.rados.rados.0.trial178.stdout:254: done (6 left) 2026-02-21T22:16:51.715 INFO:tasks.rados.rados.0.trial178.stdout:255: write oid 18 current snap is 27 2026-02-21T22:16:51.715 INFO:tasks.rados.rados.0.trial178.stdout:255: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2026-02-21T22:16:51.720 INFO:tasks.rados.rados.0.trial178.stdout:255: writing trial17818531-18 from 699191 to 1231270 tid 1 2026-02-21T22:16:51.724 INFO:tasks.rados.rados.0.trial178.stdout:255: writing trial17818531-18 from 1779524 to 2346881 tid 2 2026-02-21T22:16:51.729 INFO:tasks.rados.rados.0.trial178.stdout:255: writing trial17818531-18 from 2837151 to 3503359 tid 3 2026-02-21T22:16:51.729 INFO:tasks.rados.rados.0.trial178.stdout:256: read oid 26 snap 21 2026-02-21T22:16:51.729 INFO:tasks.rados.rados.0.trial178.stdout:256: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:51.729 INFO:tasks.rados.rados.0.trial178.stdout:257: copy_from oid 30 from oid 32 current snap is 27 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: finishing write tid 3 to trial17818531-35 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: finishing write tid 4 to trial17818531-35 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: finishing write tid 5 to trial17818531-35 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: oid 35 updating version 0 to 107 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: oid 35 version 107 is already newer than 105 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: oid 35 updating version 107 to 108 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: oid 35 version 108 is already newer than 106 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 108 (ObjNum 110 snap 27 seq_num 110) dirty exists 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:248: left oid 35 (ObjNum 110 snap 27 seq_num 110) 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:249: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:16:51.730 INFO:tasks.rados.rados.0.trial178.stdout:249: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:249: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 47 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:250: finishing write tid 1 to trial17818531-24 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:250: finishing write tid 2 to trial17818531-24 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:251: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:51.731 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 175641 2026-02-21T22:16:51.758 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:51.797 INFO:tasks.rados.rados.0.trial178.stdout:250: finishing write tid 3 to trial17818531-24 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: finishing write tid 4 to trial17818531-24 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: finishing write tid 5 to trial17818531-24 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: oid 24 updating version 0 to 69 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: oid 24 version 69 is already newer than 66 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: oid 24 version 69 is already newer than 68 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: oid 24 version 69 is already newer than 67 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 69 (ObjNum 111 snap 27 seq_num 111) dirty exists 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:250: left oid 24 (ObjNum 111 snap 27 seq_num 111) 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 1 to trial17818531-23 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 2 to trial17818531-23 2026-02-21T22:16:51.798 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 3 to trial17818531-23 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 4 to trial17818531-23 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 5 to trial17818531-23 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: finishing write tid 6 to trial17818531-23 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: oid 23 updating version 0 to 25 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: oid 23 updating version 25 to 28 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: oid 23 version 28 is already newer than 26 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: oid 23 version 28 is already newer than 27 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: oid 23 version 28 is already newer than 24 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 28 (ObjNum 112 snap 27 seq_num 112) dirty exists 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:252: left oid 23 (ObjNum 112 snap 27 seq_num 112) 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:255: finishing write tid 1 to trial17818531-18 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:255: finishing write tid 2 to trial17818531-18 2026-02-21T22:16:51.799 INFO:tasks.rados.rados.0.trial178.stdout:255: finishing write tid 3 to trial17818531-18 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: finishing write tid 4 to trial17818531-18 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: finishing write tid 5 to trial17818531-18 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: oid 18 updating version 0 to 45 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: oid 18 version 45 is already newer than 42 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: oid 18 version 45 is already newer than 43 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: oid 18 version 45 is already newer than 44 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 45 (ObjNum 113 snap 27 seq_num 113) dirty exists 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:255: left oid 18 (ObjNum 113 snap 27 seq_num 113) 2026-02-21T22:16:51.800 INFO:tasks.rados.rados.0.trial178.stdout:256: expect (ObjNum 53 snap 1 seq_num 53) 2026-02-21T22:16:51.813 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2461573 2026-02-21T22:16:51.818 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:51.849 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 309427 2026-02-21T22:16:51.877 INFO:tasks.rados.rados.0.trial178.stdout:257: finishing copy_from racing read to trial17818531-30 2026-02-21T22:16:51.877 INFO:tasks.rados.rados.0.trial178.stdout:257: finishing copy_from to trial17818531-30 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 85 (ObjNum 91 snap 17 seq_num 91) dirty exists 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:248: done (8 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:249: done (7 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:250: done (6 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:251: done (5 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:252: done (4 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:253: done (3 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:255: done (2 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:256: done (1 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:257: done (0 left) 2026-02-21T22:16:51.878 INFO:tasks.rados.rados.0.trial178.stdout:258: write (excl) oid 11 current snap is 27 2026-02-21T22:16:51.879 INFO:tasks.rados.rados.0.trial178.stdout:258: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2026-02-21T22:16:51.886 INFO:tasks.rados.rados.0.trial178.stdout:258: writing trial17818531-11 from 715998 to 1323326 tid 1 2026-02-21T22:16:51.893 INFO:tasks.rados.rados.0.trial178.stdout:258: writing trial17818531-11 from 1838006 to 2265374 tid 2 2026-02-21T22:16:51.900 INFO:tasks.rados.rados.0.trial178.stdout:258: writing trial17818531-11 from 2864574 to 3342054 tid 3 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:258: writing trial17818531-11 from 3863835 to 3915998 tid 4 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:259: copy_from oid 19 from oid 9 current snap is 27 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 1 to trial17818531-11 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 2 to trial17818531-11 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 3 to trial17818531-11 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:260: read oid 50 snap 26 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:260: expect deleted 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:261: write oid 10 current snap is 27 2026-02-21T22:16:51.904 INFO:tasks.rados.rados.0.trial178.stdout:261: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2026-02-21T22:16:51.916 INFO:tasks.rados.rados.0.trial178.stdout:261: writing trial17818531-10 from 732805 to 1531735 tid 1 2026-02-21T22:16:51.925 INFO:tasks.rados.rados.0.trial178.stdout:261: writing trial17818531-10 from 2296488 to 2867514 tid 2 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:261: writing trial17818531-10 from 3291997 to 3697102 tid 3 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:262: snap_create 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 4 to trial17818531-11 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 5 to trial17818531-11 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: finishing write tid 6 to trial17818531-11 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: oid 11 updating version 0 to 116 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: oid 11 version 116 is already newer than 114 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: oid 11 version 116 is already newer than 113 2026-02-21T22:16:51.929 INFO:tasks.rados.rados.0.trial178.stdout:258: oid 11 version 116 is already newer than 115 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:258: oid 11 version 116 is already newer than 112 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 116 (ObjNum 114 snap 27 seq_num 114) dirty exists 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:258: left oid 11 (ObjNum 114 snap 27 seq_num 114) 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:259: finishing copy_from racing read to trial17818531-19 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:259: finishing copy_from to trial17818531-19 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 51 (ObjNum 95 snap 20 seq_num 95) dirty exists 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:261: finishing write tid 1 to trial17818531-10 2026-02-21T22:16:51.930 INFO:tasks.rados.rados.0.trial178.stdout:261: finishing write tid 2 to trial17818531-10 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: finishing write tid 3 to trial17818531-10 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: finishing write tid 4 to trial17818531-10 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: finishing write tid 5 to trial17818531-10 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: oid 10 updating version 0 to 67 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: oid 10 updating version 67 to 68 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: oid 10 version 68 is already newer than 66 2026-02-21T22:16:51.931 INFO:tasks.rados.rados.0.trial178.stdout:261: oid 10 updating version 68 to 69 2026-02-21T22:16:51.932 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 69 (ObjNum 115 snap 27 seq_num 115) dirty exists 2026-02-21T22:16:51.932 INFO:tasks.rados.rados.0.trial178.stdout:261: left oid 10 (ObjNum 115 snap 27 seq_num 115) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:258: done (4 left) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:259: done (3 left) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:260: done (2 left) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:261: done (1 left) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:262: done (0 left) 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:263: read oid 12 snap -1 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:263: expect deleted 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:264: read oid 33 snap -1 2026-02-21T22:16:52.720 INFO:tasks.rados.rados.0.trial178.stdout:264: expect deleted 2026-02-21T22:16:52.721 INFO:tasks.rados.rados.0.trial178.stdout:265: read oid 14 snap -1 2026-02-21T22:16:52.721 INFO:tasks.rados.rados.0.trial178.stdout:265: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:16:52.721 INFO:tasks.rados.rados.0.trial178.stdout:266: read oid 37 snap -1 2026-02-21T22:16:52.721 INFO:tasks.rados.rados.0.trial178.stdout:266: expect deleted 2026-02-21T22:16:52.721 INFO:tasks.rados.rados.0.trial178.stdout:267: snap_remove snap 19 2026-02-21T22:16:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:53.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:53.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:16:53.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:16:53.723 INFO:tasks.rados.rados.0.trial178.stdout:267: done (4 left) 2026-02-21T22:16:53.723 INFO:tasks.rados.rados.0.trial178.stdout:268: read oid 35 snap -1 2026-02-21T22:16:53.723 INFO:tasks.rados.rados.0.trial178.stdout:268: expect (ObjNum 110 snap 27 seq_num 110) 2026-02-21T22:16:53.723 INFO:tasks.rados.rados.0.trial178.stdout:269: snap_create 2026-02-21T22:16:53.723 INFO:tasks.rados.rados.0.trial178.stdout:265: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:16:53.728 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 808243 2026-02-21T22:16:53.822 INFO:tasks.rados.rados.0.trial178.stdout:268: expect (ObjNum 110 snap 27 seq_num 110) 2026-02-21T22:16:54.726 INFO:tasks.rados.rados.0.trial178.stdout:263: done (5 left) 2026-02-21T22:16:54.726 INFO:tasks.rados.rados.0.trial178.stdout:264: done (4 left) 2026-02-21T22:16:54.726 INFO:tasks.rados.rados.0.trial178.stdout:265: done (3 left) 2026-02-21T22:16:54.726 INFO:tasks.rados.rados.0.trial178.stdout:266: done (2 left) 2026-02-21T22:16:54.726 INFO:tasks.rados.rados.0.trial178.stdout:268: done (1 left) 2026-02-21T22:16:54.727 INFO:tasks.rados.rados.0.trial178.stdout:269: done (0 left) 2026-02-21T22:16:54.727 INFO:tasks.rados.rados.0.trial178.stdout:270: read oid 4 snap 21 2026-02-21T22:16:54.727 INFO:tasks.rados.rados.0.trial178.stdout:270: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:54.727 INFO:tasks.rados.rados.0.trial178.stdout:271: snap_remove snap 22 2026-02-21T22:16:55.688 INFO:tasks.rados.rados.0.trial178.stdout:271: done (1 left) 2026-02-21T22:16:55.688 INFO:tasks.rados.rados.0.trial178.stdout:272: snap_remove snap 23 2026-02-21T22:16:56.689 INFO:tasks.rados.rados.0.trial178.stdout:272: done (1 left) 2026-02-21T22:16:56.689 INFO:tasks.rados.rados.0.trial178.stdout:273: snap_remove snap 26 2026-02-21T22:16:57.692 INFO:tasks.rados.rados.0.trial178.stdout:273: done (1 left) 2026-02-21T22:16:57.692 INFO:tasks.rados.rados.0.trial178.stdout:274: write (excl) oid 49 current snap is 29 2026-02-21T22:16:57.693 INFO:tasks.rados.rados.0.trial178.stdout:274: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2026-02-21T22:16:57.700 INFO:tasks.rados.rados.0.trial178.stdout:274: writing trial17818531-49 from 749612 to 1223791 tid 1 2026-02-21T22:16:57.708 INFO:tasks.rados.rados.0.trial178.stdout:274: writing trial17818531-49 from 1954970 to 2386007 tid 2 2026-02-21T22:16:57.716 INFO:tasks.rados.rados.0.trial178.stdout:274: writing trial17818531-49 from 2919420 to 3535797 tid 3 2026-02-21T22:16:57.716 INFO:tasks.rados.rados.0.trial178.stdout:275: read oid 18 snap -1 2026-02-21T22:16:57.716 INFO:tasks.rados.rados.0.trial178.stdout:275: expect (ObjNum 113 snap 27 seq_num 113) 2026-02-21T22:16:57.716 INFO:tasks.rados.rados.0.trial178.stdout:276: write (excl) oid 36 current snap is 29 2026-02-21T22:16:57.716 INFO:tasks.rados.rados.0.trial178.stdout:276: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2026-02-21T22:16:57.725 INFO:tasks.rados.rados.0.trial178.stdout:276: writing trial17818531-36 from 766419 to 1315847 tid 1 2026-02-21T22:16:57.733 INFO:tasks.rados.rados.0.trial178.stdout:276: writing trial17818531-36 from 2013452 to 2588147 tid 2 2026-02-21T22:16:57.739 INFO:tasks.rados.rados.0.trial178.stdout:276: writing trial17818531-36 from 3346843 to 3774492 tid 3 2026-02-21T22:16:57.739 INFO:tasks.rados.rados.0.trial178.stdout:277: read oid 50 snap 28 2026-02-21T22:16:57.740 INFO:tasks.rados.rados.0.trial178.stdout:277: expect deleted 2026-02-21T22:16:57.740 INFO:tasks.rados.rados.0.trial178.stdout:278: write oid 17 current snap is 29 2026-02-21T22:16:57.740 INFO:tasks.rados.rados.0.trial178.stdout:278: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2026-02-21T22:16:57.744 INFO:tasks.rados.rados.0.trial178.stdout:278: writing trial17818531-17 from 783226 to 1407903 tid 1 2026-02-21T22:16:57.748 INFO:tasks.rados.rados.0.trial178.stdout:278: writing trial17818531-17 from 2071934 to 2506640 tid 2 2026-02-21T22:16:57.753 INFO:tasks.rados.rados.0.trial178.stdout:278: writing trial17818531-17 from 2974266 to 3613187 tid 3 2026-02-21T22:16:57.753 INFO:tasks.rados.rados.0.trial178.stdout:279: read oid 22 snap -1 2026-02-21T22:16:57.753 INFO:tasks.rados.rados.0.trial178.stdout:279: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:57.754 INFO:tasks.rados.rados.0.trial178.stdout:280: read oid 39 snap -1 2026-02-21T22:16:57.754 INFO:tasks.rados.rados.0.trial178.stdout:280: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:16:57.754 INFO:tasks.rados.rados.0.trial178.stdout:281: write (excl) oid 44 current snap is 29 2026-02-21T22:16:57.754 INFO:tasks.rados.rados.0.trial178.stdout:281: seq_num 119 ranges {400033=699926,1614063=385970} 2026-02-21T22:16:57.757 INFO:tasks.rados.rados.0.trial178.stdout:281: writing trial17818531-44 from 400033 to 1099959 tid 1 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:281: writing trial17818531-44 from 1614063 to 2000033 tid 2 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:282: read oid 6 snap -1 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:282: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:283: delete oid 9 current snap is 29 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:270: expect (ObjNum 50 snap 0 seq_num 50) 2026-02-21T22:16:57.761 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:57.833 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:57.859 INFO:tasks.rados.rados.0.trial178.stdout:274: finishing write tid 1 to trial17818531-49 2026-02-21T22:16:57.859 INFO:tasks.rados.rados.0.trial178.stdout:274: finishing write tid 2 to trial17818531-49 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: finishing write tid 3 to trial17818531-49 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: finishing write tid 4 to trial17818531-49 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: finishing write tid 5 to trial17818531-49 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: oid 49 updating version 0 to 42 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: oid 49 updating version 42 to 43 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: oid 49 updating version 43 to 45 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: oid 49 version 45 is already newer than 44 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 45 (ObjNum 116 snap 29 seq_num 116) dirty exists 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:274: left oid 49 (ObjNum 116 snap 29 seq_num 116) 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:275: expect (ObjNum 113 snap 27 seq_num 113) 2026-02-21T22:16:57.860 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 235361 2026-02-21T22:16:57.963 INFO:tasks.rados.rados.0.trial178.stdout:276: finishing write tid 1 to trial17818531-36 2026-02-21T22:16:57.963 INFO:tasks.rados.rados.0.trial178.stdout:276: finishing write tid 2 to trial17818531-36 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: finishing write tid 3 to trial17818531-36 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: finishing write tid 4 to trial17818531-36 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: finishing write tid 5 to trial17818531-36 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: oid 36 updating version 0 to 32 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: oid 36 version 32 is already newer than 30 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: oid 36 updating version 32 to 33 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: oid 36 version 33 is already newer than 31 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 33 (ObjNum 117 snap 29 seq_num 117) dirty exists 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:276: left oid 36 (ObjNum 117 snap 29 seq_num 117) 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:278: finishing write tid 1 to trial17818531-17 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:278: finishing write tid 2 to trial17818531-17 2026-02-21T22:16:57.964 INFO:tasks.rados.rados.0.trial178.stdout:278: finishing write tid 3 to trial17818531-17 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:278: finishing write tid 4 to trial17818531-17 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:270: done (10 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:274: done (9 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:275: done (8 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:276: done (7 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:277: done (6 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:283: done (5 left) 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:284: write oid 24 current snap is 29 2026-02-21T22:16:57.965 INFO:tasks.rados.rados.0.trial178.stdout:284: seq_num 121 ranges {433647=450424,1331027=582033} 2026-02-21T22:16:57.969 INFO:tasks.rados.rados.0.trial178.stdout:284: writing trial17818531-24 from 433647 to 884071 tid 1 2026-02-21T22:16:57.976 INFO:tasks.rados.rados.0.trial178.stdout:284: writing trial17818531-24 from 1331027 to 1913060 tid 2 2026-02-21T22:16:57.976 INFO:tasks.rados.rados.0.trial178.stdout:285: snap_remove snap 24 2026-02-21T22:16:58.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:16:58.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:16:58.697 INFO:tasks.rados.rados.0.trial178.stdout:285: done (6 left) 2026-02-21T22:16:58.697 INFO:tasks.rados.rados.0.trial178.stdout:286: write oid 35 current snap is 29 2026-02-21T22:16:58.697 INFO:tasks.rados.rados.0.trial178.stdout:286: seq_num 122 ranges {450454=642026,1505862=544592} 2026-02-21T22:16:58.704 INFO:tasks.rados.rados.0.trial178.stdout:286: writing trial17818531-35 from 450454 to 1092480 tid 1 2026-02-21T22:16:58.711 INFO:tasks.rados.rados.0.trial178.stdout:286: writing trial17818531-35 from 1505862 to 2050454 tid 2 2026-02-21T22:16:58.711 INFO:tasks.rados.rados.0.trial178.stdout:287: copy_from oid 7 from oid 9 current snap is 29 2026-02-21T22:16:58.711 INFO:tasks.rados.rados.0.trial178.stdout:279: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:16:58.718 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1747428 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: finishing write tid 5 to trial17818531-17 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: oid 17 updating version 0 to 62 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: oid 17 version 62 is already newer than 61 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: oid 17 version 62 is already newer than 59 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: oid 17 version 62 is already newer than 60 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 62 (ObjNum 118 snap 29 seq_num 118) dirty exists 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:278: left oid 17 (ObjNum 118 snap 29 seq_num 118) 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:281: finishing write tid 1 to trial17818531-44 2026-02-21T22:16:58.792 INFO:tasks.rados.rados.0.trial178.stdout:280: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:16:58.793 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:58.858 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: finishing write tid 2 to trial17818531-44 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: finishing write tid 3 to trial17818531-44 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: finishing write tid 4 to trial17818531-44 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: oid 44 updating version 0 to 52 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: oid 44 version 52 is already newer than 51 2026-02-21T22:16:58.890 INFO:tasks.rados.rados.0.trial178.stdout:281: oid 44 updating version 52 to 53 2026-02-21T22:16:58.891 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 53 (ObjNum 119 snap 29 seq_num 119) dirty exists 2026-02-21T22:16:58.891 INFO:tasks.rados.rados.0.trial178.stdout:281: left oid 44 (ObjNum 119 snap 29 seq_num 119) 2026-02-21T22:16:58.891 INFO:tasks.rados.rados.0.trial178.stdout:282: expect (ObjNum 16 snap 0 seq_num 16) 2026-02-21T22:16:58.898 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2183275 2026-02-21T22:16:58.901 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1402964 2026-02-21T22:16:58.913 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 958007 2026-02-21T22:16:58.922 INFO:tasks.rados.rados.0.trial178.stdout:284: finishing write tid 1 to trial17818531-24 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: finishing write tid 2 to trial17818531-24 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: finishing write tid 3 to trial17818531-24 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: finishing write tid 4 to trial17818531-24 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: oid 24 updating version 0 to 109 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: oid 24 version 109 is already newer than 108 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: oid 24 version 109 is already newer than 107 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 109 (ObjNum 121 snap 29 seq_num 121) dirty exists 2026-02-21T22:16:58.923 INFO:tasks.rados.rados.0.trial178.stdout:284: left oid 24 (ObjNum 121 snap 29 seq_num 121) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:286: finishing write tid 1 to trial17818531-35 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:287: finishing copy_from to trial17818531-7 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:287: got expected ENOENT (src dne) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:287: finishing copy_from racing read to trial17818531-7 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:286: finishing write tid 2 to trial17818531-35 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:286: finishing write tid 3 to trial17818531-35 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:278: done (7 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:279: done (6 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:280: done (5 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:281: done (4 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:282: done (3 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:284: done (2 left) 2026-02-21T22:16:58.924 INFO:tasks.rados.rados.0.trial178.stdout:287: done (1 left) 2026-02-21T22:16:58.925 INFO:tasks.rados.rados.0.trial178.stdout:288: write (excl) oid 47 current snap is 29 2026-02-21T22:16:58.925 INFO:tasks.rados.rados.0.trial178.stdout:288: seq_num 123 ranges {467261=717275,1847991=219270} 2026-02-21T22:16:58.931 INFO:tasks.rados.rados.0.trial178.stdout:288: writing trial17818531-47 from 467261 to 1184536 tid 1 2026-02-21T22:16:58.937 INFO:tasks.rados.rados.0.trial178.stdout:288: writing trial17818531-47 from 1847991 to 2067261 tid 2 2026-02-21T22:16:58.937 INFO:tasks.rados.rados.0.trial178.stdout:289: read oid 20 snap -1 2026-02-21T22:16:58.937 INFO:tasks.rados.rados.0.trial178.stdout:289: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:16:58.937 INFO:tasks.rados.rados.0.trial178.stdout:290: snap_remove snap 27 2026-02-21T22:16:59.701 INFO:tasks.rados.rados.0.trial178.stdout:290: done (3 left) 2026-02-21T22:16:59.701 INFO:tasks.rados.rados.0.trial178.stdout:291: read oid 3 snap 28 2026-02-21T22:16:59.701 INFO:tasks.rados.rados.0.trial178.stdout:291: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:292: read oid 50 snap -1 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:292: expect deleted 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:293: write oid 32 current snap is 29 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:286: finishing write tid 4 to trial17818531-35 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:286: oid 35 updating version 0 to 129 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:286: oid 35 updating version 129 to 130 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:286: oid 35 version 130 is already newer than 128 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 130 (ObjNum 122 snap 29 seq_num 122) dirty exists 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:286: left oid 35 (ObjNum 122 snap 29 seq_num 122) 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:288: finishing write tid 1 to trial17818531-47 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:288: finishing write tid 2 to trial17818531-47 2026-02-21T22:16:59.702 INFO:tasks.rados.rados.0.trial178.stdout:288: finishing write tid 3 to trial17818531-47 2026-02-21T22:16:59.703 INFO:tasks.rados.rados.0.trial178.stdout:293: seq_num 124 ranges {484068=792524,1906473=177595} 2026-02-21T22:16:59.709 INFO:tasks.rados.rados.0.trial178.stdout:293: writing trial17818531-32 from 484068 to 1276592 tid 1 2026-02-21T22:16:59.715 INFO:tasks.rados.rados.0.trial178.stdout:293: writing trial17818531-32 from 1906473 to 2084068 tid 2 2026-02-21T22:16:59.715 INFO:tasks.rados.rados.0.trial178.stdout:286: done (5 left) 2026-02-21T22:16:59.715 INFO:tasks.rados.rados.0.trial178.stdout:294: rollback oid 1 current snap is 29 2026-02-21T22:16:59.715 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 28 2026-02-21T22:16:59.715 INFO:tasks.rados.rados.0.trial178.stdout:288: finishing write tid 4 to trial17818531-47 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:288: oid 47 updating version 0 to 119 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:288: oid 47 version 119 is already newer than 118 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:288: oid 47 version 119 is already newer than 117 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 119 (ObjNum 123 snap 29 seq_num 123) dirty exists 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:288: left oid 47 (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:16:59.716 INFO:tasks.rados.rados.0.trial178.stdout:289: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:16:59.823 INFO:tasks.rados.rados.0.trial178.stdout:291: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:16:59.823 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:16:59.889 INFO:tasks.rados.rados.0.trial178.stdout:293: finishing write tid 1 to trial17818531-32 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:294: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:294: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: finishing write tid 2 to trial17818531-32 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:294: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 45 (ObjNum 0 snap 0 seq_num 0) dirty exists 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: finishing write tid 3 to trial17818531-32 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: finishing write tid 4 to trial17818531-32 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: oid 32 updating version 0 to 136 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: oid 32 version 136 is already newer than 135 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:293: oid 32 version 136 is already newer than 134 2026-02-21T22:16:59.890 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 136 (ObjNum 124 snap 29 seq_num 124) dirty exists 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:293: left oid 32 (ObjNum 124 snap 29 seq_num 124) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:288: done (5 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:289: done (4 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:291: done (3 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:292: done (2 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:293: done (1 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:294: done (0 left) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:295: read oid 8 snap -1 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:295: expect (ObjNum 102 snap 24 seq_num 102) 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:296: write (excl) oid 18 current snap is 29 2026-02-21T22:16:59.891 INFO:tasks.rados.rados.0.trial178.stdout:296: seq_num 125 ranges {500875=467773,1564955=535920} 2026-02-21T22:16:59.895 INFO:tasks.rados.rados.0.trial178.stdout:296: writing trial17818531-18 from 500875 to 968648 tid 1 2026-02-21T22:16:59.904 INFO:tasks.rados.rados.0.trial178.stdout:296: writing trial17818531-18 from 1564955 to 2100875 tid 2 2026-02-21T22:16:59.904 INFO:tasks.rados.rados.0.trial178.stdout:297: write oid 32 current snap is 29 2026-02-21T22:16:59.904 INFO:tasks.rados.rados.0.trial178.stdout:297: seq_num 126 ranges {517682=543022,1623437=494245} 2026-02-21T22:16:59.911 INFO:tasks.rados.rados.0.trial178.stdout:297: writing trial17818531-32 from 517682 to 1060704 tid 1 2026-02-21T22:16:59.918 INFO:tasks.rados.rados.0.trial178.stdout:297: writing trial17818531-32 from 1623437 to 2117682 tid 2 2026-02-21T22:16:59.918 INFO:tasks.rados.rados.0.trial178.stdout:298: read oid 26 snap -1 2026-02-21T22:16:59.919 INFO:tasks.rados.rados.0.trial178.stdout:298: expect deleted 2026-02-21T22:16:59.919 INFO:tasks.rados.rados.0.trial178.stdout:299: rollback oid 25 current snap is 29 2026-02-21T22:16:59.919 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 25 to 21 2026-02-21T22:16:59.919 INFO:tasks.rados.rados.0.trial178.stdout:295: expect (ObjNum 102 snap 24 seq_num 102) 2026-02-21T22:16:59.978 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 737007 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: finishing write tid 1 to trial17818531-18 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: finishing write tid 2 to trial17818531-18 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: finishing write tid 3 to trial17818531-18 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: finishing write tid 4 to trial17818531-18 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: oid 18 updating version 0 to 86 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: oid 18 updating version 86 to 88 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:296: oid 18 version 88 is already newer than 87 2026-02-21T22:17:00.066 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 88 (ObjNum 125 snap 29 seq_num 125) dirty exists 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:296: left oid 18 (ObjNum 125 snap 29 seq_num 125) 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:297: finishing write tid 1 to trial17818531-32 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:297: finishing write tid 2 to trial17818531-32 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:297: finishing write tid 3 to trial17818531-32 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:295: done (4 left) 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:296: done (3 left) 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:298: done (2 left) 2026-02-21T22:17:00.067 INFO:tasks.rados.rados.0.trial178.stdout:300: snap_remove snap 28 2026-02-21T22:17:00.691 INFO:tasks.rados.rados.0.trial178.stdout:300: done (2 left) 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:301: rollback oid 42 current snap is 29 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 21 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:297: finishing write tid 4 to trial17818531-32 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:297: oid 32 updating version 0 to 137 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:297: oid 32 updating version 137 to 139 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:297: oid 32 version 139 is already newer than 138 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 139 (ObjNum 126 snap 29 seq_num 126) dirty exists 2026-02-21T22:17:00.692 INFO:tasks.rados.rados.0.trial178.stdout:297: left oid 32 (ObjNum 126 snap 29 seq_num 126) 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:299: finishing rollback tid 0 to trial17818531-25 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:299: finishing rollback tid 1 to trial17818531-25 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 52 (ObjNum 145 snap 0 seq_num 101626304) dirty dne 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:297: done (2 left) 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:299: done (1 left) 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:302: copy_from oid 13 from oid 48 current snap is 29 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:303: read oid 22 snap -1 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:303: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:17:00.693 INFO:tasks.rados.rados.0.trial178.stdout:304: snap_create 2026-02-21T22:17:00.696 INFO:tasks.rados.rados.0.trial178.stdout:303: expect (ObjNum 65 snap 4 seq_num 65) 2026-02-21T22:17:00.755 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:301: finishing rollback tid 0 to trial17818531-42 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:301: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:301: finishing rollback tid 2 to trial17818531-42 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 127 (ObjNum 41 snap 0 seq_num 41) dirty exists 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:302: finishing copy_from racing read to trial17818531-13 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:302: finishing copy_from to trial17818531-13 2026-02-21T22:17:00.789 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 22 (ObjNum 17 snap 0 seq_num 17) dirty exists 2026-02-21T22:17:01.691 INFO:tasks.rados.rados.0.trial178.stdout:301: done (3 left) 2026-02-21T22:17:01.691 INFO:tasks.rados.rados.0.trial178.stdout:302: done (2 left) 2026-02-21T22:17:01.691 INFO:tasks.rados.rados.0.trial178.stdout:303: done (1 left) 2026-02-21T22:17:01.691 INFO:tasks.rados.rados.0.trial178.stdout:304: done (0 left) 2026-02-21T22:17:01.691 INFO:tasks.rados.rados.0.trial178.stdout:305: write (excl) oid 24 current snap is 30 2026-02-21T22:17:01.692 INFO:tasks.rados.rados.0.trial178.stdout:305: seq_num 127 ranges {534489=618271,1565566=568923} 2026-02-21T22:17:01.698 INFO:tasks.rados.rados.0.trial178.stdout:305: writing trial17818531-24 from 534489 to 1152760 tid 1 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:305: writing trial17818531-24 from 1565566 to 2134489 tid 2 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:306: read oid 20 snap -1 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:306: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:307: read oid 8 snap -1 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:307: expect (ObjNum 102 snap 24 seq_num 102) 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:308: snap_remove snap 29 2026-02-21T22:17:01.705 INFO:tasks.rados.rados.0.trial178.stdout:305: finishing write tid 1 to trial17818531-24 2026-02-21T22:17:02.703 INFO:tasks.rados.rados.0.trial178.stdout:308: done (3 left) 2026-02-21T22:17:02.703 INFO:tasks.rados.rados.0.trial178.stdout:309: snap_remove snap 21 2026-02-21T22:17:03.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:03.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:03.706 INFO:tasks.rados.rados.0.trial178.stdout:309: done (3 left) 2026-02-21T22:17:03.706 INFO:tasks.rados.rados.0.trial178.stdout:310: write oid 26 current snap is 30 2026-02-21T22:17:03.706 INFO:tasks.rados.rados.0.trial178.stdout:310: seq_num 128 ranges {551296=693520,2024048=127248} 2026-02-21T22:17:03.714 INFO:tasks.rados.rados.0.trial178.stdout:310: writing trial17818531-26 from 551296 to 1244816 tid 1 2026-02-21T22:17:03.721 INFO:tasks.rados.rados.0.trial178.stdout:310: writing trial17818531-26 from 2024048 to 2151296 tid 2 2026-02-21T22:17:03.721 INFO:tasks.rados.rados.0.trial178.stdout:311: write (excl) oid 19 current snap is 30 2026-02-21T22:17:03.721 INFO:tasks.rados.rados.0.trial178.stdout:311: seq_num 129 ranges {568103=768769,2082530=85573} 2026-02-21T22:17:03.729 INFO:tasks.rados.rados.0.trial178.stdout:311: writing trial17818531-19 from 568103 to 1336872 tid 1 2026-02-21T22:17:03.735 INFO:tasks.rados.rados.0.trial178.stdout:311: writing trial17818531-19 from 2082530 to 2168103 tid 2 2026-02-21T22:17:03.735 INFO:tasks.rados.rados.0.trial178.stdout:312: snap_create 2026-02-21T22:17:03.735 INFO:tasks.rados.rados.0.trial178.stdout:306: expect (ObjNum 74 snap 10 seq_num 74) 2026-02-21T22:17:03.745 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1256622 2026-02-21T22:17:03.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:03.837 INFO:tasks.rados.rados.0.trial178.stdout:305: finishing write tid 2 to trial17818531-24 2026-02-21T22:17:03.837 INFO:tasks.rados.rados.0.trial178.stdout:305: finishing write tid 3 to trial17818531-24 2026-02-21T22:17:03.837 INFO:tasks.rados.rados.0.trial178.stdout:305: finishing write tid 4 to trial17818531-24 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:305: oid 24 updating version 0 to 133 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:305: oid 24 updating version 133 to 134 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:305: oid 24 updating version 134 to 135 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 135 (ObjNum 127 snap 30 seq_num 127) dirty exists 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:305: left oid 24 (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:17:03.838 INFO:tasks.rados.rados.0.trial178.stdout:307: expect (ObjNum 102 snap 24 seq_num 102) 2026-02-21T22:17:03.851 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2383239 2026-02-21T22:17:03.877 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:03.924 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: finishing write tid 1 to trial17818531-26 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: finishing write tid 2 to trial17818531-26 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: finishing write tid 3 to trial17818531-26 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: finishing write tid 4 to trial17818531-26 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: oid 26 updating version 0 to 106 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: oid 26 version 106 is already newer than 105 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: oid 26 version 106 is already newer than 104 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 106 (ObjNum 128 snap 30 seq_num 128) dirty exists 2026-02-21T22:17:03.970 INFO:tasks.rados.rados.0.trial178.stdout:310: left oid 26 (ObjNum 128 snap 30 seq_num 128) 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: finishing write tid 1 to trial17818531-19 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: finishing write tid 2 to trial17818531-19 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: finishing write tid 3 to trial17818531-19 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: finishing write tid 4 to trial17818531-19 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: oid 19 updating version 0 to 150 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: oid 19 updating version 150 to 151 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: oid 19 updating version 151 to 152 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 152 (ObjNum 129 snap 30 seq_num 129) dirty exists 2026-02-21T22:17:03.971 INFO:tasks.rados.rados.0.trial178.stdout:311: left oid 19 (ObjNum 129 snap 30 seq_num 129) 2026-02-21T22:17:04.707 INFO:tasks.rados.rados.0.trial178.stdout:305: done (5 left) 2026-02-21T22:17:04.707 INFO:tasks.rados.rados.0.trial178.stdout:306: done (4 left) 2026-02-21T22:17:04.707 INFO:tasks.rados.rados.0.trial178.stdout:307: done (3 left) 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:310: done (2 left) 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:311: done (1 left) 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:312: done (0 left) 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:313: read oid 31 snap -1 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:313: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:17:04.708 INFO:tasks.rados.rados.0.trial178.stdout:314: delete oid 23 current snap is 31 2026-02-21T22:17:04.709 INFO:tasks.rados.rados.0.trial178.stdout:314: done (1 left) 2026-02-21T22:17:04.710 INFO:tasks.rados.rados.0.trial178.stdout:315: write oid 37 current snap is 31 2026-02-21T22:17:04.710 INFO:tasks.rados.rados.0.trial178.stdout:315: seq_num 131 ranges {601717=635620,1799494=402223} 2026-02-21T22:17:04.718 INFO:tasks.rados.rados.0.trial178.stdout:315: writing trial17818531-37 from 601717 to 1237337 tid 1 2026-02-21T22:17:04.725 INFO:tasks.rados.rados.0.trial178.stdout:315: writing trial17818531-37 from 1799494 to 2201717 tid 2 2026-02-21T22:17:04.725 INFO:tasks.rados.rados.0.trial178.stdout:316: read oid 3 snap -1 2026-02-21T22:17:04.725 INFO:tasks.rados.rados.0.trial178.stdout:316: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:17:04.725 INFO:tasks.rados.rados.0.trial178.stdout:317: snap_remove snap 30 2026-02-21T22:17:05.691 INFO:tasks.rados.rados.0.trial178.stdout:317: done (3 left) 2026-02-21T22:17:05.691 INFO:tasks.rados.rados.0.trial178.stdout:318: read oid 35 snap -1 2026-02-21T22:17:05.691 INFO:tasks.rados.rados.0.trial178.stdout:318: expect (ObjNum 122 snap 29 seq_num 122) 2026-02-21T22:17:05.691 INFO:tasks.rados.rados.0.trial178.stdout:319: write oid 20 current snap is 31 2026-02-21T22:17:05.691 INFO:tasks.rados.rados.0.trial178.stdout:319: seq_num 132 ranges {618524=710869,1857976=360548} 2026-02-21T22:17:05.699 INFO:tasks.rados.rados.0.trial178.stdout:319: writing trial17818531-20 from 618524 to 1329393 tid 1 2026-02-21T22:17:05.706 INFO:tasks.rados.rados.0.trial178.stdout:319: writing trial17818531-20 from 1857976 to 2218524 tid 2 2026-02-21T22:17:05.706 INFO:tasks.rados.rados.0.trial178.stdout:320: copy_from oid 9 from oid 39 current snap is 31 2026-02-21T22:17:05.706 INFO:tasks.rados.rados.0.trial178.stdout:313: expect (ObjNum 62 snap 4 seq_num 62) 2026-02-21T22:17:05.706 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:05.768 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2131900 2026-02-21T22:17:05.806 INFO:tasks.rados.rados.0.trial178.stdout:315: finishing write tid 1 to trial17818531-37 2026-02-21T22:17:05.806 INFO:tasks.rados.rados.0.trial178.stdout:315: finishing write tid 2 to trial17818531-37 2026-02-21T22:17:05.806 INFO:tasks.rados.rados.0.trial178.stdout:315: finishing write tid 3 to trial17818531-37 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:315: finishing write tid 4 to trial17818531-37 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:315: oid 37 updating version 0 to 50 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:315: oid 37 updating version 50 to 52 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:315: oid 37 version 52 is already newer than 51 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 52 (ObjNum 131 snap 31 seq_num 131) dirty exists 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:315: left oid 37 (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:17:05.807 INFO:tasks.rados.rados.0.trial178.stdout:316: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:17:05.855 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:05.880 INFO:tasks.rados.rados.0.trial178.stdout:318: expect (ObjNum 122 snap 29 seq_num 122) 2026-02-21T22:17:05.881 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 198128 2026-02-21T22:17:05.907 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1328703 2026-02-21T22:17:05.915 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:05.935 INFO:tasks.rados.rados.0.trial178.stdout:319: finishing write tid 1 to trial17818531-20 2026-02-21T22:17:05.935 INFO:tasks.rados.rados.0.trial178.stdout:319: finishing write tid 2 to trial17818531-20 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: finishing write tid 3 to trial17818531-20 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: finishing write tid 4 to trial17818531-20 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: oid 20 updating version 0 to 77 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: oid 20 version 77 is already newer than 76 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: oid 20 version 77 is already newer than 75 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 77 (ObjNum 132 snap 31 seq_num 132) dirty exists 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:319: left oid 20 (ObjNum 132 snap 31 seq_num 132) 2026-02-21T22:17:05.936 INFO:tasks.rados.rados.0.trial178.stdout:320: finishing copy_from racing read to trial17818531-9 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:320: finishing copy_from to trial17818531-9 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 33 (ObjNum 68 snap 7 seq_num 68) dirty exists 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:313: done (5 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:315: done (4 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:316: done (3 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:318: done (2 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:319: done (1 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:320: done (0 left) 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:321: copy_from oid 33 from oid 38 current snap is 31 2026-02-21T22:17:05.937 INFO:tasks.rados.rados.0.trial178.stdout:322: delete oid 9 current snap is 31 2026-02-21T22:17:05.938 INFO:tasks.rados.rados.0.trial178.stdout:322: done (1 left) 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:323: delete oid 26 current snap is 31 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:323: done (1 left) 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:324: copy_from oid 8 from oid 35 current snap is 31 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:325: read oid 43 snap -1 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:325: expect (ObjNum 88 snap 17 seq_num 88) 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:326: read oid 28 snap -1 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:326: expect (ObjNum 103 snap 25 seq_num 103) 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:327: snap_create 2026-02-21T22:17:05.939 INFO:tasks.rados.rados.0.trial178.stdout:325: expect (ObjNum 88 snap 17 seq_num 88) 2026-02-21T22:17:05.952 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2045680 2026-02-21T22:17:06.022 INFO:tasks.rados.rados.0.trial178.stdout:324: finishing copy_from racing read to trial17818531-8 2026-02-21T22:17:06.022 INFO:tasks.rados.rados.0.trial178.stdout:324: finishing copy_from to trial17818531-8 2026-02-21T22:17:06.022 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 130 (ObjNum 122 snap 29 seq_num 122) dirty exists 2026-02-21T22:17:06.023 INFO:tasks.rados.rados.0.trial178.stdout:326: expect (ObjNum 103 snap 25 seq_num 103) 2026-02-21T22:17:06.080 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2038275 2026-02-21T22:17:06.124 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1248169 2026-02-21T22:17:06.169 INFO:tasks.rados.rados.0.trial178.stdout:321: finishing copy_from to trial17818531-33 2026-02-21T22:17:06.169 INFO:tasks.rados.rados.0.trial178.stdout:321: got expected ENOENT (src dne) 2026-02-21T22:17:06.169 INFO:tasks.rados.rados.0.trial178.stdout:321: finishing copy_from racing read to trial17818531-33 2026-02-21T22:17:06.713 INFO:tasks.rados.rados.0.trial178.stdout:321: done (4 left) 2026-02-21T22:17:06.713 INFO:tasks.rados.rados.0.trial178.stdout:324: done (3 left) 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:325: done (2 left) 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:326: done (1 left) 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:327: done (0 left) 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:328: rollback oid 47 current snap is 32 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 31 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:329: read oid 41 snap -1 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:329: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:17:06.714 INFO:tasks.rados.rados.0.trial178.stdout:330: delete oid 4 current snap is 32 2026-02-21T22:17:06.716 INFO:tasks.rados.rados.0.trial178.stdout:328: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:17:06.716 INFO:tasks.rados.rados.0.trial178.stdout:328: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:17:06.716 INFO:tasks.rados.rados.0.trial178.stdout:328: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:17:06.716 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 157 (ObjNum 123 snap 29 seq_num 123) dirty exists 2026-02-21T22:17:06.718 INFO:tasks.rados.rados.0.trial178.stdout:328: done (2 left) 2026-02-21T22:17:06.718 INFO:tasks.rados.rados.0.trial178.stdout:330: done (1 left) 2026-02-21T22:17:06.718 INFO:tasks.rados.rados.0.trial178.stdout:331: write (excl) oid 32 current snap is 32 2026-02-21T22:17:06.718 INFO:tasks.rados.rados.0.trial178.stdout:331: seq_num 136 ranges {685752=611865,1975551=310201} 2026-02-21T22:17:06.727 INFO:tasks.rados.rados.0.trial178.stdout:331: writing trial17818531-32 from 685752 to 1297617 tid 1 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:331: writing trial17818531-32 from 1975551 to 2285752 tid 2 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:332: read oid 50 snap -1 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:332: expect deleted 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:333: read oid 2 snap 31 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:333: expect deleted 2026-02-21T22:17:06.734 INFO:tasks.rados.rados.0.trial178.stdout:334: copy_from oid 38 from oid 21 current snap is 32 2026-02-21T22:17:06.735 INFO:tasks.rados.rados.0.trial178.stdout:329: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:17:06.735 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 531380 2026-02-21T22:17:06.766 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 594057 2026-02-21T22:17:06.784 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:06.804 INFO:tasks.rados.rados.0.trial178.stdout:331: finishing write tid 1 to trial17818531-32 2026-02-21T22:17:06.804 INFO:tasks.rados.rados.0.trial178.stdout:329: done (4 left) 2026-02-21T22:17:06.804 INFO:tasks.rados.rados.0.trial178.stdout:333: done (3 left) 2026-02-21T22:17:06.804 INFO:tasks.rados.rados.0.trial178.stdout:335: read oid 24 snap 31 2026-02-21T22:17:06.804 INFO:tasks.rados.rados.0.trial178.stdout:335: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:336: copy_from oid 27 from oid 10 current snap is 32 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: finishing write tid 2 to trial17818531-32 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: finishing write tid 3 to trial17818531-32 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: finishing write tid 4 to trial17818531-32 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: oid 32 updating version 0 to 154 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: oid 32 updating version 154 to 155 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: oid 32 version 155 is already newer than 153 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 155 (ObjNum 136 snap 32 seq_num 136) dirty exists 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:331: left oid 32 (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:334: finishing copy_from to trial17818531-38 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:334: got expected ENOENT (src dne) 2026-02-21T22:17:06.805 INFO:tasks.rados.rados.0.trial178.stdout:334: finishing copy_from racing read to trial17818531-38 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:331: done (4 left) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:332: done (3 left) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:334: done (2 left) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:337: copy_from oid 23 from oid 30 current snap is 32 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:338: read oid 41 snap -1 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:338: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:339: read oid 47 snap -1 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:339: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:340: read oid 36 snap -1 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:340: expect (ObjNum 117 snap 29 seq_num 117) 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:341: rollback oid 12 current snap is 32 2026-02-21T22:17:06.806 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 31 2026-02-21T22:17:06.807 INFO:tasks.rados.rados.0.trial178.stdout:342: snap_create 2026-02-21T22:17:06.807 INFO:tasks.rados.rados.0.trial178.stdout:341: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:17:06.807 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 113 snap 0 seq_num 101572192) dirty dne 2026-02-21T22:17:06.809 INFO:tasks.rados.rados.0.trial178.stdout:338: expect (ObjNum 35 snap 0 seq_num 35) 2026-02-21T22:17:06.809 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:06.842 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:06.866 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:06.886 INFO:tasks.rados.rados.0.trial178.stdout:335: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:17:06.890 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 834426 2026-02-21T22:17:06.907 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:06.955 INFO:tasks.rados.rados.0.trial178.stdout:339: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:06.958 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1324406 2026-02-21T22:17:06.977 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:07.026 INFO:tasks.rados.rados.0.trial178.stdout:340: expect (ObjNum 117 snap 29 seq_num 117) 2026-02-21T22:17:07.079 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 341882 2026-02-21T22:17:07.129 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 370301 2026-02-21T22:17:07.179 INFO:tasks.rados.rados.0.trial178.stdout:336: finishing copy_from racing read to trial17818531-27 2026-02-21T22:17:07.179 INFO:tasks.rados.rados.0.trial178.stdout:336: finishing copy_from to trial17818531-27 2026-02-21T22:17:07.180 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 69 (ObjNum 115 snap 27 seq_num 115) dirty exists 2026-02-21T22:17:07.180 INFO:tasks.rados.rados.0.trial178.stdout:337: finishing copy_from racing read to trial17818531-23 2026-02-21T22:17:07.180 INFO:tasks.rados.rados.0.trial178.stdout:337: finishing copy_from to trial17818531-23 2026-02-21T22:17:07.180 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 85 (ObjNum 91 snap 17 seq_num 91) dirty exists 2026-02-21T22:17:07.720 INFO:tasks.rados.rados.0.trial178.stdout:335: done (7 left) 2026-02-21T22:17:07.720 INFO:tasks.rados.rados.0.trial178.stdout:336: done (6 left) 2026-02-21T22:17:07.720 INFO:tasks.rados.rados.0.trial178.stdout:337: done (5 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:338: done (4 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:339: done (3 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:340: done (2 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:341: done (1 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:342: done (0 left) 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:343: write oid 41 current snap is 33 2026-02-21T22:17:07.721 INFO:tasks.rados.rados.0.trial178.stdout:343: seq_num 137 ranges {702559=403467,1750386=495032} 2026-02-21T22:17:07.727 INFO:tasks.rados.rados.0.trial178.stdout:343: writing trial17818531-41 from 702559 to 1106026 tid 1 2026-02-21T22:17:07.735 INFO:tasks.rados.rados.0.trial178.stdout:343: writing trial17818531-41 from 1750386 to 2245418 tid 2 2026-02-21T22:17:07.735 INFO:tasks.rados.rados.0.trial178.stdout:344: snap_create 2026-02-21T22:17:07.736 INFO:tasks.rados.rados.0.trial178.stdout:343: finishing write tid 1 to trial17818531-41 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: finishing write tid 2 to trial17818531-41 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: finishing write tid 3 to trial17818531-41 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: finishing write tid 4 to trial17818531-41 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: oid 41 updating version 0 to 81 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: oid 41 version 81 is already newer than 79 2026-02-21T22:17:07.738 INFO:tasks.rados.rados.0.trial178.stdout:343: oid 41 version 81 is already newer than 80 2026-02-21T22:17:07.739 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 81 (ObjNum 137 snap 33 seq_num 137) dirty exists 2026-02-21T22:17:07.739 INFO:tasks.rados.rados.0.trial178.stdout:343: left oid 41 (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:17:08.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:08.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:08.720 INFO:tasks.rados.rados.0.trial178.stdout:343: done (1 left) 2026-02-21T22:17:08.720 INFO:tasks.rados.rados.0.trial178.stdout:344: done (0 left) 2026-02-21T22:17:08.720 INFO:tasks.rados.rados.0.trial178.stdout:345: copy_from oid 23 from oid 47 current snap is 34 2026-02-21T22:17:08.720 INFO:tasks.rados.rados.0.trial178.stdout:346: write oid 6 current snap is 34 2026-02-21T22:17:08.720 INFO:tasks.rados.rados.0.trial178.stdout:346: seq_num 138 ranges {719366=478716,1808868=510498} 2026-02-21T22:17:08.728 INFO:tasks.rados.rados.0.trial178.stdout:346: writing trial17818531-6 from 719366 to 1198082 tid 1 2026-02-21T22:17:08.735 INFO:tasks.rados.rados.0.trial178.stdout:346: writing trial17818531-6 from 1808868 to 2319366 tid 2 2026-02-21T22:17:08.736 INFO:tasks.rados.rados.0.trial178.stdout:347: snap_create 2026-02-21T22:17:08.736 INFO:tasks.rados.rados.0.trial178.stdout:345: finishing copy_from racing read to trial17818531-23 2026-02-21T22:17:08.736 INFO:tasks.rados.rados.0.trial178.stdout:345: finishing copy_from to trial17818531-23 2026-02-21T22:17:08.736 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 157 (ObjNum 123 snap 29 seq_num 123) dirty exists 2026-02-21T22:17:08.736 INFO:tasks.rados.rados.0.trial178.stdout:346: finishing write tid 1 to trial17818531-6 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: finishing write tid 2 to trial17818531-6 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: finishing write tid 3 to trial17818531-6 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: finishing write tid 4 to trial17818531-6 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: oid 6 updating version 0 to 110 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: oid 6 version 110 is already newer than 109 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: oid 6 updating version 110 to 111 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 111 (ObjNum 138 snap 34 seq_num 138) dirty exists 2026-02-21T22:17:08.738 INFO:tasks.rados.rados.0.trial178.stdout:346: left oid 6 (ObjNum 138 snap 34 seq_num 138) 2026-02-21T22:17:09.727 INFO:tasks.rados.rados.0.trial178.stdout:345: done (2 left) 2026-02-21T22:17:09.727 INFO:tasks.rados.rados.0.trial178.stdout:346: done (1 left) 2026-02-21T22:17:09.727 INFO:tasks.rados.rados.0.trial178.stdout:347: done (0 left) 2026-02-21T22:17:09.727 INFO:tasks.rados.rados.0.trial178.stdout:348: read oid 26 snap -1 2026-02-21T22:17:09.728 INFO:tasks.rados.rados.0.trial178.stdout:348: expect deleted 2026-02-21T22:17:09.728 INFO:tasks.rados.rados.0.trial178.stdout:349: write (excl) oid 4 current snap is 35 2026-02-21T22:17:09.728 INFO:tasks.rados.rados.0.trial178.stdout:349: seq_num 139 ranges {736173=553965,1867350=468823} 2026-02-21T22:17:09.735 INFO:tasks.rados.rados.0.trial178.stdout:349: writing trial17818531-4 from 736173 to 1290138 tid 1 2026-02-21T22:17:09.742 INFO:tasks.rados.rados.0.trial178.stdout:349: writing trial17818531-4 from 1867350 to 2336173 tid 2 2026-02-21T22:17:09.743 INFO:tasks.rados.rados.0.trial178.stdout:350: write oid 48 current snap is 35 2026-02-21T22:17:09.743 INFO:tasks.rados.rados.0.trial178.stdout:350: seq_num 140 ranges {752980=629214,1809479=543501} 2026-02-21T22:17:09.751 INFO:tasks.rados.rados.0.trial178.stdout:350: writing trial17818531-48 from 752980 to 1382194 tid 1 2026-02-21T22:17:09.758 INFO:tasks.rados.rados.0.trial178.stdout:350: writing trial17818531-48 from 1809479 to 2352980 tid 2 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:351: snap_create 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: finishing write tid 1 to trial17818531-4 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: finishing write tid 2 to trial17818531-4 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: finishing write tid 3 to trial17818531-4 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: finishing write tid 4 to trial17818531-4 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: oid 4 updating version 0 to 64 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: oid 4 updating version 64 to 65 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: oid 4 updating version 65 to 66 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 66 (ObjNum 139 snap 35 seq_num 139) dirty exists 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:349: left oid 4 (ObjNum 139 snap 35 seq_num 139) 2026-02-21T22:17:09.759 INFO:tasks.rados.rados.0.trial178.stdout:350: finishing write tid 1 to trial17818531-48 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: finishing write tid 2 to trial17818531-48 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: finishing write tid 3 to trial17818531-48 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: finishing write tid 4 to trial17818531-48 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: oid 48 updating version 0 to 56 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: oid 48 version 56 is already newer than 55 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:350: oid 48 version 56 is already newer than 54 2026-02-21T22:17:09.760 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 56 (ObjNum 140 snap 35 seq_num 140) dirty exists 2026-02-21T22:17:09.761 INFO:tasks.rados.rados.0.trial178.stdout:350: left oid 48 (ObjNum 140 snap 35 seq_num 140) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:348: done (3 left) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:349: done (2 left) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:350: done (1 left) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:351: done (0 left) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:352: read oid 3 snap 33 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:352: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:353: write oid 1 current snap is 36 2026-02-21T22:17:10.692 INFO:tasks.rados.rados.0.trial178.stdout:353: seq_num 141 ranges {769787=704463,2267961=101826} 2026-02-21T22:17:10.701 INFO:tasks.rados.rados.0.trial178.stdout:353: writing trial17818531-1 from 769787 to 1474250 tid 1 2026-02-21T22:17:10.707 INFO:tasks.rados.rados.0.trial178.stdout:353: writing trial17818531-1 from 2267961 to 2369787 tid 2 2026-02-21T22:17:10.707 INFO:tasks.rados.rados.0.trial178.stdout:354: write (excl) oid 9 current snap is 36 2026-02-21T22:17:10.707 INFO:tasks.rados.rados.0.trial178.stdout:354: seq_num 142 ranges {786594=779712,2326443=60151} 2026-02-21T22:17:10.718 INFO:tasks.rados.rados.0.trial178.stdout:354: writing trial17818531-9 from 786594 to 1566306 tid 1 2026-02-21T22:17:10.723 INFO:tasks.rados.rados.0.trial178.stdout:354: writing trial17818531-9 from 2326443 to 2386594 tid 2 2026-02-21T22:17:10.723 INFO:tasks.rados.rados.0.trial178.stdout:355: snap_remove snap 35 2026-02-21T22:17:11.695 INFO:tasks.rados.rados.0.trial178.stdout:355: done (3 left) 2026-02-21T22:17:11.695 INFO:tasks.rados.rados.0.trial178.stdout:356: snap_remove snap 32 2026-02-21T22:17:12.697 INFO:tasks.rados.rados.0.trial178.stdout:356: done (3 left) 2026-02-21T22:17:12.697 INFO:tasks.rados.rados.0.trial178.stdout:357: snap_create 2026-02-21T22:17:12.697 INFO:tasks.rados.rados.0.trial178.stdout:352: expect (ObjNum 101 snap 24 seq_num 101) 2026-02-21T22:17:12.786 INFO:tasks.rados.rados.0.trial178.stdout:353: finishing write tid 1 to trial17818531-1 2026-02-21T22:17:12.786 INFO:tasks.rados.rados.0.trial178.stdout:353: finishing write tid 2 to trial17818531-1 2026-02-21T22:17:12.786 INFO:tasks.rados.rados.0.trial178.stdout:353: finishing write tid 3 to trial17818531-1 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:353: finishing write tid 4 to trial17818531-1 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:353: oid 1 updating version 0 to 59 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:353: oid 1 version 59 is already newer than 58 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:353: oid 1 updating version 59 to 60 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 60 (ObjNum 141 snap 36 seq_num 141) dirty exists 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:353: left oid 1 (ObjNum 141 snap 36 seq_num 141) 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: finishing write tid 1 to trial17818531-9 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: finishing write tid 2 to trial17818531-9 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: finishing write tid 3 to trial17818531-9 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: finishing write tid 4 to trial17818531-9 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: oid 9 updating version 0 to 84 2026-02-21T22:17:12.787 INFO:tasks.rados.rados.0.trial178.stdout:354: oid 9 version 84 is already newer than 82 2026-02-21T22:17:12.788 INFO:tasks.rados.rados.0.trial178.stdout:354: oid 9 version 84 is already newer than 83 2026-02-21T22:17:12.788 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 84 (ObjNum 142 snap 36 seq_num 142) dirty exists 2026-02-21T22:17:12.788 INFO:tasks.rados.rados.0.trial178.stdout:354: left oid 9 (ObjNum 142 snap 36 seq_num 142) 2026-02-21T22:17:13.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:13.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:13.697 INFO:tasks.rados.rados.0.trial178.stdout:352: done (3 left) 2026-02-21T22:17:13.697 INFO:tasks.rados.rados.0.trial178.stdout:353: done (2 left) 2026-02-21T22:17:13.698 INFO:tasks.rados.rados.0.trial178.stdout:354: done (1 left) 2026-02-21T22:17:13.698 INFO:tasks.rados.rados.0.trial178.stdout:357: done (0 left) 2026-02-21T22:17:13.698 INFO:tasks.rados.rados.0.trial178.stdout:358: write (excl) oid 50 current snap is 37 2026-02-21T22:17:13.698 INFO:tasks.rados.rados.0.trial178.stdout:358: seq_num 143 ranges {403401=454961,1584925=506039} 2026-02-21T22:17:13.703 INFO:tasks.rados.rados.0.trial178.stdout:358: writing trial17818531-50 from 403401 to 858362 tid 1 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:358: writing trial17818531-50 from 1584925 to 2090964 tid 2 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:359: read oid 42 snap -1 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:359: expect (ObjNum 41 snap 0 seq_num 41) 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:360: read oid 1 snap -1 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:360: expect (ObjNum 141 snap 36 seq_num 141) 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:361: read oid 36 snap 31 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:361: expect (ObjNum 117 snap 29 seq_num 117) 2026-02-21T22:17:13.712 INFO:tasks.rados.rados.0.trial178.stdout:362: snap_remove snap 34 2026-02-21T22:17:14.698 INFO:tasks.rados.rados.0.trial178.stdout:362: done (4 left) 2026-02-21T22:17:14.699 INFO:tasks.rados.rados.0.trial178.stdout:363: write oid 43 current snap is 37 2026-02-21T22:17:14.699 INFO:tasks.rados.rados.0.trial178.stdout:363: seq_num 144 ranges {420208=530210,1527054=766050} 2026-02-21T22:17:14.705 INFO:tasks.rados.rados.0.trial178.stdout:363: writing trial17818531-43 from 420208 to 950418 tid 1 2026-02-21T22:17:14.714 INFO:tasks.rados.rados.0.trial178.stdout:363: writing trial17818531-43 from 1527054 to 2293104 tid 2 2026-02-21T22:17:14.714 INFO:tasks.rados.rados.0.trial178.stdout:364: snap_create 2026-02-21T22:17:14.714 INFO:tasks.rados.rados.0.trial178.stdout:358: finishing write tid 1 to trial17818531-50 2026-02-21T22:17:14.714 INFO:tasks.rados.rados.0.trial178.stdout:358: finishing write tid 2 to trial17818531-50 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: finishing write tid 3 to trial17818531-50 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: finishing write tid 4 to trial17818531-50 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: oid 50 updating version 0 to 44 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: oid 50 updating version 44 to 45 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: oid 50 updating version 45 to 46 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 46 (ObjNum 143 snap 37 seq_num 143) dirty exists 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:358: left oid 50 (ObjNum 143 snap 37 seq_num 143) 2026-02-21T22:17:14.715 INFO:tasks.rados.rados.0.trial178.stdout:360: expect (ObjNum 141 snap 36 seq_num 141) 2026-02-21T22:17:14.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 367871 2026-02-21T22:17:14.803 INFO:tasks.rados.rados.0.trial178.stdout:361: expect (ObjNum 117 snap 29 seq_num 117) 2026-02-21T22:17:14.900 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:14.950 INFO:tasks.rados.rados.0.trial178.stdout:359: expect (ObjNum 41 snap 0 seq_num 41) 2026-02-21T22:17:14.950 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:14.988 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: finishing write tid 1 to trial17818531-43 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: finishing write tid 2 to trial17818531-43 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: finishing write tid 3 to trial17818531-43 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: finishing write tid 4 to trial17818531-43 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: oid 43 updating version 0 to 17 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: oid 43 version 17 is already newer than 15 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:363: oid 43 version 17 is already newer than 16 2026-02-21T22:17:15.008 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 17 (ObjNum 144 snap 37 seq_num 144) dirty exists 2026-02-21T22:17:15.009 INFO:tasks.rados.rados.0.trial178.stdout:363: left oid 43 (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:17:15.692 INFO:tasks.rados.rados.0.trial178.stdout:358: done (5 left) 2026-02-21T22:17:15.692 INFO:tasks.rados.rados.0.trial178.stdout:359: done (4 left) 2026-02-21T22:17:15.693 INFO:tasks.rados.rados.0.trial178.stdout:360: done (3 left) 2026-02-21T22:17:15.693 INFO:tasks.rados.rados.0.trial178.stdout:361: done (2 left) 2026-02-21T22:17:15.693 INFO:tasks.rados.rados.0.trial178.stdout:363: done (1 left) 2026-02-21T22:17:15.693 INFO:tasks.rados.rados.0.trial178.stdout:364: done (0 left) 2026-02-21T22:17:15.693 INFO:tasks.rados.rados.0.trial178.stdout:365: delete oid 30 current snap is 38 2026-02-21T22:17:15.696 INFO:tasks.rados.rados.0.trial178.stdout:365: done (0 left) 2026-02-21T22:17:15.696 INFO:tasks.rados.rados.0.trial178.stdout:366: snap_remove snap 31 2026-02-21T22:17:16.693 INFO:tasks.rados.rados.0.trial178.stdout:366: done (0 left) 2026-02-21T22:17:16.693 INFO:tasks.rados.rados.0.trial178.stdout:367: delete oid 19 current snap is 38 2026-02-21T22:17:16.698 INFO:tasks.rados.rados.0.trial178.stdout:367: done (0 left) 2026-02-21T22:17:16.698 INFO:tasks.rados.rados.0.trial178.stdout:368: write oid 13 current snap is 38 2026-02-21T22:17:16.698 INFO:tasks.rados.rados.0.trial178.stdout:368: seq_num 147 ranges {470629=472310,1418853=513377} 2026-02-21T22:17:16.704 INFO:tasks.rados.rados.0.trial178.stdout:368: writing trial17818531-13 from 470629 to 942939 tid 1 2026-02-21T22:17:16.711 INFO:tasks.rados.rados.0.trial178.stdout:368: writing trial17818531-13 from 1418853 to 1932230 tid 2 2026-02-21T22:17:16.711 INFO:tasks.rados.rados.0.trial178.stdout:369: write (excl) oid 30 current snap is 38 2026-02-21T22:17:16.711 INFO:tasks.rados.rados.0.trial178.stdout:368: finishing write tid 1 to trial17818531-13 2026-02-21T22:17:16.712 INFO:tasks.rados.rados.0.trial178.stdout:369: seq_num 148 ranges {487436=547559,1760982=657035} 2026-02-21T22:17:16.718 INFO:tasks.rados.rados.0.trial178.stdout:369: writing trial17818531-30 from 487436 to 1034995 tid 1 2026-02-21T22:17:16.728 INFO:tasks.rados.rados.0.trial178.stdout:369: writing trial17818531-30 from 1760982 to 2418017 tid 2 2026-02-21T22:17:16.728 INFO:tasks.rados.rados.0.trial178.stdout:370: snap_create 2026-02-21T22:17:16.728 INFO:tasks.rados.rados.0.trial178.stdout:368: finishing write tid 2 to trial17818531-13 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: finishing write tid 3 to trial17818531-13 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: finishing write tid 4 to trial17818531-13 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: oid 13 updating version 0 to 159 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: oid 13 updating version 159 to 161 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: oid 13 version 161 is already newer than 160 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 161 (ObjNum 147 snap 38 seq_num 147) dirty exists 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:368: left oid 13 (ObjNum 147 snap 38 seq_num 147) 2026-02-21T22:17:16.729 INFO:tasks.rados.rados.0.trial178.stdout:369: finishing write tid 1 to trial17818531-30 2026-02-21T22:17:16.730 INFO:tasks.rados.rados.0.trial178.stdout:369: finishing write tid 2 to trial17818531-30 2026-02-21T22:17:16.730 INFO:tasks.rados.rados.0.trial178.stdout:369: finishing write tid 3 to trial17818531-30 2026-02-21T22:17:16.730 INFO:tasks.rados.rados.0.trial178.stdout:369: finishing write tid 4 to trial17818531-30 2026-02-21T22:17:16.731 INFO:tasks.rados.rados.0.trial178.stdout:369: oid 30 updating version 0 to 90 2026-02-21T22:17:16.731 INFO:tasks.rados.rados.0.trial178.stdout:369: oid 30 updating version 90 to 92 2026-02-21T22:17:16.731 INFO:tasks.rados.rados.0.trial178.stdout:369: oid 30 version 92 is already newer than 91 2026-02-21T22:17:16.731 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 92 (ObjNum 148 snap 38 seq_num 148) dirty exists 2026-02-21T22:17:16.731 INFO:tasks.rados.rados.0.trial178.stdout:369: left oid 30 (ObjNum 148 snap 38 seq_num 148) 2026-02-21T22:17:17.699 INFO:tasks.rados.rados.0.trial178.stdout:368: done (2 left) 2026-02-21T22:17:17.699 INFO:tasks.rados.rados.0.trial178.stdout:369: done (1 left) 2026-02-21T22:17:17.699 INFO:tasks.rados.rados.0.trial178.stdout:370: done (0 left) 2026-02-21T22:17:17.699 INFO:tasks.rados.rados.0.trial178.stdout:371: snap_remove snap 36 2026-02-21T22:17:17.701 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:17:17.709 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:17:17.709 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:17:17.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:17:17.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:17:17.712 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:17:17.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:17:17.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:17:17.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:17:17.716 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:17:17.716 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:17:18.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:18.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:18.699 INFO:tasks.rados.rados.0.trial178.stdout:371: done (0 left) 2026-02-21T22:17:18.700 INFO:tasks.rados.rados.0.trial178.stdout:372: rollback oid 15 current snap is 39 2026-02-21T22:17:18.700 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 38 2026-02-21T22:17:18.700 INFO:tasks.rados.rados.0.trial178.stdout:373: delete oid 21 current snap is 39 2026-02-21T22:17:18.700 INFO:tasks.rados.rados.0.trial178.stdout:373: done (1 left) 2026-02-21T22:17:18.700 INFO:tasks.rados.rados.0.trial178.stdout:374: snap_create 2026-02-21T22:17:18.704 INFO:tasks.rados.rados.0.trial178.stdout:372: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:17:18.704 INFO:tasks.rados.rados.0.trial178.stdout:372: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:17:18.704 INFO:tasks.rados.rados.0.trial178.stdout:372: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:17:18.705 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 89 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:17:19.703 INFO:tasks.rados.rados.0.trial178.stdout:372: done (1 left) 2026-02-21T22:17:19.703 INFO:tasks.rados.rados.0.trial178.stdout:374: done (0 left) 2026-02-21T22:17:19.704 INFO:tasks.rados.rados.0.trial178.stdout:375: write oid 7 current snap is 40 2026-02-21T22:17:19.704 INFO:tasks.rados.rados.0.trial178.stdout:375: seq_num 150 ranges {521050=698057,1877946=643104} 2026-02-21T22:17:19.711 INFO:tasks.rados.rados.0.trial178.stdout:375: writing trial17818531-7 from 521050 to 1219107 tid 1 2026-02-21T22:17:19.720 INFO:tasks.rados.rados.0.trial178.stdout:375: writing trial17818531-7 from 1877946 to 2521050 tid 2 2026-02-21T22:17:19.720 INFO:tasks.rados.rados.0.trial178.stdout:376: delete oid 22 current snap is 40 2026-02-21T22:17:19.721 INFO:tasks.rados.rados.0.trial178.stdout:375: finishing write tid 1 to trial17818531-7 2026-02-21T22:17:19.724 INFO:tasks.rados.rados.0.trial178.stdout:376: done (1 left) 2026-02-21T22:17:19.724 INFO:tasks.rados.rados.0.trial178.stdout:377: read oid 34 snap 38 2026-02-21T22:17:19.724 INFO:tasks.rados.rados.0.trial178.stdout:377: expect deleted 2026-02-21T22:17:19.724 INFO:tasks.rados.rados.0.trial178.stdout:378: snap_create 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: finishing write tid 2 to trial17818531-7 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: finishing write tid 3 to trial17818531-7 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: finishing write tid 4 to trial17818531-7 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: oid 7 updating version 0 to 166 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: oid 7 updating version 166 to 167 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:375: oid 7 updating version 167 to 168 2026-02-21T22:17:19.734 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 168 (ObjNum 150 snap 40 seq_num 150) dirty exists 2026-02-21T22:17:19.735 INFO:tasks.rados.rados.0.trial178.stdout:375: left oid 7 (ObjNum 150 snap 40 seq_num 150) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:375: done (2 left) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:377: done (1 left) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:378: done (0 left) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:379: read oid 23 snap 39 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:379: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:380: read oid 20 snap -1 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:380: expect (ObjNum 132 snap 31 seq_num 132) 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:381: write (excl) oid 39 current snap is 41 2026-02-21T22:17:20.694 INFO:tasks.rados.rados.0.trial178.stdout:381: seq_num 152 ranges {554664=448555,1478557=664373} 2026-02-21T22:17:20.700 INFO:tasks.rados.rados.0.trial178.stdout:381: writing trial17818531-39 from 554664 to 1003219 tid 1 2026-02-21T22:17:20.707 INFO:tasks.rados.rados.0.trial178.stdout:381: writing trial17818531-39 from 1478557 to 2142930 tid 2 2026-02-21T22:17:20.708 INFO:tasks.rados.rados.0.trial178.stdout:382: write (excl) oid 50 current snap is 41 2026-02-21T22:17:20.708 INFO:tasks.rados.rados.0.trial178.stdout:382: seq_num 153 ranges {571471=640157,1653392=524384} 2026-02-21T22:17:20.715 INFO:tasks.rados.rados.0.trial178.stdout:382: writing trial17818531-50 from 571471 to 1211628 tid 1 2026-02-21T22:17:20.722 INFO:tasks.rados.rados.0.trial178.stdout:382: writing trial17818531-50 from 1653392 to 2177776 tid 2 2026-02-21T22:17:20.722 INFO:tasks.rados.rados.0.trial178.stdout:383: write (excl) oid 22 current snap is 41 2026-02-21T22:17:20.722 INFO:tasks.rados.rados.0.trial178.stdout:383: seq_num 154 ranges {588278=715406,1711874=668042} 2026-02-21T22:17:20.731 INFO:tasks.rados.rados.0.trial178.stdout:383: writing trial17818531-22 from 588278 to 1303684 tid 1 2026-02-21T22:17:20.738 INFO:tasks.rados.rados.0.trial178.stdout:383: writing trial17818531-22 from 1711874 to 2379916 tid 2 2026-02-21T22:17:20.738 INFO:tasks.rados.rados.0.trial178.stdout:384: write (excl) oid 18 current snap is 41 2026-02-21T22:17:20.738 INFO:tasks.rados.rados.0.trial178.stdout:384: seq_num 155 ranges {605085=790655,2170356=434729} 2026-02-21T22:17:20.744 INFO:tasks.rados.rados.0.trial178.stdout:384: writing trial17818531-18 from 605085 to 1395740 tid 1 2026-02-21T22:17:20.749 INFO:tasks.rados.rados.0.trial178.stdout:384: writing trial17818531-18 from 2170356 to 2605085 tid 2 2026-02-21T22:17:20.750 INFO:tasks.rados.rados.0.trial178.stdout:385: rollback oid 8 current snap is 41 2026-02-21T22:17:20.750 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 33 2026-02-21T22:17:20.750 INFO:tasks.rados.rados.0.trial178.stdout:379: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:20.750 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:20.815 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2012558 2026-02-21T22:17:20.820 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 873355 2026-02-21T22:17:20.839 INFO:tasks.rados.rados.0.trial178.stdout:380: expect (ObjNum 132 snap 31 seq_num 132) 2026-02-21T22:17:20.860 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 658917 2026-02-21T22:17:20.874 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 415048 2026-02-21T22:17:20.894 INFO:tasks.rados.rados.0.trial178.stdout:381: finishing write tid 1 to trial17818531-39 2026-02-21T22:17:20.894 INFO:tasks.rados.rados.0.trial178.stdout:381: finishing write tid 2 to trial17818531-39 2026-02-21T22:17:20.894 INFO:tasks.rados.rados.0.trial178.stdout:381: finishing write tid 3 to trial17818531-39 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:381: finishing write tid 4 to trial17818531-39 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:381: oid 39 updating version 0 to 88 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:381: oid 39 updating version 88 to 89 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:381: oid 39 updating version 89 to 90 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 90 (ObjNum 152 snap 41 seq_num 152) dirty exists 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:381: left oid 39 (ObjNum 152 snap 41 seq_num 152) 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:382: finishing write tid 1 to trial17818531-50 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:382: finishing write tid 2 to trial17818531-50 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:382: finishing write tid 3 to trial17818531-50 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:382: finishing write tid 4 to trial17818531-50 2026-02-21T22:17:20.895 INFO:tasks.rados.rados.0.trial178.stdout:382: oid 50 updating version 0 to 52 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:382: oid 50 version 52 is already newer than 51 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:382: oid 50 version 52 is already newer than 50 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 52 (ObjNum 153 snap 41 seq_num 153) dirty exists 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:382: left oid 50 (ObjNum 153 snap 41 seq_num 153) 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: finishing write tid 1 to trial17818531-22 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: finishing write tid 2 to trial17818531-22 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: finishing write tid 3 to trial17818531-22 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: finishing write tid 4 to trial17818531-22 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: oid 22 updating version 0 to 53 2026-02-21T22:17:20.896 INFO:tasks.rados.rados.0.trial178.stdout:383: oid 22 updating version 53 to 54 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:383: oid 22 updating version 54 to 55 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 55 (ObjNum 154 snap 41 seq_num 154) dirty exists 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:383: left oid 22 (ObjNum 154 snap 41 seq_num 154) 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:385: finishing rollback tid 0 to trial17818531-8 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:384: finishing write tid 1 to trial17818531-18 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:385: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:384: finishing write tid 2 to trial17818531-18 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:385: finishing rollback tid 2 to trial17818531-8 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 172 (ObjNum 122 snap 29 seq_num 122) dirty exists 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:384: finishing write tid 3 to trial17818531-18 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:384: finishing write tid 4 to trial17818531-18 2026-02-21T22:17:20.897 INFO:tasks.rados.rados.0.trial178.stdout:384: oid 18 updating version 0 to 93 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:384: oid 18 updating version 93 to 94 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:384: oid 18 updating version 94 to 95 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 95 (ObjNum 155 snap 41 seq_num 155) dirty exists 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:384: left oid 18 (ObjNum 155 snap 41 seq_num 155) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:379: done (6 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:380: done (5 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:381: done (4 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:382: done (3 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:383: done (2 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:384: done (1 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:385: done (0 left) 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:386: rollback oid 12 current snap is 41 2026-02-21T22:17:20.898 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 40 2026-02-21T22:17:20.899 INFO:tasks.rados.rados.0.trial178.stdout:387: read oid 23 snap 40 2026-02-21T22:17:20.899 INFO:tasks.rados.rados.0.trial178.stdout:387: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:20.899 INFO:tasks.rados.rados.0.trial178.stdout:388: snap_remove snap 38 2026-02-21T22:17:21.710 INFO:tasks.rados.rados.0.trial178.stdout:388: done (2 left) 2026-02-21T22:17:21.710 INFO:tasks.rados.rados.0.trial178.stdout:389: rollback oid 39 current snap is 41 2026-02-21T22:17:21.710 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 40 2026-02-21T22:17:21.710 INFO:tasks.rados.rados.0.trial178.stdout:387: expect (ObjNum 123 snap 29 seq_num 123) 2026-02-21T22:17:21.710 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:21.771 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1826461 2026-02-21T22:17:21.805 INFO:tasks.rados.rados.0.trial178.stdout:386: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:17:21.805 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 689 snap 0 seq_num 1353603102) dirty dne 2026-02-21T22:17:21.805 INFO:tasks.rados.rados.0.trial178.stdout:389: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:389: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:386: done (2 left) 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:387: done (1 left) 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:390: snap_create 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:389: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:17:21.806 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 93 (ObjNum 68 snap 7 seq_num 68) dirty exists 2026-02-21T22:17:22.713 INFO:tasks.rados.rados.0.trial178.stdout:389: done (1 left) 2026-02-21T22:17:22.713 INFO:tasks.rados.rados.0.trial178.stdout:390: done (0 left) 2026-02-21T22:17:22.713 INFO:tasks.rados.rados.0.trial178.stdout:391: snap_remove snap 33 2026-02-21T22:17:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:17:23] 2026-02-21T22:17:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:17:23, mode upmap, max misplaced 0.050000 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'unique_pool_0', 'rbd'] 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:17:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:17:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:17:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:17:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:17:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:17:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:17:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:17:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:17:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:17:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:17:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:17:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:17:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:17:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:17:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:17:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:17:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:17:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:17:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:17:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:17:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:17:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:17:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "102", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:17:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:17:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:17:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 102, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 43, 2026-02-21T22:17:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:17:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:17:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00026453512686270257 of space, bias 1.0, pg target 0.07936053805881077 quantized to 32 (current 16) 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:17:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:17:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:17:23.714 INFO:tasks.rados.rados.0.trial178.stdout:391: done (0 left) 2026-02-21T22:17:23.714 INFO:tasks.rados.rados.0.trial178.stdout:392: snap_create 2026-02-21T22:17:24.715 INFO:tasks.rados.rados.0.trial178.stdout:392: done (0 left) 2026-02-21T22:17:24.715 INFO:tasks.rados.rados.0.trial178.stdout:393: read oid 40 snap 40 2026-02-21T22:17:24.715 INFO:tasks.rados.rados.0.trial178.stdout:393: expect (ObjNum 14 snap 0 seq_num 14) 2026-02-21T22:17:24.715 INFO:tasks.rados.rados.0.trial178.stdout:394: write (excl) oid 34 current snap is 43 2026-02-21T22:17:24.715 INFO:tasks.rados.rados.0.trial178.stdout:394: seq_num 156 ranges {621892=465904,1712485=671711} 2026-02-21T22:17:24.722 INFO:tasks.rados.rados.0.trial178.stdout:394: writing trial17818531-34 from 621892 to 1087796 tid 1 2026-02-21T22:17:24.731 INFO:tasks.rados.rados.0.trial178.stdout:394: writing trial17818531-34 from 1712485 to 2384196 tid 2 2026-02-21T22:17:24.731 INFO:tasks.rados.rados.0.trial178.stdout:395: snap_create 2026-02-21T22:17:24.731 INFO:tasks.rados.rados.0.trial178.stdout:393: expect (ObjNum 14 snap 0 seq_num 14) 2026-02-21T22:17:24.737 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 887726 2026-02-21T22:17:24.754 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: finishing write tid 1 to trial17818531-34 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: finishing write tid 2 to trial17818531-34 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: finishing write tid 3 to trial17818531-34 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: finishing write tid 4 to trial17818531-34 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: oid 34 updating version 0 to 61 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: oid 34 version 61 is already newer than 60 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:394: oid 34 version 61 is already newer than 59 2026-02-21T22:17:24.798 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 61 (ObjNum 156 snap 43 seq_num 156) dirty exists 2026-02-21T22:17:24.799 INFO:tasks.rados.rados.0.trial178.stdout:394: left oid 34 (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:17:25.694 INFO:tasks.rados.rados.0.trial178.stdout:393: done (2 left) 2026-02-21T22:17:25.695 INFO:tasks.rados.rados.0.trial178.stdout:394: done (1 left) 2026-02-21T22:17:25.695 INFO:tasks.rados.rados.0.trial178.stdout:395: done (0 left) 2026-02-21T22:17:25.695 INFO:tasks.rados.rados.0.trial178.stdout:396: read oid 34 snap -1 2026-02-21T22:17:25.695 INFO:tasks.rados.rados.0.trial178.stdout:396: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:17:25.695 INFO:tasks.rados.rados.0.trial178.stdout:397: snap_create 2026-02-21T22:17:25.700 INFO:tasks.rados.rados.0.trial178.stdout:396: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:17:25.714 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 482581 2026-02-21T22:17:26.695 INFO:tasks.rados.rados.0.trial178.stdout:396: done (1 left) 2026-02-21T22:17:26.695 INFO:tasks.rados.rados.0.trial178.stdout:397: done (0 left) 2026-02-21T22:17:26.696 INFO:tasks.rados.rados.0.trial178.stdout:398: copy_from oid 44 from oid 25 current snap is 45 2026-02-21T22:17:26.696 INFO:tasks.rados.rados.0.trial178.stdout:399: delete oid 33 current snap is 45 2026-02-21T22:17:26.700 INFO:tasks.rados.rados.0.trial178.stdout:399: done (1 left) 2026-02-21T22:17:26.701 INFO:tasks.rados.rados.0.trial178.stdout:400: write oid 2 current snap is 45 2026-02-21T22:17:26.701 INFO:tasks.rados.rados.0.trial178.stdout:400: seq_num 158 ranges {655506=616402,1829449=675380} 2026-02-21T22:17:26.709 INFO:tasks.rados.rados.0.trial178.stdout:400: writing trial17818531-2 from 655506 to 1271908 tid 1 2026-02-21T22:17:26.717 INFO:tasks.rados.rados.0.trial178.stdout:400: writing trial17818531-2 from 1829449 to 2504829 tid 2 2026-02-21T22:17:26.718 INFO:tasks.rados.rados.0.trial178.stdout:401: snap_create 2026-02-21T22:17:26.718 INFO:tasks.rados.rados.0.trial178.stdout:398: finishing copy_from to trial17818531-44 2026-02-21T22:17:26.718 INFO:tasks.rados.rados.0.trial178.stdout:398: got expected ENOENT (src dne) 2026-02-21T22:17:26.718 INFO:tasks.rados.rados.0.trial178.stdout:398: finishing copy_from racing read to trial17818531-44 2026-02-21T22:17:26.718 INFO:tasks.rados.rados.0.trial178.stdout:400: finishing write tid 1 to trial17818531-2 2026-02-21T22:17:26.720 INFO:tasks.rados.rados.0.trial178.stdout:400: finishing write tid 2 to trial17818531-2 2026-02-21T22:17:26.720 INFO:tasks.rados.rados.0.trial178.stdout:400: finishing write tid 3 to trial17818531-2 2026-02-21T22:17:26.720 INFO:tasks.rados.rados.0.trial178.stdout:400: finishing write tid 4 to trial17818531-2 2026-02-21T22:17:26.720 INFO:tasks.rados.rados.0.trial178.stdout:400: oid 2 updating version 0 to 119 2026-02-21T22:17:26.720 INFO:tasks.rados.rados.0.trial178.stdout:400: oid 2 version 119 is already newer than 118 2026-02-21T22:17:26.721 INFO:tasks.rados.rados.0.trial178.stdout:400: oid 2 updating version 119 to 120 2026-02-21T22:17:26.721 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 120 (ObjNum 158 snap 45 seq_num 158) dirty exists 2026-02-21T22:17:26.721 INFO:tasks.rados.rados.0.trial178.stdout:400: left oid 2 (ObjNum 158 snap 45 seq_num 158) 2026-02-21T22:17:27.696 INFO:tasks.rados.rados.0.trial178.stdout:398: done (2 left) 2026-02-21T22:17:27.697 INFO:tasks.rados.rados.0.trial178.stdout:400: done (1 left) 2026-02-21T22:17:27.697 INFO:tasks.rados.rados.0.trial178.stdout:401: done (0 left) 2026-02-21T22:17:27.697 INFO:tasks.rados.rados.0.trial178.stdout:402: snap_create 2026-02-21T22:17:28.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:28.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:28.697 INFO:tasks.rados.rados.0.trial178.stdout:402: done (0 left) 2026-02-21T22:17:28.697 INFO:tasks.rados.rados.0.trial178.stdout:403: read oid 37 snap -1 2026-02-21T22:17:28.697 INFO:tasks.rados.rados.0.trial178.stdout:403: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:17:28.697 INFO:tasks.rados.rados.0.trial178.stdout:404: write oid 42 current snap is 47 2026-02-21T22:17:28.697 INFO:tasks.rados.rados.0.trial178.stdout:404: seq_num 159 ranges {672313=691651,1887931=419038} 2026-02-21T22:17:28.706 INFO:tasks.rados.rados.0.trial178.stdout:404: writing trial17818531-42 from 672313 to 1363964 tid 1 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:404: writing trial17818531-42 from 1887931 to 2306969 tid 2 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:405: read oid 43 snap -1 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:405: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:406: read oid 50 snap -1 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:406: expect (ObjNum 153 snap 41 seq_num 153) 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:407: write (excl) oid 25 current snap is 47 2026-02-21T22:17:28.713 INFO:tasks.rados.rados.0.trial178.stdout:407: seq_num 160 ranges {689120=483253,1946413=679049} 2026-02-21T22:17:28.720 INFO:tasks.rados.rados.0.trial178.stdout:407: writing trial17818531-25 from 689120 to 1172373 tid 1 2026-02-21T22:17:28.730 INFO:tasks.rados.rados.0.trial178.stdout:407: writing trial17818531-25 from 1946413 to 2625462 tid 2 2026-02-21T22:17:28.730 INFO:tasks.rados.rados.0.trial178.stdout:408: copy_from oid 5 from oid 34 current snap is 47 2026-02-21T22:17:28.731 INFO:tasks.rados.rados.0.trial178.stdout:403: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:17:28.736 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 832821 2026-02-21T22:17:28.752 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:28.795 INFO:tasks.rados.rados.0.trial178.stdout:404: finishing write tid 1 to trial17818531-42 2026-02-21T22:17:28.796 INFO:tasks.rados.rados.0.trial178.stdout:404: finishing write tid 2 to trial17818531-42 2026-02-21T22:17:28.796 INFO:tasks.rados.rados.0.trial178.stdout:404: finishing write tid 3 to trial17818531-42 2026-02-21T22:17:28.796 INFO:tasks.rados.rados.0.trial178.stdout:405: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:17:28.870 INFO:tasks.rados.rados.0.trial178.stdout:406: expect (ObjNum 153 snap 41 seq_num 153) 2026-02-21T22:17:28.900 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1151550 2026-02-21T22:17:28.919 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:28.948 INFO:tasks.rados.rados.0.trial178.stdout:404: finishing write tid 4 to trial17818531-42 2026-02-21T22:17:28.948 INFO:tasks.rados.rados.0.trial178.stdout:404: oid 42 updating version 0 to 185 2026-02-21T22:17:28.948 INFO:tasks.rados.rados.0.trial178.stdout:404: oid 42 version 185 is already newer than 184 2026-02-21T22:17:28.948 INFO:tasks.rados.rados.0.trial178.stdout:404: oid 42 updating version 185 to 186 2026-02-21T22:17:28.948 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 186 (ObjNum 159 snap 47 seq_num 159) dirty exists 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:404: left oid 42 (ObjNum 159 snap 47 seq_num 159) 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:407: finishing write tid 1 to trial17818531-25 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:407: finishing write tid 2 to trial17818531-25 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:407: finishing write tid 3 to trial17818531-25 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:403: done (5 left) 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:404: done (4 left) 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:405: done (3 left) 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:406: done (2 left) 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:409: rollback oid 46 current snap is 47 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 44 2026-02-21T22:17:28.949 INFO:tasks.rados.rados.0.trial178.stdout:407: finishing write tid 4 to trial17818531-25 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:407: oid 25 updating version 0 to 61 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:407: oid 25 version 61 is already newer than 60 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:407: oid 25 version 61 is already newer than 59 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 61 (ObjNum 160 snap 47 seq_num 160) dirty exists 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:407: left oid 25 (ObjNum 160 snap 47 seq_num 160) 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:408: finishing copy_from racing read to trial17818531-5 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:408: finishing copy_from to trial17818531-5 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 61 (ObjNum 156 snap 43 seq_num 156) dirty exists 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:407: done (2 left) 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:408: done (1 left) 2026-02-21T22:17:28.950 INFO:tasks.rados.rados.0.trial178.stdout:410: copy_from oid 5 from oid 25 current snap is 47 2026-02-21T22:17:28.951 INFO:tasks.rados.rados.0.trial178.stdout:411: write (excl) oid 19 current snap is 47 2026-02-21T22:17:28.951 INFO:tasks.rados.rados.0.trial178.stdout:411: seq_num 161 ranges {705927=558502,2004895=422707} 2026-02-21T22:17:28.955 INFO:tasks.rados.rados.0.trial178.stdout:411: writing trial17818531-19 from 705927 to 1264429 tid 1 2026-02-21T22:17:28.963 INFO:tasks.rados.rados.0.trial178.stdout:411: writing trial17818531-19 from 2004895 to 2427602 tid 2 2026-02-21T22:17:28.963 INFO:tasks.rados.rados.0.trial178.stdout:412: snap_remove snap 46 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:412: done (3 left) 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:413: read oid 4 snap -1 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:413: expect (ObjNum 139 snap 35 seq_num 139) 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:414: write (excl) oid 13 current snap is 47 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:409: finishing rollback tid 0 to trial17818531-46 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:409: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:17:29.704 INFO:tasks.rados.rados.0.trial178.stdout:414: seq_num 162 ranges {722734=633751,2063377=659357} 2026-02-21T22:17:29.713 INFO:tasks.rados.rados.0.trial178.stdout:414: writing trial17818531-13 from 722734 to 1356485 tid 1 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:414: writing trial17818531-13 from 2063377 to 2722734 tid 2 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:415: delete oid 17 current snap is 47 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:409: finishing rollback tid 2 to trial17818531-46 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 160 (ObjNum 45 snap 0 seq_num 45) dirty exists 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:411: finishing write tid 1 to trial17818531-19 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:410: finishing copy_from racing read to trial17818531-5 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:410: finishing copy_from to trial17818531-5 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 61 (ObjNum 160 snap 47 seq_num 160) dirty exists 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:411: finishing write tid 2 to trial17818531-19 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:411: finishing write tid 3 to trial17818531-19 2026-02-21T22:17:29.723 INFO:tasks.rados.rados.0.trial178.stdout:411: finishing write tid 4 to trial17818531-19 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:411: oid 19 updating version 0 to 189 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:411: oid 19 version 189 is already newer than 188 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:411: oid 19 version 189 is already newer than 187 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 189 (ObjNum 161 snap 47 seq_num 161) dirty exists 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:411: left oid 19 (ObjNum 161 snap 47 seq_num 161) 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:413: expect (ObjNum 139 snap 35 seq_num 139) 2026-02-21T22:17:29.724 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 597946 2026-02-21T22:17:29.762 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2202162 2026-02-21T22:17:29.763 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 663201 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: finishing write tid 1 to trial17818531-13 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: finishing write tid 2 to trial17818531-13 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: finishing write tid 3 to trial17818531-13 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: finishing write tid 4 to trial17818531-13 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: oid 13 updating version 0 to 167 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: oid 13 updating version 167 to 168 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: oid 13 updating version 168 to 169 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 169 (ObjNum 162 snap 47 seq_num 162) dirty exists 2026-02-21T22:17:29.780 INFO:tasks.rados.rados.0.trial178.stdout:414: left oid 13 (ObjNum 162 snap 47 seq_num 162) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:409: done (5 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:410: done (4 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:411: done (3 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:413: done (2 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:414: done (1 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:415: done (0 left) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:416: read oid 14 snap -1 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:416: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:17:29.781 INFO:tasks.rados.rados.0.trial178.stdout:417: delete oid 7 current snap is 47 2026-02-21T22:17:29.783 INFO:tasks.rados.rados.0.trial178.stdout:417: done (1 left) 2026-02-21T22:17:29.783 INFO:tasks.rados.rados.0.trial178.stdout:418: rollback oid 39 current snap is 47 2026-02-21T22:17:29.783 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 45 2026-02-21T22:17:29.784 INFO:tasks.rados.rados.0.trial178.stdout:419: write oid 45 current snap is 47 2026-02-21T22:17:29.784 INFO:tasks.rados.rados.0.trial178.stdout:419: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2026-02-21T22:17:29.791 INFO:tasks.rados.rados.0.trial178.stdout:419: writing trial17818531-45 from 773155 to 1232653 tid 1 2026-02-21T22:17:29.797 INFO:tasks.rados.rados.0.trial178.stdout:419: writing trial17818531-45 from 1722470 to 2152515 tid 2 2026-02-21T22:17:29.802 INFO:tasks.rados.rados.0.trial178.stdout:419: writing trial17818531-45 from 2648676 to 2773155 tid 3 2026-02-21T22:17:29.802 INFO:tasks.rados.rados.0.trial178.stdout:420: rollback oid 35 current snap is 47 2026-02-21T22:17:29.802 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 35 to 42 2026-02-21T22:17:29.802 INFO:tasks.rados.rados.0.trial178.stdout:416: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:17:29.821 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2088728 2026-02-21T22:17:29.866 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2998741 2026-02-21T22:17:29.867 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:29.899 INFO:tasks.rados.rados.0.trial178.stdout:418: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:17:29.899 INFO:tasks.rados.rados.0.trial178.stdout:418: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:17:29.899 INFO:tasks.rados.rados.0.trial178.stdout:418: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:17:29.899 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 101 (ObjNum 68 snap 7 seq_num 68) dirty exists 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: finishing write tid 1 to trial17818531-45 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: finishing write tid 2 to trial17818531-45 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: finishing write tid 3 to trial17818531-45 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:420: finishing rollback tid 0 to trial17818531-35 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: finishing write tid 4 to trial17818531-45 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: finishing write tid 5 to trial17818531-45 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: oid 45 updating version 0 to 82 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: oid 45 version 82 is already newer than 79 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: oid 45 version 82 is already newer than 81 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:419: oid 45 version 82 is already newer than 80 2026-02-21T22:17:29.900 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 82 (ObjNum 165 snap 47 seq_num 165) dirty exists 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:419: left oid 45 (ObjNum 165 snap 47 seq_num 165) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:420: finishing rollback tid 1 to trial17818531-35 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:420: finishing rollback tid 2 to trial17818531-35 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 173 (ObjNum 122 snap 29 seq_num 122) dirty exists 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:416: done (3 left) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:418: done (2 left) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:419: done (1 left) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:420: done (0 left) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:421: read oid 43 snap -1 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:421: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:422: read oid 33 snap 44 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:422: expect deleted 2026-02-21T22:17:29.901 INFO:tasks.rados.rados.0.trial178.stdout:423: write (excl) oid 35 current snap is 47 2026-02-21T22:17:29.902 INFO:tasks.rados.rados.0.trial178.stdout:423: seq_num 166 ranges {789962=534747,1780952=690056} 2026-02-21T22:17:29.907 INFO:tasks.rados.rados.0.trial178.stdout:423: writing trial17818531-35 from 789962 to 1324709 tid 1 2026-02-21T22:17:29.915 INFO:tasks.rados.rados.0.trial178.stdout:423: writing trial17818531-35 from 1780952 to 2471008 tid 2 2026-02-21T22:17:29.915 INFO:tasks.rados.rados.0.trial178.stdout:424: rollback oid 15 current snap is 47 2026-02-21T22:17:29.915 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 39 2026-02-21T22:17:29.915 INFO:tasks.rados.rados.0.trial178.stdout:421: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:17:30.005 INFO:tasks.rados.rados.0.trial178.stdout:423: finishing write tid 1 to trial17818531-35 2026-02-21T22:17:30.005 INFO:tasks.rados.rados.0.trial178.stdout:423: finishing write tid 2 to trial17818531-35 2026-02-21T22:17:30.005 INFO:tasks.rados.rados.0.trial178.stdout:423: finishing write tid 3 to trial17818531-35 2026-02-21T22:17:30.005 INFO:tasks.rados.rados.0.trial178.stdout:424: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:17:30.005 INFO:tasks.rados.rados.0.trial178.stdout:424: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: finishing write tid 4 to trial17818531-35 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: oid 35 updating version 0 to 174 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: oid 35 updating version 174 to 175 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: oid 35 updating version 175 to 176 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 176 (ObjNum 166 snap 47 seq_num 166) dirty exists 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: left oid 35 (ObjNum 166 snap 47 seq_num 166) 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:424: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 99 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:421: done (3 left) 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:422: done (2 left) 2026-02-21T22:17:30.006 INFO:tasks.rados.rados.0.trial178.stdout:423: done (1 left) 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:424: done (0 left) 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:425: read oid 1 snap -1 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:425: expect (ObjNum 141 snap 36 seq_num 141) 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:426: read oid 24 snap 37 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:426: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:427: copy_from oid 28 from oid 11 current snap is 47 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:428: copy_from oid 48 from oid 36 current snap is 47 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:429: read oid 21 snap -1 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:429: expect deleted 2026-02-21T22:17:30.007 INFO:tasks.rados.rados.0.trial178.stdout:430: snap_remove snap 40 2026-02-21T22:17:30.696 INFO:tasks.rados.rados.0.trial178.stdout:430: done (5 left) 2026-02-21T22:17:30.696 INFO:tasks.rados.rados.0.trial178.stdout:431: snap_remove snap 42 2026-02-21T22:17:31.075 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-21T22:17:31.075 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2026-02-21T22:17:31.076 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T22:17:31.076 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T22:17:31.094 INFO:tasks.ceph.mon.a:Started 2026-02-21T22:17:31.094 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.b 2026-02-21T22:17:31.094 INFO:tasks.mon_thrash:Sending STOP to mon b 2026-02-21T22:17:31.094 INFO:tasks.ceph.mon.b:Sent signal 19 2026-02-21T22:17:31.094 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:17:33.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:33.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:38.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:38.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:43.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:43.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:46.094 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.b 2026-02-21T22:17:46.094 INFO:tasks.mon_thrash:Sending CONT to mon b 2026-02-21T22:17:46.095 INFO:tasks.ceph.mon.b:Sent signal 18 2026-02-21T22:17:46.095 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:17:46.095 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 10 monclient: get_monmap_and_config 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 10 monclient: build_initial_monmap 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 10 monclient: monmap: 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:17:46.155944+0000 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:17:46.155944+0000 2026-02-21T22:17:46.156 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:46.157 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:46.157 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:46.157 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:17:46.157 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:17:46.157 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 1 Processor -- start 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 1 -- start start 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.150+0000 7f12e085b640 10 monclient: init 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-c con 0x7f12dc084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-b con 0x7f12dc086e90 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-a con 0x7f12dc183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc086a70 con 0x7f12dc183350 2026-02-21T22:17:46.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc08b130 con 0x7f12dc084860 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc085340 con 0x7f12dc086e90 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient(hunting): authenticate will time out at 1275.392221s 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 0x7f12dc181c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:35550/0 (socket says 10.20.193.102:35550) 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9d74640 1 -- 10.20.193.102:0/1433115826 learned_addr learned my addr 10.20.193.102:0/1433115826 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9d74640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 legacy=0x7f12dc181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12dad76640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.102:6789/0 conn(0x7f12dc183350 legacy=0x7f12dc1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476728936 0 0) 0x7f12dc085340 con 0x7f12dc086e90 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient(hunting): _negotiate 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:46.163 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient(hunting): my global_id is 4424 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12c8003620 con 0x7f12dc086e90 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12da575640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.178:6789/0 conn(0x7f12dc084860 legacy=0x7f12dc084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 997391633 0 0) 0x7f12c8003620 con 0x7f12dc086e90 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.178:6789/0 conn(0x7f12dc084860 legacy=0x7f12dc084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.102:6789/0 conn(0x7f12dc183350 legacy=0x7f12dc1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: _finish_hunting 0 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 monclient: found mon.noname-b 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc086790 con 0x7f12dc086e90 2026-02-21T22:17:46.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: _finish_auth 0 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: _check_auth_tickets 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f12c4003100 con 0x7f12dc086e90 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: dump: 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:46.165 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 5 monclient: authenticate success, global_id 4424 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f12c40033d0 con 0x7f12dc086e90 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 1 -- 10.20.193.102:0/1433115826 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f12c4004b50 con 0x7f12dc086e90 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12d9573640 10 monclient: dump: 2026-02-21T22:17:46.166 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: get_monmap_and_config success 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 4 set_mon_vals no callback set 2026-02-21T22:17:46.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: shutdown 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- 10.20.193.102:0/1433115826 >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 legacy=0x7f12dc181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- 10.20.193.102:0/1433115826 shutdown_connections 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- 10.20.193.102:0/1433115826 >> 10.20.193.102:0/1433115826 conn(0x7f12dc091a50 msgr2=0x7f12dc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- 10.20.193.102:0/1433115826 shutdown_connections 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- 10.20.193.102:0/1433115826 wait complete. 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: build_initial_monmap 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: monmap: 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:17:46.168 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 Processor -- start 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 1 -- start start 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.154+0000 7f12e085b640 10 monclient: init 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:17:46.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-a con 0x7f12dc084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-c con 0x7f12dc086e90 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient(hunting): picked mon.noname-b con 0x7f12dc183350 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc080620 con 0x7f12dc084860 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc088a70 con 0x7f12dc183350 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12dc0818f0 con 0x7f12dc086e90 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient(hunting): authenticate will time out at 1275.396231s 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12da575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f12dc084860 0x7f12dc131d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51420/0 (socket says 10.20.193.102:51420) 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12da575640 1 -- 10.20.193.102:0/1866284980 learned_addr learned my addr 10.20.193.102:0/1866284980 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:46.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12d9d74640 1 --1- 10.20.193.102:0/1866284980 >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 0x7f12dc172ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:35566/0 (socket says 10.20.193.102:35566) 2026-02-21T22:17:46.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12d9d74640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 legacy=0x7f12dc172ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12da575640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.102:6789/0 conn(0x7f12dc084860 legacy=0x7f12dc131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174254308 0 0) 0x7f12dc0818f0 con 0x7f12dc086e90 2026-02-21T22:17:46.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): my global_id is 4427 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12a8003620 con 0x7f12dc086e90 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12dad76640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.178:6789/0 conn(0x7f12dc183350 legacy=0x7f12dc174710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3199411061 0 0) 0x7f12a8003620 con 0x7f12dc086e90 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f12dc0818f0 con 0x7f12dc086e90 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f12c40048e0 con 0x7f12dc086e90 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): dump: 2026-02-21T22:17:46.174 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:46.175 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1987645647 0 0) 0x7f12dc0818f0 con 0x7f12dc086e90 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.178:6789/0 conn(0x7f12dc183350 legacy=0x7f12dc174710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.102:6789/0 conn(0x7f12dc084860 legacy=0x7f12dc131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 monclient: found mon.c 2026-02-21T22:17:46.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc07eb60 con 0x7f12dc086e90 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 5 monclient: authenticate success, global_id 4427 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _renew_subs 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc080ec0 con 0x7f12dc086e90 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _renew_subs 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f12dc07e0b0 con 0x7f12dc086e90 2026-02-21T22:17:46.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f12c40033d0 con 0x7f12dc086e90 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f12c4005320 con 0x7f12dc086e90 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 10 monclient: dump: 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:46.178 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12d8d72640 4 set_mon_vals no callback set 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc0008e0 con 0x7f12dc086e90 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f12c40119b0 con 0x7f12dc086e90 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.158+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(110..110 src has 1..110) ==== 3987+0+0 (unknown 3133087533 0 0) 0x7f12c404c760 con 0x7f12dc086e90 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.162+0000 7f12da575640 10 monclient: get_auth_request con 0x7f12a803d400 auth_method 0 2026-02-21T22:17:46.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.162+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f12c40176d0 con 0x7f12dc086e90 2026-02-21T22:17:46.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.162+0000 7f12c2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:46.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.162+0000 7f12c2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:17:46.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:17:46.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:46.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:46.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f12dc07e3f0 con 0x7f12dc086e90 2026-02-21T22:17:46.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12c2ffd640 1 -- 10.20.193.102:0/1866284980 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 101310050) 0x7f12c401c980 con 0x7f12dc086e90 2026-02-21T22:17:46.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12c2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:46.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12c2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:17:46.282 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:17:46.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.178:6800/2549949439 conn(0x7f12a803d400 legacy=0x7f12a805d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 10 monclient: shutdown 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 >> v1:10.20.193.102:6790/0 conn(0x7f12dc086e90 legacy=0x7f12dc172ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 shutdown_connections 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 >> 10.20.193.102:0/1866284980 conn(0x7f12dc091a50 msgr2=0x7f12dc08a870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 shutdown_connections 2026-02-21T22:17:46.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:46.278+0000 7f12e085b640 1 -- 10.20.193.102:0/1866284980 wait complete. 2026-02-21T22:17:48.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:48.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:49.292 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:17:49.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: get_monmap_and_config 2026-02-21T22:17:49.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: build_initial_monmap 2026-02-21T22:17:49.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: monmap: 2026-02-21T22:17:49.364 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:49.364 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:17:49.363884+0000 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:17:49.363884+0000 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 1 Processor -- start 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 1 -- start start 2026-02-21T22:17:49.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: init 2026-02-21T22:17:49.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:49.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-c con 0x7f897c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-a con 0x7f897c086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-b con 0x7f897c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c086a70 con 0x7f897c086e90 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.358+0000 7f8983b5b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c08b130 con 0x7f897c084860 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c085340 con 0x7f897c183350 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:49.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient(hunting): authenticate will time out at 1278.600011s 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89810cf640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 0x7f897c181c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51432/0 (socket says 10.20.193.102:51432) 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89810cf640 1 -- 10.20.193.102:0/2121040260 learned_addr learned my addr 10.20.193.102:0/2121040260 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89820d1640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.102:6790/0 conn(0x7f897c183350 legacy=0x7f897c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89810cf640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 legacy=0x7f897c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89818d0640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.178:6789/0 conn(0x7f897c084860 legacy=0x7f897c084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231505552 0 0) 0x7f897c085340 con 0x7f897c183350 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _negotiate 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): my global_id is 4430 2026-02-21T22:17:49.372 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8964003620 con 0x7f897c183350 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074965392 0 0) 0x7f897c086a70 con 0x7f897c086e90 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _negotiate 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient(hunting): my global_id is 34098 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f897c085340 con 0x7f897c086e90 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893840726 0 0) 0x7f8964003620 con 0x7f897c183350 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.178:6789/0 conn(0x7f897c084860 legacy=0x7f897c084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 legacy=0x7f897c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: _finish_hunting 0 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 monclient: found mon.noname-b 2026-02-21T22:17:49.373 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c086790 con 0x7f897c183350 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: _finish_auth 0 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: _check_auth_tickets 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8978004660 con 0x7f897c183350 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: dump: 2026-02-21T22:17:49.374 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 5 monclient: authenticate success, global_id 4430 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8978004930 con 0x7f897c183350 2026-02-21T22:17:49.375 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 1 -- 10.20.193.102:0/2121040260 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8978004dd0 con 0x7f897c183350 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f89808ce640 10 monclient: dump: 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:49.376 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient: get_monmap_and_config success 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 4 set_mon_vals no callback set 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient: shutdown 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- 10.20.193.102:0/2121040260 >> v1:10.20.193.102:6790/0 conn(0x7f897c183350 legacy=0x7f897c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- 10.20.193.102:0/2121040260 shutdown_connections 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- 10.20.193.102:0/2121040260 >> 10.20.193.102:0/2121040260 conn(0x7f897c091a50 msgr2=0x7f897c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- 10.20.193.102:0/2121040260 shutdown_connections 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 1 -- 10.20.193.102:0/2121040260 wait complete. 2026-02-21T22:17:49.377 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient: build_initial_monmap 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.362+0000 7f8983b5b640 10 monclient: monmap: 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 Processor -- start 2026-02-21T22:17:49.378 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- start start 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: init 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-b con 0x7f897c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-a con 0x7f897c086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient(hunting): picked mon.noname-c con 0x7f897c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c080b20 con 0x7f897c086e90 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c080e00 con 0x7f897c084860 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897c0876c0 con 0x7f897c183350 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient(hunting): authenticate will time out at 1278.604659s 2026-02-21T22:17:49.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f89810cf640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 0x7f897c172f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51436/0 (socket says 10.20.193.102:51436) 2026-02-21T22:17:49.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f89810cf640 1 -- 10.20.193.102:0/3199501153 learned_addr learned my addr 10.20.193.102:0/3199501153 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f89810cf640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 legacy=0x7f897c172f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f89820d1640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.102:6790/0 conn(0x7f897c183350 legacy=0x7f897c174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f89818d0640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.178:6789/0 conn(0x7f897c084860 legacy=0x7f897c131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2908042368 0 0) 0x7f897c080b20 con 0x7f897c086e90 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:17:49.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): my global_id is 34101 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f894c003620 con 0x7f897c086e90 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751205706 0 0) 0x7f897c0876c0 con 0x7f897c183350 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): my global_id is 4433 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f897c080b20 con 0x7f897c183350 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4087132474 0 0) 0x7f894c003620 con 0x7f897c086e90 2026-02-21T22:17:49.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f897c0876c0 con 0x7f897c086e90 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1590096050 0 0) 0x7f897c080b20 con 0x7f897c183350 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f894c003620 con 0x7f897c183350 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8978004dd0 con 0x7f897c183350 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): dump: 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.385 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f89700048e0 con 0x7f897c086e90 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient(hunting): dump: 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.386 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2255954051 0 0) 0x7f897c0876c0 con 0x7f897c086e90 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.102:6790/0 conn(0x7f897c183350 legacy=0x7f897c174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.178:6789/0 conn(0x7f897c084860 legacy=0x7f897c131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 monclient: found mon.a 2026-02-21T22:17:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c05e160 con 0x7f897c086e90 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 5 monclient: authenticate success, global_id 34101 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _renew_subs 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f897c080840 con 0x7f897c086e90 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _renew_subs 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f897c089dc0 con 0x7f897c086e90 2026-02-21T22:17:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8970004d30 con 0x7f897c086e90 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8970005090 con 0x7f897c086e90 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896bfff640 4 set_mon_vals no callback set 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f896a7fc640 10 monclient: dump: 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:49.389 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.366+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c0008e0 con 0x7f897c086e90 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8970011720 con 0x7f897c086e90 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(111..111 src has 1..111) ==== 3915+0+0 (unknown 3815281235 0 0) 0x7f897004c820 con 0x7f897c086e90 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f89818d0640 10 monclient: get_auth_request con 0x7f894c03d600 auth_method 0 2026-02-21T22:17:49.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f89700177d0 con 0x7f897c086e90 2026-02-21T22:17:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f896a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.370+0000 7f896a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f897c08c6b0 con 0x7f897c086e90 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f896a7fc640 1 -- 10.20.193.102:0/3199501153 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 1055872706) 0x7f89700170b0 con 0x7f897c086e90 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f896a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:49.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f896a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:17:49.494 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.178:6800/2549949439 conn(0x7f894c03d600 legacy=0x7f894c05dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 10 monclient: shutdown 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 >> v1:10.20.193.102:6789/0 conn(0x7f897c086e90 legacy=0x7f897c172f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 shutdown_connections 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 >> 10.20.193.102:0/3199501153 conn(0x7f897c091a50 msgr2=0x7f897c087c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 shutdown_connections 2026-02-21T22:17:49.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:49.490+0000 7f8983b5b640 1 -- 10.20.193.102:0/3199501153 wait complete. 2026-02-21T22:17:52.504 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:17:52.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: get_monmap_and_config 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: build_initial_monmap 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: monmap: 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:17:52.575598+0000 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:17:52.575598+0000 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:52.577 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 1 Processor -- start 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 1 -- start start 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: init 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient(hunting): picked mon.noname-c con 0x7f680808c4e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:52.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient(hunting): picked mon.noname-a con 0x7f680808d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient(hunting): picked mon.noname-b con 0x7f680808d890 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680808bd50 con 0x7f680808d410 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680808c8b0 con 0x7f680808c4e0 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6808083560 con 0x7f680808d890 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680e943640 10 monclient(hunting): authenticate will time out at 1281.811573s 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680ceb9640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f680808d890 0x7f68081aa1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38966/0 (socket says 10.20.193.102:38966) 2026-02-21T22:17:52.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.570+0000 7f680ceb9640 1 -- 10.20.193.102:0/529751615 learned_addr learned my addr 10.20.193.102:0/529751615 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:52.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f680ceb9640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.102:6790/0 conn(0x7f680808d890 legacy=0x7f68081aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f68077fe640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.102:6789/0 conn(0x7f680808d410 legacy=0x7f6808188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6807fff640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.178:6789/0 conn(0x7f680808c4e0 legacy=0x7f680808cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626644871 0 0) 0x7f680808bd50 con 0x7f680808d410 2026-02-21T22:17:52.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:17:52.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 10 monclient(hunting): my global_id is 34104 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.574+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67ec003620 con 0x7f680808d410 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3865756874 0 0) 0x7f67ec003620 con 0x7f680808d410 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.102:6790/0 conn(0x7f680808d890 legacy=0x7f68081aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.178:6789/0 conn(0x7f680808c4e0 legacy=0x7f680808cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 monclient: found mon.noname-a 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:52.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808084bc0 con 0x7f680808d410 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: _finish_auth 0 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67f8003160 con 0x7f680808d410 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 5 monclient: authenticate success, global_id 34104 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:17:52.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: dump: 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:17:52.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67f8003430 con 0x7f680808d410 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 1 -- 10.20.193.102:0/529751615 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67f80038c0 con 0x7f680808d410 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6806ffd640 10 monclient: dump: 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:52.587 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: get_monmap_and_config success 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 4 set_mon_vals no callback set 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: shutdown 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- 10.20.193.102:0/529751615 >> v1:10.20.193.102:6789/0 conn(0x7f680808d410 legacy=0x7f6808188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- 10.20.193.102:0/529751615 shutdown_connections 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- 10.20.193.102:0/529751615 >> 10.20.193.102:0/529751615 conn(0x7f6808091a50 msgr2=0x7f6808091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- 10.20.193.102:0/529751615 shutdown_connections 2026-02-21T22:17:52.588 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- 10.20.193.102:0/529751615 wait complete. 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: build_initial_monmap 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: monmap: 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:17:52.589 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 Processor -- start 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- start start 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: init 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient(hunting): picked mon.noname-c con 0x7f680808c4e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient(hunting): picked mon.noname-b con 0x7f680808d410 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient(hunting): picked mon.noname-a con 0x7f680808d890 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68081bfe40 con 0x7f680808d890 2026-02-21T22:17:52.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680805ceb0 con 0x7f680808d410 2026-02-21T22:17:52.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68081dc150 con 0x7f680808c4e0 2026-02-21T22:17:52.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:52.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f680e943640 10 monclient(hunting): authenticate will time out at 1281.818212s 2026-02-21T22:17:52.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6807fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f680808c4e0 0x7f68081dba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38976/0 (socket says 10.20.193.102:38976) 2026-02-21T22:17:52.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.578+0000 7f6807fff640 1 -- 10.20.193.102:0/2484509068 learned_addr learned my addr 10.20.193.102:0/2484509068 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:52.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6807fff640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.102:6790/0 conn(0x7f680808c4e0 legacy=0x7f68081dba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680ceb9640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.102:6789/0 conn(0x7f680808d890 legacy=0x7f6808241220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674297590 0 0) 0x7f68081bfe40 con 0x7f680808d890 2026-02-21T22:17:52.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): my global_id is 34107 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67d8003620 con 0x7f680808d890 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f68077fe640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.178:6789/0 conn(0x7f680808d410 legacy=0x7f680821faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 656735441 0 0) 0x7f67d8003620 con 0x7f680808d890 2026-02-21T22:17:52.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f68081bfe40 con 0x7f680808d890 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67fc004840 con 0x7f680808d890 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): dump: 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:52.596 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1091469903 0 0) 0x7f68081bfe40 con 0x7f680808d890 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.102:6790/0 conn(0x7f680808c4e0 legacy=0x7f68081dba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.178:6789/0 conn(0x7f680808d410 legacy=0x7f680821faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 monclient: found mon.a 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808085fa0 con 0x7f680808d890 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: _finish_auth 0 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 5 monclient: authenticate success, global_id 34107 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _renew_subs 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f68081bf810 con 0x7f680808d890 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _renew_subs 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f68081be8b0 con 0x7f680808d890 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67fc0060a0 con 0x7f680808d890 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:52.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67fc006470 con 0x7f680808d890 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 10 monclient: dump: 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:52.599 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f68067fc640 4 set_mon_vals no callback set 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68080008e0 con 0x7f680808d890 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f67fc006740 con 0x7f680808d890 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(111..111 src has 1..111) ==== 3915+0+0 (unknown 3815281235 0 0) 0x7f67fc04dad0 con 0x7f680808d890 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.582+0000 7f6807fff640 10 monclient: get_auth_request con 0x7f67d803d450 auth_method 0 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.586+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f6808083560 con 0x7f680808d890 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.586+0000 7f6804ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:52.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.586+0000 7f6804ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f680e943640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f680e943640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f680e943640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6808090810 con 0x7f680808d890 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f6804ff9640 1 -- 10.20.193.102:0/2484509068 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 2009386786) 0x7f67fc018a90 con 0x7f680808d890 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f6804ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:52.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.698+0000 7f6804ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:17:52.706 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":12,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.178:6800/2549949439 conn(0x7f67d803d450 legacy=0x7f67d805d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 10 monclient: shutdown 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 >> v1:10.20.193.102:6789/0 conn(0x7f680808d890 legacy=0x7f6808241220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 shutdown_connections 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 >> 10.20.193.102:0/2484509068 conn(0x7f6808091a50 msgr2=0x7f680805e100 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 shutdown_connections 2026-02-21T22:17:52.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:52.702+0000 7f680e943640 1 -- 10.20.193.102:0/2484509068 wait complete. 2026-02-21T22:17:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:53.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:53.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:17:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:17:55.715 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:17:55.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: get_monmap_and_config 2026-02-21T22:17:55.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: build_initial_monmap 2026-02-21T22:17:55.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: monmap: 2026-02-21T22:17:55.779 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:55.779 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:17:55.778561+0000 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:17:55.778561+0000 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 1 Processor -- start 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 1 -- start start 2026-02-21T22:17:55.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: init 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-c con 0x7f103c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-b con 0x7f103c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-a con 0x7f103c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c086a10 con 0x7f103c08d8e0 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c08b130 con 0x7f103c08ce00 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c08bd60 con 0x7f103c08b7f0 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f1043c6a640 10 monclient(hunting): authenticate will time out at 1285.014449s 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f10411de640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f103c08b7f0 0x7f103c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38990/0 (socket says 10.20.193.102:38990) 2026-02-21T22:17:55.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.774+0000 7f10411de640 1 -- 10.20.193.102:0/3227986433 learned_addr learned my addr 10.20.193.102:0/3227986433 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:55.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.778+0000 7f10411de640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.102:6790/0 conn(0x7f103c08b7f0 legacy=0x7f103c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:55.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.778+0000 7f10421e0640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.102:6789/0 conn(0x7f103c08d8e0 legacy=0x7f103c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:55.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:55.778+0000 7f10419df640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.178:6789/0 conn(0x7f103c08ce00 legacy=0x7f103c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31850299 0 0) 0x7f103c08bd60 con 0x7f103c08b7f0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient(hunting): _negotiate 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient(hunting): my global_id is 4436 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1024003620 con 0x7f103c08b7f0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 69074441 0 0) 0x7f1024003620 con 0x7f103c08b7f0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.178:6789/0 conn(0x7f103c08ce00 legacy=0x7f103c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.102:6789/0 conn(0x7f103c08d8e0 legacy=0x7f103c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: _finish_hunting 0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 monclient: found mon.noname-b 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c08b370 con 0x7f103c08b7f0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: _finish_auth 0 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: _check_auth_tickets 2026-02-21T22:17:57.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1030003110 con 0x7f103c08b7f0 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: dump: 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 5 monclient: authenticate success, global_id 4436 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f10300033e0 con 0x7f103c08b7f0 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 1 -- 10.20.193.102:0/3227986433 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1030004b50 con 0x7f103c08b7f0 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10409dd640 10 monclient: dump: 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: get_monmap_and_config success 2026-02-21T22:17:57.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 4 set_mon_vals no callback set 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: shutdown 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- 10.20.193.102:0/3227986433 >> v1:10.20.193.102:6790/0 conn(0x7f103c08b7f0 legacy=0x7f103c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- 10.20.193.102:0/3227986433 shutdown_connections 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- 10.20.193.102:0/3227986433 >> 10.20.193.102:0/3227986433 conn(0x7f103c091a50 msgr2=0x7f103c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- 10.20.193.102:0/3227986433 shutdown_connections 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- 10.20.193.102:0/3227986433 wait complete. 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: build_initial_monmap 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: monmap: 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:17:57.171 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 Processor -- start 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- start start 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: init 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: _reopen_session rank -1 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-a con 0x7f103c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-c con 0x7f103c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient(hunting): picked mon.noname-b con 0x7f103c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c080670 con 0x7f103c08b7f0 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c083620 con 0x7f103c08d8e0 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f103c05e3a0 con 0x7f103c08ce00 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient(hunting): _renew_subs 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f1043c6a640 10 monclient(hunting): authenticate will time out at 1286.407015s 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10419df640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f103c08b7f0 0x7f103c07f850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55512/0 (socket says 10.20.193.102:55512) 2026-02-21T22:17:57.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.166+0000 7f10419df640 1 -- 10.20.193.102:0/1254133919 learned_addr learned my addr 10.20.193.102:0/1254133919 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:17:57.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f10419df640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.102:6789/0 conn(0x7f103c08b7f0 legacy=0x7f103c07f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:57.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f10411de640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.102:6790/0 conn(0x7f103c08ce00 legacy=0x7f103c07ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:57.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f10421e0640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.178:6789/0 conn(0x7f103c08d8e0 legacy=0x7f103c1956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969194679 0 0) 0x7f103c05e3a0 con 0x7f103c08ce00 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): my global_id is 4439 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f100c003620 con 0x7f103c08ce00 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f10421e0640 1 --1- 10.20.193.102:0/1254133919 >> v1:10.20.193.178:6789/0 conn(0x7f103c08d8e0 0x7f103c1956b0 :-1 s=OPENED pgs=181 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f103c083620 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1146292781 0 0) 0x7f100c003620 con 0x7f103c08ce00 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f103c05e3a0 con 0x7f103c08ce00 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f10300048e0 con 0x7f103c08ce00 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): dump: 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:57.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2200925371 0 0) 0x7f103c05e3a0 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.178:6789/0 conn(0x7f103c08d8e0 legacy=0x7f103c1956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.102:6789/0 conn(0x7f103c08b7f0 legacy=0x7f103c07f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 monclient: found mon.c 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c08a3f0 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106959334 0 0) 0x7f103c083620 con 0x7f103c08d8e0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 5 monclient: authenticate success, global_id 4439 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _renew_subs 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f103c083060 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _renew_subs 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f103c0819a0 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c0008e0 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f10300033e0 con 0x7f103c08ce00 2026-02-21T22:17:57.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1030005190 con 0x7f103c08ce00 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 10 monclient: dump: 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102bfff640 4 set_mon_vals no callback set 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1030011820 con 0x7f103c08ce00 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(111..111 src has 1..111) ==== 3915+0+0 (unknown 3815281235 0 0) 0x7f103004c640 con 0x7f103c08ce00 2026-02-21T22:17:57.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.170+0000 7f10419df640 10 monclient: get_auth_request con 0x7f100c03d410 auth_method 0 2026-02-21T22:17:57.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.174+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f1030017650 con 0x7f103c08ce00 2026-02-21T22:17:57.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.174+0000 7f102a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:17:57.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.174+0000 7f102a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:17:57.225 INFO:tasks.rados.rados.0.trial178.stdout:431: done (5 left) 2026-02-21T22:17:57.225 INFO:tasks.rados.rados.0.trial178.stdout:432: rollback oid 47 current snap is 47 2026-02-21T22:17:57.225 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 44 2026-02-21T22:17:57.225 INFO:tasks.rados.rados.0.trial178.stdout:425: expect (ObjNum 141 snap 36 seq_num 141) 2026-02-21T22:17:57.230 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 752806 2026-02-21T22:17:57.288 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:57.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f103c090810 con 0x7f103c08ce00 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f102a7fc640 1 -- 10.20.193.102:0/1254133919 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 368210733) 0x7f1030016f30 con 0x7f103c08ce00 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f102a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:17:57.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f102a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:17:57.300 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:17:57.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.178:6800/2549949439 conn(0x7f100c03d410 legacy=0x7f100c05d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 10 monclient: shutdown 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 >> v1:10.20.193.102:6790/0 conn(0x7f103c08ce00 legacy=0x7f103c07ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 shutdown_connections 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 >> 10.20.193.102:0/1254133919 conn(0x7f103c091a50 msgr2=0x7f103c089570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 shutdown_connections 2026-02-21T22:17:57.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:17:57.294+0000 7f1043c6a640 1 -- 10.20.193.102:0/1254133919 wait complete. 2026-02-21T22:17:57.312 INFO:tasks.rados.rados.0.trial178.stdout:426: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:17:57.312 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:57.337 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:57.387 INFO:tasks.rados.rados.0.trial178.stdout:428: finishing copy_from racing read to trial17818531-48 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:427: finishing copy_from racing read to trial17818531-28 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:428: finishing copy_from to trial17818531-48 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 33 (ObjNum 117 snap 29 seq_num 117) dirty exists 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:427: finishing copy_from to trial17818531-28 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 116 (ObjNum 114 snap 27 seq_num 114) dirty exists 2026-02-21T22:17:57.388 INFO:tasks.rados.rados.0.trial178.stdout:425: done (5 left) 2026-02-21T22:17:57.389 INFO:tasks.rados.rados.0.trial178.stdout:426: done (4 left) 2026-02-21T22:17:57.389 INFO:tasks.rados.rados.0.trial178.stdout:427: done (3 left) 2026-02-21T22:17:57.389 INFO:tasks.rados.rados.0.trial178.stdout:428: done (2 left) 2026-02-21T22:17:57.389 INFO:tasks.rados.rados.0.trial178.stdout:429: done (1 left) 2026-02-21T22:17:57.389 INFO:tasks.rados.rados.0.trial178.stdout:433: snap_create 2026-02-21T22:17:57.390 INFO:tasks.rados.rados.0.trial178.stdout:432: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:17:57.390 INFO:tasks.rados.rados.0.trial178.stdout:432: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:17:57.390 INFO:tasks.rados.rados.0.trial178.stdout:432: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:17:57.390 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 203 (ObjNum 123 snap 29 seq_num 123) dirty exists 2026-02-21T22:17:58.270 INFO:tasks.rados.rados.0.trial178.stdout:432: done (1 left) 2026-02-21T22:17:58.270 INFO:tasks.rados.rados.0.trial178.stdout:433: done (0 left) 2026-02-21T22:17:58.270 INFO:tasks.rados.rados.0.trial178.stdout:434: snap_remove snap 37 2026-02-21T22:17:58.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:17:58.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:17:59.231 INFO:tasks.rados.rados.0.trial178.stdout:434: done (0 left) 2026-02-21T22:17:59.231 INFO:tasks.rados.rados.0.trial178.stdout:435: copy_from oid 30 from oid 17 current snap is 48 2026-02-21T22:17:59.231 INFO:tasks.rados.rados.0.trial178.stdout:436: write oid 8 current snap is 48 2026-02-21T22:17:59.231 INFO:tasks.rados.rados.0.trial178.stdout:436: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2026-02-21T22:17:59.238 INFO:tasks.rados.rados.0.trial178.stdout:436: writing trial17818531-8 from 406769 to 1016765 tid 1 2026-02-21T22:17:59.244 INFO:tasks.rados.rados.0.trial178.stdout:436: writing trial17818531-8 from 1439434 to 1873148 tid 2 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:436: writing trial17818531-8 from 2303522 to 2806769 tid 3 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:437: rollback oid 3 current snap is 48 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:435: finishing copy_from to trial17818531-30 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:435: got expected ENOENT (src dne) 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:435: finishing copy_from racing read to trial17818531-30 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:436: finishing write tid 1 to trial17818531-8 2026-02-21T22:17:59.251 INFO:tasks.rados.rados.0.trial178.stdout:436: finishing write tid 2 to trial17818531-8 2026-02-21T22:17:59.252 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 39 2026-02-21T22:17:59.252 INFO:tasks.rados.rados.0.trial178.stdout:435: done (2 left) 2026-02-21T22:17:59.252 INFO:tasks.rados.rados.0.trial178.stdout:438: write oid 31 current snap is 48 2026-02-21T22:17:59.252 INFO:tasks.rados.rados.0.trial178.stdout:438: seq_num 168 ranges {423576=401598,1497916=693725} 2026-02-21T22:17:59.256 INFO:tasks.rados.rados.0.trial178.stdout:438: writing trial17818531-31 from 423576 to 825174 tid 1 2026-02-21T22:17:59.266 INFO:tasks.rados.rados.0.trial178.stdout:438: writing trial17818531-31 from 1497916 to 2191641 tid 2 2026-02-21T22:17:59.266 INFO:tasks.rados.rados.0.trial178.stdout:439: read oid 27 snap -1 2026-02-21T22:17:59.266 INFO:tasks.rados.rados.0.trial178.stdout:439: expect (ObjNum 115 snap 27 seq_num 115) 2026-02-21T22:17:59.266 INFO:tasks.rados.rados.0.trial178.stdout:440: read oid 42 snap -1 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:440: expect (ObjNum 159 snap 47 seq_num 159) 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:441: read oid 16 snap 43 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:441: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:442: delete oid 41 current snap is 48 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: finishing write tid 3 to trial17818531-8 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: finishing write tid 4 to trial17818531-8 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: finishing write tid 5 to trial17818531-8 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: oid 8 updating version 0 to 222 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: oid 8 version 222 is already newer than 219 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: oid 8 version 222 is already newer than 220 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: oid 8 version 222 is already newer than 221 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 222 (ObjNum 167 snap 48 seq_num 167) dirty exists 2026-02-21T22:17:59.267 INFO:tasks.rados.rados.0.trial178.stdout:436: left oid 8 (ObjNum 167 snap 48 seq_num 167) 2026-02-21T22:17:59.268 INFO:tasks.rados.rados.0.trial178.stdout:437: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:17:59.268 INFO:tasks.rados.rados.0.trial178.stdout:437: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:17:59.268 INFO:tasks.rados.rados.0.trial178.stdout:437: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:17:59.268 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 192 (ObjNum 101 snap 24 seq_num 101) dirty exists 2026-02-21T22:17:59.268 INFO:tasks.rados.rados.0.trial178.stdout:438: finishing write tid 1 to trial17818531-31 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: finishing write tid 2 to trial17818531-31 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: finishing write tid 3 to trial17818531-31 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: finishing write tid 4 to trial17818531-31 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: oid 31 updating version 0 to 65 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: oid 31 version 65 is already newer than 63 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: oid 31 version 65 is already newer than 64 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 65 (ObjNum 168 snap 48 seq_num 168) dirty exists 2026-02-21T22:17:59.269 INFO:tasks.rados.rados.0.trial178.stdout:438: left oid 31 (ObjNum 168 snap 48 seq_num 168) 2026-02-21T22:17:59.270 INFO:tasks.rados.rados.0.trial178.stdout:441: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:17:59.270 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:59.300 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 434600 2026-02-21T22:17:59.339 INFO:tasks.rados.rados.0.trial178.stdout:439: expect (ObjNum 115 snap 27 seq_num 115) 2026-02-21T22:17:59.357 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2046883 2026-02-21T22:17:59.437 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:59.490 INFO:tasks.rados.rados.0.trial178.stdout:440: expect (ObjNum 159 snap 47 seq_num 159) 2026-02-21T22:17:59.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:59.519 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:59.548 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:17:59.577 INFO:tasks.rados.rados.0.trial178.stdout:436: done (6 left) 2026-02-21T22:17:59.577 INFO:tasks.rados.rados.0.trial178.stdout:437: done (5 left) 2026-02-21T22:17:59.577 INFO:tasks.rados.rados.0.trial178.stdout:438: done (4 left) 2026-02-21T22:17:59.577 INFO:tasks.rados.rados.0.trial178.stdout:439: done (3 left) 2026-02-21T22:17:59.578 INFO:tasks.rados.rados.0.trial178.stdout:440: done (2 left) 2026-02-21T22:17:59.578 INFO:tasks.rados.rados.0.trial178.stdout:441: done (1 left) 2026-02-21T22:17:59.578 INFO:tasks.rados.rados.0.trial178.stdout:442: done (0 left) 2026-02-21T22:17:59.578 INFO:tasks.rados.rados.0.trial178.stdout:443: write (excl) oid 5 current snap is 48 2026-02-21T22:17:59.578 INFO:tasks.rados.rados.0.trial178.stdout:443: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2026-02-21T22:17:59.583 INFO:tasks.rados.rados.0.trial178.stdout:443: writing trial17818531-5 from 457190 to 1009286 tid 1 2026-02-21T22:17:59.591 INFO:tasks.rados.rados.0.trial178.stdout:443: writing trial17818531-5 from 1614880 to 2195921 tid 2 2026-02-21T22:17:59.596 INFO:tasks.rados.rados.0.trial178.stdout:443: writing trial17818531-5 from 2785791 to 2857190 tid 3 2026-02-21T22:17:59.596 INFO:tasks.rados.rados.0.trial178.stdout:444: copy_from oid 17 from oid 10 current snap is 48 2026-02-21T22:17:59.596 INFO:tasks.rados.rados.0.trial178.stdout:443: finishing write tid 1 to trial17818531-5 2026-02-21T22:17:59.596 INFO:tasks.rados.rados.0.trial178.stdout:443: finishing write tid 2 to trial17818531-5 2026-02-21T22:17:59.596 INFO:tasks.rados.rados.0.trial178.stdout:445: delete oid 31 current snap is 48 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: finishing write tid 3 to trial17818531-5 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: finishing write tid 4 to trial17818531-5 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: finishing write tid 5 to trial17818531-5 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: oid 5 updating version 0 to 117 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: oid 5 version 117 is already newer than 116 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: oid 5 version 117 is already newer than 115 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: oid 5 updating version 117 to 118 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 118 (ObjNum 170 snap 48 seq_num 170) dirty exists 2026-02-21T22:17:59.599 INFO:tasks.rados.rados.0.trial178.stdout:443: left oid 5 (ObjNum 170 snap 48 seq_num 170) 2026-02-21T22:17:59.600 INFO:tasks.rados.rados.0.trial178.stdout:443: done (2 left) 2026-02-21T22:17:59.600 INFO:tasks.rados.rados.0.trial178.stdout:445: done (1 left) 2026-02-21T22:17:59.600 INFO:tasks.rados.rados.0.trial178.stdout:446: write oid 11 current snap is 48 2026-02-21T22:17:59.600 INFO:tasks.rados.rados.0.trial178.stdout:446: seq_num 172 ranges {490804=702594,1615491=584710,2724284=166520} 2026-02-21T22:17:59.607 INFO:tasks.rados.rados.0.trial178.stdout:446: writing trial17818531-11 from 490804 to 1193398 tid 1 2026-02-21T22:17:59.614 INFO:tasks.rados.rados.0.trial178.stdout:446: writing trial17818531-11 from 1615491 to 2200201 tid 2 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:446: writing trial17818531-11 from 2724284 to 2890804 tid 3 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:447: snap_create 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:444: finishing copy_from racing read to trial17818531-17 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:444: finishing copy_from to trial17818531-17 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 69 (ObjNum 115 snap 27 seq_num 115) dirty exists 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:446: finishing write tid 1 to trial17818531-11 2026-02-21T22:17:59.619 INFO:tasks.rados.rados.0.trial178.stdout:446: finishing write tid 2 to trial17818531-11 2026-02-21T22:17:59.620 INFO:tasks.rados.rados.0.trial178.stdout:446: finishing write tid 3 to trial17818531-11 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: finishing write tid 4 to trial17818531-11 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: finishing write tid 5 to trial17818531-11 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: oid 11 updating version 0 to 197 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: oid 11 version 197 is already newer than 195 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: oid 11 version 197 is already newer than 194 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: oid 11 version 197 is already newer than 196 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 197 (ObjNum 172 snap 48 seq_num 172) dirty exists 2026-02-21T22:17:59.621 INFO:tasks.rados.rados.0.trial178.stdout:446: left oid 11 (ObjNum 172 snap 48 seq_num 172) 2026-02-21T22:18:00.310 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: get_monmap_and_config 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: monmap: 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:00.379753+0000 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:00.379753+0000 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.379 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 1 Processor -- start 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 1 -- start start 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: init 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-c con 0x7f88e013f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-a con 0x7f88e013fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-b con 0x7f88e01401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e0092490 con 0x7f88e013fd90 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e00571f0 con 0x7f88e013f9c0 2026-02-21T22:18:00.380 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e00574d0 con 0x7f88e01401f0 2026-02-21T22:18:00.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5c75640 10 monclient(hunting): authenticate will time out at 1289.615670s 2026-02-21T22:18:00.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88d7fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f88e013fd90 0x7f88e0183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55516/0 (socket says 10.20.193.102:55516) 2026-02-21T22:18:00.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88d7fff640 1 -- 10.20.193.102:0/3443718116 learned_addr learned my addr 10.20.193.102:0/3443718116 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.381 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.374+0000 7f88e5474640 1 --1- 10.20.193.102:0/3443718116 >> v1:10.20.193.102:6790/0 conn(0x7f88e01401f0 0x7f88e01a5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39002/0 (socket says 10.20.193.102:39002) 2026-02-21T22:18:00.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5474640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.102:6790/0 conn(0x7f88e01401f0 legacy=0x7f88e01a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d7fff640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.102:6789/0 conn(0x7f88e013fd90 legacy=0x7f88e0183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e4c73640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.178:6789/0 conn(0x7f88e013f9c0 legacy=0x7f88e00906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090822721 0 0) 0x7f88e00574d0 con 0x7f88e01401f0 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): my global_id is 4442 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88cc003620 con 0x7f88e01401f0 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196029135 0 0) 0x7f88e0092490 con 0x7f88e013fd90 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient(hunting): my global_id is 34110 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88e00574d0 con 0x7f88e013fd90 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2827306121 0 0) 0x7f88cc003620 con 0x7f88e01401f0 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.178:6789/0 conn(0x7f88e013f9c0 legacy=0x7f88e00906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.102:6789/0 conn(0x7f88e013fd90 legacy=0x7f88e0183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 monclient: found mon.noname-b 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e00577b0 con 0x7f88e01401f0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88dc004620 con 0x7f88e01401f0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 5 monclient: authenticate success, global_id 4442 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: dump: 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f88dc0048f0 con 0x7f88e01401f0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 1 -- 10.20.193.102:0/3443718116 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88dc005fa0 con 0x7f88e01401f0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88d77fe640 10 monclient: dump: 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 4 set_mon_vals no callback set 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 10 monclient: shutdown 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- 10.20.193.102:0/3443718116 >> v1:10.20.193.102:6790/0 conn(0x7f88e01401f0 legacy=0x7f88e01a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- 10.20.193.102:0/3443718116 shutdown_connections 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- 10.20.193.102:0/3443718116 >> 10.20.193.102:0/3443718116 conn(0x7f88e005e3f0 msgr2=0x7f88e0095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- 10.20.193.102:0/3443718116 shutdown_connections 2026-02-21T22:18:00.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- 10.20.193.102:0/3443718116 wait complete. 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 10 monclient: monmap: 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 Processor -- start 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 1 -- start start 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.378+0000 7f88e5c75640 10 monclient: init 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-a con 0x7f88e013f9c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-c con 0x7f88e013fd90 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient(hunting): picked mon.noname-b con 0x7f88e01401f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e008fa40 con 0x7f88e013f9c0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e0133320 con 0x7f88e01401f0 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e00590d0 con 0x7f88e013fd90 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient(hunting): authenticate will time out at 1289.620357s 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e4c73640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f88e013f9c0 0x7f88e013e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55530/0 (socket says 10.20.193.102:55530) 2026-02-21T22:18:00.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e4c73640 1 -- 10.20.193.102:0/2839482012 learned_addr learned my addr 10.20.193.102:0/2839482012 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e4c73640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.102:6789/0 conn(0x7f88e013f9c0 legacy=0x7f88e013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d7fff640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.102:6790/0 conn(0x7f88e013fd90 legacy=0x7f88e0193da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5474640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.178:6789/0 conn(0x7f88e01401f0 legacy=0x7f88e01954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215073200 0 0) 0x7f88d0002e00 con 0x7f88e013f9c0 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): my global_id is 34113 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b0003620 con 0x7f88e013f9c0 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731344711 0 0) 0x7f88e00590d0 con 0x7f88e013fd90 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): my global_id is 4445 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88d0002e00 con 0x7f88e013fd90 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086480957 0 0) 0x7f88e0133320 con 0x7f88e01401f0 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): my global_id is 4414 2026-02-21T22:18:00.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88e00590d0 con 0x7f88e01401f0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1054310324 0 0) 0x7f88b0003620 con 0x7f88e013f9c0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88e0133320 con 0x7f88e013f9c0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88d0003710 con 0x7f88e013f9c0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): dump: 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 518276346 0 0) 0x7f88d0002e00 con 0x7f88e013fd90 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88b0003620 con 0x7f88e013fd90 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88c80032d0 con 0x7f88e013fd90 2026-02-21T22:18:00.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): dump: 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2562302103 0 0) 0x7f88e00590d0 con 0x7f88e01401f0 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88d0002e00 con 0x7f88e01401f0 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88dc005df0 con 0x7f88e01401f0 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient(hunting): dump: 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.390 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2532479376 0 0) 0x7f88e0133320 con 0x7f88e013f9c0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.102:6790/0 conn(0x7f88e013fd90 legacy=0x7f88e0193da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.178:6789/0 conn(0x7f88e01401f0 legacy=0x7f88e01954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 monclient: found mon.a 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e0130fc0 con 0x7f88e013f9c0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 5 monclient: authenticate success, global_id 34113 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _renew_subs 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f88e0133040 con 0x7f88e013f9c0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _renew_subs 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f88e0152ca0 con 0x7f88e013f9c0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f88d0003cf0 con 0x7f88e013f9c0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f88d0004080 con 0x7f88e013f9c0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d57fa640 10 monclient: dump: 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88d6ffd640 4 set_mon_vals no callback set 2026-02-21T22:18:00.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.382+0000 7f88e5c75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88e5c75640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88e5c75640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e00008e0 con 0x7f88e013f9c0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f88d00042d0 con 0x7f88e013f9c0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(114..114 src has 1..114) ==== 3975+0+0 (unknown 3169867183 0 0) 0x7f88d004c910 con 0x7f88e013f9c0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d7fff640 10 monclient: get_auth_request con 0x7f88b003d850 auth_method 0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f88e008fa40 con 0x7f88e013f9c0 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.386+0000 7f88d57fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:00.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:18:00.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:00.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88e0093ce0 con 0x7f88e013f9c0 2026-02-21T22:18:00.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88d57fa640 1 -- 10.20.193.102:0/2839482012 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 841776363) 0x7f88d0017880 con 0x7f88e013f9c0 2026-02-21T22:18:00.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88d57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:00.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88d57fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:00.514 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:18:00.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.178:6800/2549949439 conn(0x7f88b003d850 legacy=0x7f88b005dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 10 monclient: shutdown 2026-02-21T22:18:00.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 >> v1:10.20.193.102:6789/0 conn(0x7f88e013f9c0 legacy=0x7f88e013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 shutdown_connections 2026-02-21T22:18:00.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 >> 10.20.193.102:0/2839482012 conn(0x7f88e005e3f0 msgr2=0x7f88e0092870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 shutdown_connections 2026-02-21T22:18:00.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.506+0000 7f88e5c75640 1 -- 10.20.193.102:0/2839482012 wait complete. 2026-02-21T22:18:00.522 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:18:00.522 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:18:00.523 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.578+0000 7f815bee3640 10 monclient: get_monmap_and_config 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.578+0000 7f815bee3640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.578+0000 7f815bee3640 10 monclient: monmap: 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:00.584494+0000 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:00.584494+0000 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.584 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:00.585 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:00.585 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 1 Processor -- start 2026-02-21T22:18:00.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 1 -- start start 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient: init 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-b con 0x7f815408c4e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-a con 0x7f815408d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-c con 0x7f815408d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815408bd50 con 0x7f815408d410 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815408c8b0 con 0x7f815408d890 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8154083560 con 0x7f815408c4e0 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 10 monclient(hunting): authenticate will time out at 1289.820423s 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159457640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 0x7f8154188a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55536/0 (socket says 10.20.193.102:55536) 2026-02-21T22:18:00.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159457640 1 -- 10.20.193.102:0/1466070290 learned_addr learned my addr 10.20.193.102:0/1466070290 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159457640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f8154188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159c58640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 legacy=0x7f815408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876872085 0 0) 0x7f815408bd50 con 0x7f815408d410 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): my global_id is 34116 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f813c003620 con 0x7f815408d410 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815a459640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f81541aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465547551 0 0) 0x7f8144002cb0 con 0x7f815408c4e0 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): my global_id is 4448 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815408bd50 con 0x7f815408c4e0 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 506636968 0 0) 0x7f815408bd50 con 0x7f815408c4e0 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f81541aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f8154188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 monclient: found mon.noname-b 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8154084bc0 con 0x7f815408c4e0 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 5 monclient: authenticate success, global_id 4448 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8144003330 con 0x7f815408c4e0 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: dump: 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.590 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f815bee3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159c58640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 legacy=0x7f815408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 10 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159c58640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 legacy=0x7f815408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159c58640 1 --1- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 0x7f815408cd00 :-1 s=OPENED pgs=186 gs=1 cs=1 l=1).handle_message read tag failed 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8159c58640 1 --1- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 0x7f815408cd00 :-1 s=OPENED pgs=186 gs=1 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6790/0 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f815408c4e0 legacy=0x7f815408cd00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): picked mon.c con 0x7f813c004d30 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): picked mon.a con 0x7f815408d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): picked mon.b con 0x7f815408d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f815408bd50 con 0x7f815408d410 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f813c003620 con 0x7f815408d890 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8144002cb0 con 0x7f813c004d30 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.582+0000 7f8158c56640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.102:6790/0 2026-02-21T22:18:00.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8159457640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f813c045b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8159c58640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f813c004d30 legacy=0x7f813c005100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432326716 0 0) 0x7f815408bd50 con 0x7f815408d410 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f813c06af20 con 0x7f815408d410 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751820525 0 0) 0x7f8144002cb0 con 0x7f813c004d30 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f815408bd50 con 0x7f813c004d30 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815a459640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f813c067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 3946361460 0 0) 0x7f813c06af20 con 0x7f815408d410 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6790/0 conn(0x7f813c004d30 legacy=0x7f813c005100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f813c067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 monclient: found mon.a 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f813c06c0b0 con 0x7f815408d410 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8148004b50 con 0x7f815408d410 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: dump: 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 1 -- 10.20.193.102:0/1466070290 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8148004da0 con 0x7f815408d410 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8158c56640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 4 set_mon_vals no callback set 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: shutdown 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- 10.20.193.102:0/1466070290 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f813c045b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- 10.20.193.102:0/1466070290 shutdown_connections 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- 10.20.193.102:0/1466070290 >> 10.20.193.102:0/1466070290 conn(0x7f8154091a50 msgr2=0x7f8154091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- 10.20.193.102:0/1466070290 shutdown_connections 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- 10.20.193.102:0/1466070290 wait complete. 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: monmap: 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 Processor -- start 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- start start 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: init 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-c con 0x7f813c004d30 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-a con 0x7f815408d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient(hunting): picked mon.noname-b con 0x7f815408d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815405c790 con 0x7f815408d410 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815405b670 con 0x7f815408d890 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815405a430 con 0x7f813c004d30 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f815bee3640 10 monclient(hunting): authenticate will time out at 1289.827674s 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8159c58640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f813c004d30 0x7f815405dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39022/0 (socket says 10.20.193.102:39022) 2026-02-21T22:18:00.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.586+0000 7f8159c58640 1 -- 10.20.193.102:0/553065721 learned_addr learned my addr 10.20.193.102:0/553065721 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f8159c58640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6790/0 conn(0x7f813c004d30 legacy=0x7f815405dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f8159457640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f815421f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945089035 0 0) 0x7f815405a430 con 0x7f813c004d30 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): my global_id is 4451 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815a459640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f8154240fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8124003620 con 0x7f813c004d30 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505899487 0 0) 0x7f815405c790 con 0x7f815408d410 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): my global_id is 34119 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815405a430 con 0x7f815408d410 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133617917 0 0) 0x7f815405b670 con 0x7f815408d890 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): my global_id is 4420 2026-02-21T22:18:00.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815405c790 con 0x7f815408d890 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 590425274 0 0) 0x7f8124003620 con 0x7f813c004d30 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f815405b670 con 0x7f813c004d30 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8144002c20 con 0x7f813c004d30 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): dump: 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3448916736 0 0) 0x7f815405a430 con 0x7f815408d410 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8124003620 con 0x7f815408d410 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8148002db0 con 0x7f815408d410 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient(hunting): dump: 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.600 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3728567412 0 0) 0x7f815405b670 con 0x7f813c004d30 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.178:6789/0 conn(0x7f815408d890 legacy=0x7f8154240fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f815408d410 legacy=0x7f815421f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 monclient: found mon.c 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815405bef0 con 0x7f813c004d30 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 5 monclient: authenticate success, global_id 4451 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 10 monclient: _renew_subs 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f815405c1d0 con 0x7f813c004d30 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 10 monclient: _renew_subs 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f815405e3d0 con 0x7f813c004d30 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8144003bd0 con 0x7f813c004d30 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8144003f60 con 0x7f813c004d30 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f8143fff640 4 set_mon_vals no callback set 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 10 monclient: dump: 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f81540008e0 con 0x7f81541de770 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8144004100 con 0x7f813c004d30 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(114..114 src has 1..114) ==== 3975+0+0 (unknown 3169867183 0 0) 0x7f814404c640 con 0x7f813c004d30 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.590+0000 7f815a459640 10 monclient: get_auth_request con 0x7f812403d5c0 auth_method 0 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f8159457640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541de770 legacy=0x7f81541b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494012444 0 0) 0x7f81540008e0 con 0x7f81541de770 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient(hunting): my global_id is 34122 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815405b670 con 0x7f81541de770 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2867836501 0 0) 0x7f815405b670 con 0x7f81541de770 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f81540008e0 con 0x7f81541de770 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8148004e30 con 0x7f81541de770 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 10 monclient: dump: 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.603 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2260049554 0 0) 0x7f81540008e0 con 0x7f81541de770 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.594+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81541deb60 con 0x7f81541de770 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f8148007a20 con 0x7f81541de770 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f81427fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f81427fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541de770 legacy=0x7f81541b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541de770 legacy=0x7f81541b08d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f815bee3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:00.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.598+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8154090810 con 0x7f81541b0fe0 2026-02-21T22:18:00.604 INFO:tasks.rados.rados.0.trial178.stdout:444: done (2 left) 2026-02-21T22:18:00.604 INFO:tasks.rados.rados.0.trial178.stdout:446: done (1 left) 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:447: done (0 left) 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:448: rollback oid 32 current snap is 49 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 47 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:449: rollback oid 9 current snap is 49 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 48 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:450: delete oid 46 current snap is 49 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:449: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:449: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:450: done (2 left) 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:451: copy_from oid 19 from oid 2 current snap is 49 2026-02-21T22:18:00.605 INFO:tasks.rados.rados.0.trial178.stdout:452: snap_remove snap 43 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f8159457640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541b0fe0 legacy=0x7f8154263700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529049008 0 0) 0x7f8154090810 con 0x7f81541b0fe0 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient(hunting): my global_id is 34125 2026-02-21T22:18:00.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81540008e0 con 0x7f81541b0fe0 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2503591780 0 0) 0x7f81540008e0 con 0x7f81541b0fe0 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8154090810 con 0x7f81541b0fe0 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8148005470 con 0x7f81541b0fe0 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient: dump: 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1781900553 0 0) 0x7f8154090810 con 0x7f81541b0fe0 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f81540909e0 con 0x7f81541b0fe0 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 3380080661) 0x7f8148007830 con 0x7f81541b0fe0 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541b0fe0 legacy=0x7f8154263700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f81427fc640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6789/0 conn(0x7f81541b0fe0 legacy=0x7f8154263700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 29471, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:00.608 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.609 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.610 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.611 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:00.612 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.613 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.178:6800/2549949439 conn(0x7f812403d5c0 legacy=0x7f812405da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 10 monclient: shutdown 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 >> v1:10.20.193.102:6790/0 conn(0x7f813c004d30 legacy=0x7f815405dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815a459640 1 -- 10.20.193.102:0/553065721 reap_dead start 2026-02-21T22:18:00.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815a459640 1 -- 10.20.193.102:0/553065721 reap_dead start 2026-02-21T22:18:00.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 shutdown_connections 2026-02-21T22:18:00.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 >> 10.20.193.102:0/553065721 conn(0x7f8154091a50 msgr2=0x7f815408a710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 shutdown_connections 2026-02-21T22:18:00.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.602+0000 7f815bee3640 1 -- 10.20.193.102:0/553065721 wait complete. 2026-02-21T22:18:00.616 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:00.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:18:00.616 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: get_monmap_and_config 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: monmap: 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:00.679829+0000 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:00.679829+0000 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:00.679 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 1 Processor -- start 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 1 -- start start 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: init 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-c con 0x7f7af8091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-b con 0x7f7af80909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-a con 0x7f7af81854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8057a20 con 0x7f7af81854b0 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8091d20 con 0x7f7af8091240 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8057740 con 0x7f7af80909a0 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.674+0000 7f7afe9ab640 10 monclient(hunting): authenticate will time out at 1289.915982s 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af77fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7af80909a0 0x7f7af8183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39030/0 (socket says 10.20.193.102:39030) 2026-02-21T22:18:00.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af77fe640 1 -- 10.20.193.102:0/2400549298 learned_addr learned my addr 10.20.193.102:0/2400549298 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af77fe640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.102:6790/0 conn(0x7f7af80909a0 legacy=0x7f7af8183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afcf21640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.102:6789/0 conn(0x7f7af81854b0 legacy=0x7f7af81a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af7fff640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.178:6789/0 conn(0x7f7af8091240 legacy=0x7f7af8091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039663965 0 0) 0x7f7af8057740 con 0x7f7af80909a0 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): my global_id is 4454 2026-02-21T22:18:00.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7adc003620 con 0x7f7af80909a0 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162666870 0 0) 0x7f7af8057a20 con 0x7f7af81854b0 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): my global_id is 34128 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8057740 con 0x7f7af81854b0 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455020895 0 0) 0x7f7af8091d20 con 0x7f7af8091240 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient(hunting): my global_id is 4423 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8057a20 con 0x7f7af8091240 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962011579 0 0) 0x7f7adc003620 con 0x7f7af80909a0 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.178:6789/0 conn(0x7f7af8091240 legacy=0x7f7af8091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.102:6789/0 conn(0x7f7af81854b0 legacy=0x7f7af81a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 monclient: found mon.noname-b 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af8090dc0 con 0x7f7af80909a0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7ae8003110 con 0x7f7af80909a0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: dump: 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 5 monclient: authenticate success, global_id 4454 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7ae80033e0 con 0x7f7af80909a0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 1 -- 10.20.193.102:0/2400549298 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7ae8004b50 con 0x7f7af80909a0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7af6ffd640 10 monclient: dump: 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 4 set_mon_vals no callback set 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 10 monclient: shutdown 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2400549298 >> v1:10.20.193.102:6790/0 conn(0x7f7af80909a0 legacy=0x7f7af8183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2400549298 shutdown_connections 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2400549298 >> 10.20.193.102:0/2400549298 conn(0x7f7af805e660 msgr2=0x7f7af8095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2400549298 shutdown_connections 2026-02-21T22:18:00.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2400549298 wait complete. 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.678+0000 7f7afe9ab640 10 monclient: monmap: 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 Processor -- start 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- start start 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: init 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-b con 0x7f7af80909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-a con 0x7f7af8091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient(hunting): picked mon.noname-c con 0x7f7af81854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af812bc40 con 0x7f7af8091240 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8112e90 con 0x7f7af80909a0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8112020 con 0x7f7af81854b0 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient(hunting): authenticate will time out at 1289.920732s 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af77fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7af8091240 0x7f7af81728f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55586/0 (socket says 10.20.193.102:55586) 2026-02-21T22:18:00.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af77fe640 1 -- 10.20.193.102:0/2682508138 learned_addr learned my addr 10.20.193.102:0/2682508138 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af77fe640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6789/0 conn(0x7f7af8091240 legacy=0x7f7af81728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afcf21640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af81854b0 legacy=0x7f7af8174010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af7fff640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.178:6789/0 conn(0x7f7af80909a0 legacy=0x7f7af812b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295424212 0 0) 0x7f7af8112020 con 0x7f7af81854b0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): my global_id is 4457 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ac8003620 con 0x7f7af81854b0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1162461341 0 0) 0x7f7af812bc40 con 0x7f7af8091240 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): my global_id is 34131 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8112020 con 0x7f7af8091240 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630401012 0 0) 0x7f7af8112e90 con 0x7f7af80909a0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): my global_id is 4426 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af812bc40 con 0x7f7af80909a0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1876393465 0 0) 0x7f7ac8003620 con 0x7f7af81854b0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7af8112e90 con 0x7f7af81854b0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7aec004c20 con 0x7f7af81854b0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): dump: 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2538191000 0 0) 0x7f7af8112020 con 0x7f7af8091240 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ac8003620 con 0x7f7af8091240 2026-02-21T22:18:00.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7ae80048e0 con 0x7f7af8091240 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient(hunting): dump: 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1186226058 0 0) 0x7f7af8112e90 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.178:6789/0 conn(0x7f7af80909a0 legacy=0x7f7af812b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6789/0 conn(0x7f7af8091240 legacy=0x7f7af81728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 monclient: found mon.c 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af8110b30 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 5 monclient: authenticate success, global_id 4457 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _renew_subs 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7af81125f0 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _renew_subs 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7af8111860 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7aec0050b0 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7aec005410 con 0x7f7af81854b0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af4ff9640 10 monclient: dump: 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.689 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.682+0000 7f7af67fc640 4 set_mon_vals no callback set 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7afe9ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7af80008e0 con 0x7f7af812e4d0 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f7aec0056e0 con 0x7f7af81854b0 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(115..115 src has 1..115) ==== 3947+0+0 (unknown 883009491 0 0) 0x7f7aec04da70 con 0x7f7af81854b0 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af77fe640 10 monclient: get_auth_request con 0x7f7ac803d5c0 auth_method 0 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af7fff640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af812e4d0 legacy=0x7f7af808d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2169182787 0 0) 0x7f7af80008e0 con 0x7f7af812e4d0 2026-02-21T22:18:00.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient(hunting): my global_id is 4460 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af8112e90 con 0x7f7af812e4d0 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3010739988 0 0) 0x7f7af8112e90 con 0x7f7af812e4d0 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7af80008e0 con 0x7f7af812e4d0 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7ae4003b40 con 0x7f7af812e4d0 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient: dump: 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2615785919 0 0) 0x7f7af80008e0 con 0x7f7af812e4d0 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7af812e8a0 con 0x7f7af812e4d0 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f7ae40058c0 con 0x7f7af812e4d0 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af812e4d0 legacy=0x7f7af808d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.686+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af812e4d0 legacy=0x7f7af808d030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.690+0000 7f7afe9ab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:00.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.690+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7af8095010 con 0x7f7af808d740 2026-02-21T22:18:00.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af7fff640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af808d740 legacy=0x7f7af8176740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827740014 0 0) 0x7f7af8095010 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient(hunting): my global_id is 4463 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7af80008e0 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1517941945 0 0) 0x7f7af80008e0 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7af8095010 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7ae4004360 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient: dump: 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1750971704 0 0) 0x7f7af8095010 con 0x7f7af808d740 2026-02-21T22:18:00.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7af8058480 con 0x7f7af808d740 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 1630752995) 0x7f7ae4005a70 con 0x7f7af808d740 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af808d740 legacy=0x7f7af8176740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7af4ff9640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af808d740 legacy=0x7f7af8176740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 163324, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.699 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:00.700 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:00.701 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.702 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.703 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:00.704 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.178:6800/2549949439 conn(0x7f7ac803d5c0 legacy=0x7f7ac805da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 10 monclient: shutdown 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 >> v1:10.20.193.102:6790/0 conn(0x7f7af81854b0 legacy=0x7f7af8174010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afcf21640 1 -- 10.20.193.102:0/2682508138 reap_dead start 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 shutdown_connections 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 >> 10.20.193.102:0/2682508138 conn(0x7f7af805e660 msgr2=0x7f7af8088de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 shutdown_connections 2026-02-21T22:18:00.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.694+0000 7f7afe9ab640 1 -- 10.20.193.102:0/2682508138 wait complete. 2026-02-21T22:18:00.709 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:00.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:18:00.710 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: get_monmap_and_config 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: monmap: 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:00.770740+0000 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:00.770740+0000 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:00.770 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 1 Processor -- start 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 1 -- start start 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: init 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-b con 0x7ff2c008cd00 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-c con 0x7ff2c008d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-a con 0x7ff2c008b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00890e0 con 0x7ff2c008b8d0 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c0088ea0 con 0x7ff2c008d7e0 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c008aea0 con 0x7ff2c008cd00 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c6e2c640 10 monclient(hunting): authenticate will time out at 1290.006630s 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c4ba1640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff2c008cd00 0x7ff2c008d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39068/0 (socket says 10.20.193.102:39068) 2026-02-21T22:18:00.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.766+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/3832429225 learned_addr learned my addr 10.20.193.102:0/3832429225 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.102:6790/0 conn(0x7ff2c008cd00 legacy=0x7ff2c008d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c53a2640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.102:6789/0 conn(0x7ff2c008b8d0 legacy=0x7ff2c01a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b7fff640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.178:6789/0 conn(0x7ff2c008d7e0 legacy=0x7ff2c01840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341342682 0 0) 0x7ff2c008aea0 con 0x7ff2c008cd00 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): my global_id is 4466 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2ac003620 con 0x7ff2c008cd00 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991978473 0 0) 0x7ff2c00890e0 con 0x7ff2c008b8d0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): my global_id is 34134 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c008aea0 con 0x7ff2c008b8d0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278173427 0 0) 0x7ff2c0088ea0 con 0x7ff2c008d7e0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient(hunting): my global_id is 4429 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c00890e0 con 0x7ff2c008d7e0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1851641542 0 0) 0x7ff2ac003620 con 0x7ff2c008cd00 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.178:6789/0 conn(0x7ff2c008d7e0 legacy=0x7ff2c01840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.102:6789/0 conn(0x7ff2c008b8d0 legacy=0x7ff2c01a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 monclient: found mon.noname-b 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c008a600 con 0x7ff2c008cd00 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2b0003110 con 0x7ff2c008cd00 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 5 monclient: authenticate success, global_id 4466 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:00.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: dump: 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2b00033e0 con 0x7ff2c008cd00 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 1 -- 10.20.193.102:0/3832429225 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2b0004b50 con 0x7ff2c008cd00 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 4 set_mon_vals no callback set 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2b77fe640 10 monclient: dump: 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: shutdown 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/3832429225 >> v1:10.20.193.102:6790/0 conn(0x7ff2c008cd00 legacy=0x7ff2c008d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/3832429225 shutdown_connections 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/3832429225 >> 10.20.193.102:0/3832429225 conn(0x7ff2c0091a50 msgr2=0x7ff2c0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/3832429225 shutdown_connections 2026-02-21T22:18:00.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/3832429225 wait complete. 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: monmap: 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 Processor -- start 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- start start 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: init 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-c con 0x7ff2c008b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-a con 0x7ff2c008cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient(hunting): picked mon.noname-b con 0x7ff2c008d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00820b0 con 0x7ff2c008cd00 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c007fd50 con 0x7ff2c008d7e0 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c0081810 con 0x7ff2c008b8d0 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c6e2c640 10 monclient(hunting): authenticate will time out at 1290.010843s 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c4ba1640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff2c008b8d0 0x7ff2c01533e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39076/0 (socket says 10.20.193.102:39076) 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.770+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/121438571 learned_addr learned my addr 10.20.193.102:0/121438571 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b7fff640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.102:6789/0 conn(0x7ff2c008cd00 legacy=0x7ff2c0153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.102:6790/0 conn(0x7ff2c008b8d0 legacy=0x7ff2c01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992735700 0 0) 0x7ff2c00820b0 con 0x7ff2c008cd00 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): my global_id is 34137 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff290003620 con 0x7ff2c008cd00 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606566290 0 0) 0x7ff2c0081810 con 0x7ff2c008b8d0 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): my global_id is 4469 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c00820b0 con 0x7ff2c008b8d0 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c53a2640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c008d7e0 legacy=0x7ff2c0195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3643382630 0 0) 0x7ff290003620 con 0x7ff2c008cd00 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff2c0081810 con 0x7ff2c008cd00 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2a80033c0 con 0x7ff2c008cd00 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): dump: 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.778 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 185608191 0 0) 0x7ff2c0081810 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.102:6790/0 conn(0x7ff2c008b8d0 legacy=0x7ff2c01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c008d7e0 legacy=0x7ff2c0195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 monclient: found mon.a 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c0081af0 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 5 monclient: authenticate success, global_id 34137 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 10 monclient: _renew_subs 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c0082ab0 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 10 monclient: _renew_subs 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff2c0080a80 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2a80039c0 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2a8003d50 con 0x7ff2c008cd00 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 10 monclient: dump: 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b6ffd640 4 set_mon_vals no callback set 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff2c00008e0 con 0x7ff2c007f390 2026-02-21T22:18:00.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff2a8004020 con 0x7ff2c008cd00 2026-02-21T22:18:00.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(115..115 src has 1..115) ==== 3947+0+0 (unknown 883009491 0 0) 0x7ff2a804c710 con 0x7ff2c008cd00 2026-02-21T22:18:00.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.774+0000 7ff2c53a2640 10 monclient: get_auth_request con 0x7ff29003d4b0 auth_method 0 2026-02-21T22:18:00.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007f390 legacy=0x7ff2c007a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595997267 0 0) 0x7ff2c00008e0 con 0x7ff2c007f390 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient(hunting): my global_id is 4435 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c0081810 con 0x7ff2c007f390 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2902411006 0 0) 0x7ff2c0081810 con 0x7ff2c007f390 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff2c00008e0 con 0x7ff2c007f390 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2b0004a90 con 0x7ff2c007f390 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient: dump: 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.781 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2100073937 0 0) 0x7ff2c00008e0 con 0x7ff2c007f390 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2c0090810 con 0x7ff2c007f390 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ff2b0002db0 con 0x7ff2c007f390 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007f390 legacy=0x7ff2c007a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.778+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007f390 legacy=0x7ff2c007a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2c6e2c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:00.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff2c00909c0 con 0x7ff2c007aa80 2026-02-21T22:18:00.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2c4ba1640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007aa80 legacy=0x7ff2c0197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280671542 0 0) 0x7ff2c00909c0 con 0x7ff2c007aa80 2026-02-21T22:18:00.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient(hunting): my global_id is 4438 2026-02-21T22:18:00.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c00008e0 con 0x7ff2c007aa80 2026-02-21T22:18:00.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2449671609 0 0) 0x7ff2c00008e0 con 0x7ff2c007aa80 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff2c00909c0 con 0x7ff2c007aa80 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff2b00051b0 con 0x7ff2c007aa80 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 10 monclient: dump: 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.790 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4228183310 0 0) 0x7ff2c00909c0 con 0x7ff2c007aa80 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.782+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff2c007ae50 con 0x7ff2c007aa80 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 2792558360) 0x7ff2b0007410 con 0x7ff2c007aa80 2026-02-21T22:18:00.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2b57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2b57fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007aa80 legacy=0x7ff2c0197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2b57fa640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6789/0 conn(0x7ff2c007aa80 legacy=0x7ff2c0197e50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:18:00.792 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 163392, 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.793 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.794 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:00.795 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:00.796 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.797 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.798 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.799 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:18:00.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:00.801 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff29003d4b0 legacy=0x7ff29005d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 10 monclient: shutdown 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 >> v1:10.20.193.102:6789/0 conn(0x7ff2c008cd00 legacy=0x7ff2c0153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c53a2640 1 -- 10.20.193.102:0/121438571 reap_dead start 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c53a2640 1 -- 10.20.193.102:0/121438571 reap_dead start 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 shutdown_connections 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 >> 10.20.193.102:0/121438571 conn(0x7ff2c0091a50 msgr2=0x7ff2c0085760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 shutdown_connections 2026-02-21T22:18:00.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.786+0000 7ff2c6e2c640 1 -- 10.20.193.102:0/121438571 wait complete. 2026-02-21T22:18:00.802 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:00.802 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:18:00.803 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:18:00.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: get_monmap_and_config 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: monmap: 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:00.871085+0000 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:00.871085+0000 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:00.871 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 1 Processor -- start 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 1 -- start start 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: init 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:00.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-b con 0x7f93e80584c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-a con 0x7f93e80588a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-c con 0x7f93e81435f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e80902a0 con 0x7f93e80588a0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e80921d0 con 0x7f93e81435f0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e80574a0 con 0x7f93e80584c0 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93f037c640 10 monclient(hunting): authenticate will time out at 1290.106973s 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93ed8f0640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f93e80588a0 0x7f93e8183d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55626/0 (socket says 10.20.193.102:55626) 2026-02-21T22:18:00.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.866+0000 7f93ed8f0640 1 -- 10.20.193.102:0/77025866 learned_addr learned my addr 10.20.193.102:0/77025866 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed8f0640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.102:6789/0 conn(0x7f93e80588a0 legacy=0x7f93e8183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ee0f1640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.102:6790/0 conn(0x7f93e80584c0 legacy=0x7f93e8142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ee8f2640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.178:6789/0 conn(0x7f93e81435f0 legacy=0x7f93e81a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2227237614 0 0) 0x7f93dc002c60 con 0x7f93e80588a0 2026-02-21T22:18:00.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): my global_id is 34140 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d0003620 con 0x7f93e80588a0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770193280 0 0) 0x7f93e80574a0 con 0x7f93e80584c0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): my global_id is 4472 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93dc002c60 con 0x7f93e80584c0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305514678 0 0) 0x7f93e80921d0 con 0x7f93e81435f0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient(hunting): my global_id is 4441 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93e80574a0 con 0x7f93e81435f0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2695139761 0 0) 0x7f93d0003620 con 0x7f93e80588a0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.102:6790/0 conn(0x7f93e80584c0 legacy=0x7f93e8142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.178:6789/0 conn(0x7f93e81435f0 legacy=0x7f93e81a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 monclient: found mon.noname-a 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8056640 con 0x7f93e80588a0 2026-02-21T22:18:00.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93dc0032e0 con 0x7f93e80588a0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: dump: 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 5 monclient: authenticate success, global_id 34140 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f93dc0049a0 con 0x7f93e80588a0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 1 -- 10.20.193.102:0/77025866 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93dc004d30 con 0x7f93e80588a0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed0ef640 10 monclient: dump: 2026-02-21T22:18:00.877 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 4 set_mon_vals no callback set 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: shutdown 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- 10.20.193.102:0/77025866 >> v1:10.20.193.102:6789/0 conn(0x7f93e80588a0 legacy=0x7f93e8183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- 10.20.193.102:0/77025866 shutdown_connections 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- 10.20.193.102:0/77025866 >> 10.20.193.102:0/77025866 conn(0x7f93e805e700 msgr2=0x7f93e8095ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- 10.20.193.102:0/77025866 shutdown_connections 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- 10.20.193.102:0/77025866 wait complete. 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: build_initial_monmap 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: monmap: 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:00.878 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 Processor -- start 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- start start 2026-02-21T22:18:00.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: init 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-b con 0x7f93e80588a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-c con 0x7f93e81435f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient(hunting): picked mon.noname-a con 0x7f93e8130070 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8150c00 con 0x7f93e8130070 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8059020 con 0x7f93e80588a0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8130970 con 0x7f93e81435f0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93f037c640 10 monclient(hunting): authenticate will time out at 1290.111662s 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed8f0640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f93e81435f0 0x7f93e81945b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:39108/0 (socket says 10.20.193.102:39108) 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.870+0000 7f93ed8f0640 1 -- 10.20.193.102:0/1685926741 learned_addr learned my addr 10.20.193.102:0/1685926741 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ed8f0640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.102:6790/0 conn(0x7f93e81435f0 legacy=0x7f93e81945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ee8f2640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.102:6789/0 conn(0x7f93e8130070 legacy=0x7f93e8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ee0f1640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.178:6789/0 conn(0x7f93e80588a0 legacy=0x7f93e813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650741943 0 0) 0x7f93e8130970 con 0x7f93e81435f0 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): my global_id is 4475 2026-02-21T22:18:00.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93b8003620 con 0x7f93e81435f0 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3410054887 0 0) 0x7f93e80574a0 con 0x7f93e8130070 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): my global_id is 34143 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93e8130970 con 0x7f93e8130070 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591574829 0 0) 0x7f93e8059020 con 0x7f93e80588a0 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): my global_id is 4444 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93e80574a0 con 0x7f93e80588a0 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 866540121 0 0) 0x7f93e8130970 con 0x7f93e8130070 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f93e8059020 con 0x7f93e8130070 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93e40048d0 con 0x7f93e8130070 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): dump: 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.882 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1222355905 0 0) 0x7f93b8003620 con 0x7f93e81435f0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f93e8130970 con 0x7f93e81435f0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93dc002b50 con 0x7f93e81435f0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient(hunting): dump: 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 909455598 0 0) 0x7f93e8059020 con 0x7f93e8130070 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.102:6790/0 conn(0x7f93e81435f0 legacy=0x7f93e81945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.178:6789/0 conn(0x7f93e80588a0 legacy=0x7f93e813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: _finish_hunting 0 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 monclient: found mon.a 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:00.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e812f510 con 0x7f93e8130070 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: _finish_auth 0 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: _check_auth_tickets 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 5 monclient: authenticate success, global_id 34143 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _renew_subs 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f93e8131870 con 0x7f93e8130070 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _renew_subs 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f93e8150190 con 0x7f93e8130070 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f93e4004ed0 con 0x7f93e8130070 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93e4005260 con 0x7f93e8130070 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ec8ee640 4 set_mon_vals no callback set 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 10 monclient: dump: 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:00.884 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e80008e0 con 0x7f93e8130070 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f93e40054f0 con 0x7f93e8130070 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(115..115 src has 1..115) ==== 3947+0+0 (unknown 883009491 0 0) 0x7f93e404db00 con 0x7f93e8130070 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.874+0000 7f93ee0f1640 10 monclient: get_auth_request con 0x7f93b803d5b0 auth_method 0 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.878+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f93e8150c00 con 0x7f93e8130070 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.878+0000 7f93ceffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:00.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:00.878+0000 7f93ceffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:01.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93f037c640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:18:01.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93f037c640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:18:01.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93f037c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:01.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f93e8150480 con 0x7f93e8130070 2026-02-21T22:18:01.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93ceffd640 1 -- 10.20.193.102:0/1685926741 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f93e4018a90 con 0x7f93e8130070 2026-02-21T22:18:01.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93ceffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:18:01.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.002+0000 7f93ceffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=9,config=2,health=12,kv=10,logm=67} crc {auth=1222497409,config=1618278320,health=3286817296,kv=584380320,logm=2430295145}) 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=9,config=2,health=12,kv=11,logm=66} crc {auth=2458747979,config=1618278320,health=2089678410,kv=514132712,logm=1127381027}) 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=9,config=2,health=12,kv=10,logm=67} crc {auth=1222497409,config=1618278320,health=3286817296,kv=584380320,logm=2430295145}) 2026-02-21T22:18:01.009 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=10,config=2,health=13,kv=12,logm=63} crc {auth=3196836797,config=1618278320,health=2358046931,kv=514132712,logm=2750760626}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3877672713}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=465560157}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3877672713}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3216074074}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2798036456}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2499137007}) 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.010 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2798036456}) 2026-02-21T22:18:01.011 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1843786362}) 2026-02-21T22:18:01.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.178:6800/2549949439 conn(0x7f93b803d5b0 legacy=0x7f93b805da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:01.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 10 monclient: shutdown 2026-02-21T22:18:01.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:01.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 >> v1:10.20.193.102:6789/0 conn(0x7f93e8130070 legacy=0x7f93e8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:01.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 shutdown_connections 2026-02-21T22:18:01.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 >> 10.20.193.102:0/1685926741 conn(0x7f93e805e700 msgr2=0x7f93e8092fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:01.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 shutdown_connections 2026-02-21T22:18:01.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:01.006+0000 7f93f037c640 1 -- 10.20.193.102:0/1685926741 wait complete. 2026-02-21T22:18:01.015 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.015 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2810986159}) 2026-02-21T22:18:01.015 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3380474004}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2810986159}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2940345637}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2165015359}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1000206808}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2165015359}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4214553917}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=242314108}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=724320092}) 2026-02-21T22:18:01.016 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.009+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=242314108}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=534789281}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=19,osdmap=17} crc {logm=820201820,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=626215500,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3761073207,osd_pg_creating=1527314401,osd_snap=187548620,osdmap=928856510}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=1,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=18} crc {logm=1046289862,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=626215500,mon_config_key=2063301084,monmap=3809027407,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=1527314401,osd_snap=2756566425,osdmap=1613904776}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=20,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=19,osdmap=17} crc {logm=820201820,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=626215500,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3761073207,osd_pg_creating=1527314401,osd_snap=187548620,osdmap=928856510}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=21,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=19,osdmap=16} crc {logm=2329078522,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2459201445,mon_config_key=2063301084,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3761073207,osd_pg_creating=1527314401,osd_snap=3534914644,osdmap=2811173488}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2313471526}) 2026-02-21T22:18:01.017 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=847747776}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2313471526}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3145822476}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=59} crc {osdmap=2145850156}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=55} crc {osdmap=3561444648}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=59} crc {osdmap=2145850156}) 2026-02-21T22:18:01.018 INFO:tasks.ceph.mon.b.trial178.stderr:2026-02-21T22:18:01.013+0000 7f28756ee640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=59} crc {osdmap=3855465458}) 2026-02-21T22:18:01.019 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:452: done (3 left) 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:453: rollback oid 12 current snap is 49 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 39 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:449: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 111 (ObjNum 142 snap 36 seq_num 142) dirty exists 2026-02-21T22:18:01.100 INFO:tasks.rados.rados.0.trial178.stdout:448: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:448: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:448: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 201 (ObjNum 136 snap 32 seq_num 136) dirty exists 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:451: finishing copy_from racing read to trial17818531-19 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:448: done (3 left) 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:449: done (2 left) 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:454: write oid 35 current snap is 49 2026-02-21T22:18:01.101 INFO:tasks.rados.rados.0.trial178.stdout:454: seq_num 174 ranges {524418=453092,1732455=588379,2779130=145288} 2026-02-21T22:18:01.106 INFO:tasks.rados.rados.0.trial178.stdout:454: writing trial17818531-35 from 524418 to 977510 tid 1 2026-02-21T22:18:01.115 INFO:tasks.rados.rados.0.trial178.stdout:454: writing trial17818531-35 from 1732455 to 2320834 tid 2 2026-02-21T22:18:01.119 INFO:tasks.rados.rados.0.trial178.stdout:454: writing trial17818531-35 from 2779130 to 2924418 tid 3 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:455: snap_create 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:451: finishing copy_from to trial17818531-19 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 120 (ObjNum 158 snap 45 seq_num 158) dirty exists 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:453: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 129 snap 0 seq_num 99522832) dirty dne 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:454: finishing write tid 1 to trial17818531-35 2026-02-21T22:18:01.120 INFO:tasks.rados.rados.0.trial178.stdout:454: finishing write tid 2 to trial17818531-35 2026-02-21T22:18:01.121 INFO:tasks.rados.rados.0.trial178.stdout:454: finishing write tid 3 to trial17818531-35 2026-02-21T22:18:01.121 INFO:tasks.rados.rados.0.trial178.stdout:454: finishing write tid 4 to trial17818531-35 2026-02-21T22:18:01.121 INFO:tasks.rados.rados.0.trial178.stdout:454: finishing write tid 5 to trial17818531-35 2026-02-21T22:18:01.121 INFO:tasks.rados.rados.0.trial178.stdout:454: oid 35 updating version 0 to 215 2026-02-21T22:18:01.122 INFO:tasks.rados.rados.0.trial178.stdout:454: oid 35 version 215 is already newer than 213 2026-02-21T22:18:01.122 INFO:tasks.rados.rados.0.trial178.stdout:454: oid 35 updating version 215 to 216 2026-02-21T22:18:01.122 INFO:tasks.rados.rados.0.trial178.stdout:454: oid 35 version 216 is already newer than 214 2026-02-21T22:18:01.122 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 216 (ObjNum 174 snap 49 seq_num 174) dirty exists 2026-02-21T22:18:01.122 INFO:tasks.rados.rados.0.trial178.stdout:454: left oid 35 (ObjNum 174 snap 49 seq_num 174) 2026-02-21T22:18:02.020 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:18:02.021 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: get_monmap_and_config 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: monmap: 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:02.091323+0000 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:02.091323+0000 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 1 Processor -- start 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 1 -- start start 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: init 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:18:02.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-a con 0x7fc62808e9d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-b con 0x7fc62808f4b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-c con 0x7fc6281854e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc628091cd0 con 0x7fc62808e9d0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62805e2c0 con 0x7fc6281854e0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62808cba0 con 0x7fc62808f4b0 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62f3cb640 10 monclient(hunting): authenticate will time out at 1291.327246s 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62d140640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fc62808e9d0 0x7fc62808eda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56678/0 (socket says 10.20.193.102:56678) 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62c93f640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fc62808f4b0 0x7fc628183dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:46966/0 (socket says 10.20.193.102:46966) 2026-02-21T22:18:02.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.086+0000 7fc62d140640 1 -- 10.20.193.102:0/477935437 learned_addr learned my addr 10.20.193.102:0/477935437 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62d140640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.102:6789/0 conn(0x7fc62808e9d0 legacy=0x7fc62808eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62c93f640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.102:6790/0 conn(0x7fc62808f4b0 legacy=0x7fc628183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145737424 0 0) 0x7fc628091cd0 con 0x7fc62808e9d0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): my global_id is 34146 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc610003620 con 0x7fc62808e9d0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62d941640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.178:6789/0 conn(0x7fc6281854e0 legacy=0x7fc6281a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2995645114 0 0) 0x7fc62808cba0 con 0x7fc62808f4b0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient(hunting): my global_id is 4478 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc628091cd0 con 0x7fc62808f4b0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 768323264 0 0) 0x7fc610003620 con 0x7fc62808e9d0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.102:6790/0 conn(0x7fc62808f4b0 legacy=0x7fc628183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.178:6789/0 conn(0x7fc6281854e0 legacy=0x7fc6281a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 monclient: found mon.noname-a 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62808f8d0 con 0x7fc62808e9d0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc618003110 con 0x7fc62808e9d0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: dump: 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 5 monclient: authenticate success, global_id 34146 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc618003450 con 0x7fc62808e9d0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 1 -- 10.20.193.102:0/477935437 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc618004b50 con 0x7fc62808e9d0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc617fff640 10 monclient: dump: 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:02.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 4 set_mon_vals no callback set 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: shutdown 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- 10.20.193.102:0/477935437 >> v1:10.20.193.102:6789/0 conn(0x7fc62808e9d0 legacy=0x7fc62808eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- 10.20.193.102:0/477935437 shutdown_connections 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- 10.20.193.102:0/477935437 >> 10.20.193.102:0/477935437 conn(0x7fc628055730 msgr2=0x7fc628091010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- 10.20.193.102:0/477935437 shutdown_connections 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- 10.20.193.102:0/477935437 wait complete. 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: monmap: 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 Processor -- start 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- start start 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: init 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-c con 0x7fc62808e9d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-b con 0x7fc62808f4b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient(hunting): picked mon.noname-a con 0x7fc6281854e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62812c880 con 0x7fc6281854e0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62812e340 con 0x7fc62808f4b0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62812e620 con 0x7fc62808e9d0 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62f3cb640 10 monclient(hunting): authenticate will time out at 1291.331793s 2026-02-21T22:18:02.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62d941640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fc6281854e0 0x7fc628132130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56692/0 (socket says 10.20.193.102:56692) 2026-02-21T22:18:02.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.090+0000 7fc62d941640 1 -- 10.20.193.102:0/755051730 learned_addr learned my addr 10.20.193.102:0/755051730 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc62d941640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.102:6789/0 conn(0x7fc6281854e0 legacy=0x7fc628132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc62d140640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.102:6790/0 conn(0x7fc62808e9d0 legacy=0x7fc62818cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703037884 0 0) 0x7fc62812e620 con 0x7fc62808e9d0 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): my global_id is 4481 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5f8003620 con 0x7fc62808e9d0 2026-02-21T22:18:02.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc62c93f640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.178:6789/0 conn(0x7fc62808f4b0 legacy=0x7fc628183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc62d140640 1 --1- 10.20.193.102:0/755051730 >> v1:10.20.193.102:6790/0 conn(0x7fc62808e9d0 0x7fc62818cbb0 :-1 s=OPENED pgs=198 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fc5f8003620 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 600104202 0 0) 0x7fc5f8003620 con 0x7fc62808e9d0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc62812e620 con 0x7fc62808e9d0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc618002b70 con 0x7fc62808e9d0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): dump: 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315241228 0 0) 0x7fc62812e340 con 0x7fc62808f4b0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): my global_id is 4450 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5f8003620 con 0x7fc62808f4b0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2190435983 0 0) 0x7fc5f8003620 con 0x7fc62808f4b0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc62812e340 con 0x7fc62808f4b0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc61c0033d0 con 0x7fc62808f4b0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.094+0000 7fc615ffb640 10 monclient(hunting): dump: 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.099 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.100 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.100 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.100 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.100 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.100 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1591596216 0 0) 0x7fc62812e340 con 0x7fc62808f4b0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.102:6790/0 conn(0x7fc62808e9d0 legacy=0x7fc62818cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.102:6789/0 conn(0x7fc6281854e0 legacy=0x7fc628132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 monclient: found mon.b 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62812d3e0 con 0x7fc62808f4b0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3266744569 0 0) 0x7fc62812e620 con 0x7fc62808e9d0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 5 monclient: authenticate success, global_id 4450 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _renew_subs 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc62812d670 con 0x7fc62808f4b0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _renew_subs 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:18:02.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc628133a20 con 0x7fc62808f4b0 2026-02-21T22:18:02.101 INFO:tasks.rados.rados.0.trial178.stdout:451: done (3 left) 2026-02-21T22:18:02.101 INFO:tasks.rados.rados.0.trial178.stdout:453: done (2 left) 2026-02-21T22:18:02.101 INFO:tasks.rados.rados.0.trial178.stdout:454: done (1 left) 2026-02-21T22:18:02.101 INFO:tasks.rados.rados.0.trial178.stdout:455: done (0 left) 2026-02-21T22:18:02.101 INFO:tasks.rados.rados.0.trial178.stdout:456: delete oid 1 current snap is 50 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc61c004b90 con 0x7fc62808f4b0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc61c004ef0 con 0x7fc62808f4b0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc6177fe640 4 set_mon_vals no callback set 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 10 monclient: dump: 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6280008e0 con 0x7fc62808f4b0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fc61c005110 con 0x7fc62808f4b0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7fc61c04c4d0 con 0x7fc62808f4b0 2026-02-21T22:18:02.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.098+0000 7fc62d140640 10 monclient: get_auth_request con 0x7fc5f803d640 auth_method 0 2026-02-21T22:18:02.104 INFO:tasks.rados.rados.0.trial178.stdout:456: done (0 left) 2026-02-21T22:18:02.104 INFO:tasks.rados.rados.0.trial178.stdout:457: read oid 14 snap -1 2026-02-21T22:18:02.104 INFO:tasks.rados.rados.0.trial178.stdout:457: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:18:02.104 INFO:tasks.rados.rados.0.trial178.stdout:458: delete oid 47 current snap is 50 2026-02-21T22:18:02.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.102+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fc61c017540 con 0x7fc62808f4b0 2026-02-21T22:18:02.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.102+0000 7fc615ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.102+0000 7fc615ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:02.112 INFO:tasks.rados.rados.0.trial178.stdout:457: expect (ObjNum 72 snap 9 seq_num 72) 2026-02-21T22:18:02.112 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:02.195 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:02.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc628091460 con 0x7fc62808f4b0 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc615ffb640 1 -- 10.20.193.102:0/755051730 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 493390891) 0x7fc61c016e20 con 0x7fc62808f4b0 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc615ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:02.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc615ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:02.224 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:18:02.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.178:6800/2549949439 conn(0x7fc5f803d640 legacy=0x7fc5f805daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 10 monclient: shutdown 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 >> v1:10.20.193.178:6789/0 conn(0x7fc62808f4b0 legacy=0x7fc628183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 shutdown_connections 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 >> 10.20.193.102:0/755051730 conn(0x7fc628055730 msgr2=0x7fc6281a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 shutdown_connections 2026-02-21T22:18:02.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.218+0000 7fc62f3cb640 1 -- 10.20.193.102:0/755051730 wait complete. 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:457: done (1 left) 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:458: done (0 left) 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:459: read oid 32 snap -1 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:459: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:460: rollback oid 41 current snap is 50 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 41 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:461: copy_from oid 30 from oid 10 current snap is 50 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:462: write oid 19 current snap is 50 2026-02-21T22:18:02.227 INFO:tasks.rados.rados.0.trial178.stdout:462: seq_num 177 ranges {574839=795192,1907901=452059} 2026-02-21T22:18:02.233 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:18:02.233 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:18:02.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:18:02.233 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:18:02.236 INFO:tasks.rados.rados.0.trial178.stdout:462: writing trial17818531-19 from 574839 to 1370031 tid 1 2026-02-21T22:18:02.242 INFO:tasks.rados.rados.0.trial178.stdout:462: writing trial17818531-19 from 1907901 to 2359960 tid 2 2026-02-21T22:18:02.242 INFO:tasks.rados.rados.0.trial178.stdout:463: copy_from oid 17 from oid 20 current snap is 50 2026-02-21T22:18:02.243 INFO:tasks.rados.rados.0.trial178.stdout:459: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: get_monmap_and_config 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: monmap: 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:02.293721+0000 2026-02-21T22:18:02.293 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:02.293721+0000 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:02.294 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 1 Processor -- start 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 1 -- start start 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: init 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-c con 0x7fd89c091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-a con 0x7fd89c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-b con 0x7fd89c1854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c057a20 con 0x7fd89c0909a0 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c091d20 con 0x7fd89c091240 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c057740 con 0x7fd89c1854b0 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a3cf5640 10 monclient(hunting): authenticate will time out at 1291.530013s 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a1269640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 0x7fd89c183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56702/0 (socket says 10.20.193.102:56702) 2026-02-21T22:18:02.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.290+0000 7fd8a1269640 1 -- 10.20.193.102:0/3869730132 learned_addr learned my addr 10.20.193.102:0/3869730132 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a1269640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 legacy=0x7fd89c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a226b640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.102:6790/0 conn(0x7fd89c1854b0 legacy=0x7fd89c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825769259 0 0) 0x7fd89c057a20 con 0x7fd89c0909a0 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): my global_id is 34152 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd884003620 con 0x7fd89c0909a0 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511289192 0 0) 0x7fd89c057740 con 0x7fd89c1854b0 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient(hunting): my global_id is 4484 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c057a20 con 0x7fd89c1854b0 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a1a6a640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.178:6789/0 conn(0x7fd89c091240 legacy=0x7fd89c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2476957805 0 0) 0x7fd884003620 con 0x7fd89c0909a0 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.102:6790/0 conn(0x7fd89c1854b0 legacy=0x7fd89c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.178:6789/0 conn(0x7fd89c091240 legacy=0x7fd89c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 monclient: found mon.noname-a 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c090dc0 con 0x7fd89c0909a0 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd890003190 con 0x7fd89c0909a0 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 5 monclient: authenticate success, global_id 34152 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: dump: 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.308 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd890003420 con 0x7fd89c0909a0 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 1 -- 10.20.193.102:0/3869730132 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd890004b50 con 0x7fd89c0909a0 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a0a68640 10 monclient: dump: 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.309 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 4 set_mon_vals no callback set 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: shutdown 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/3869730132 >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 legacy=0x7fd89c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/3869730132 shutdown_connections 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/3869730132 >> 10.20.193.102:0/3869730132 conn(0x7fd89c05e660 msgr2=0x7fd89c095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/3869730132 shutdown_connections 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/3869730132 wait complete. 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: monmap: 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:02.311 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 Processor -- start 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- start start 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: init 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-a con 0x7fd89c0909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-b con 0x7fd89c091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient(hunting): picked mon.noname-c con 0x7fd89c1854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c1122e0 con 0x7fd89c0909a0 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c12bf00 con 0x7fd89c091240 2026-02-21T22:18:02.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c113150 con 0x7fd89c1854b0 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a3cf5640 10 monclient(hunting): authenticate will time out at 1291.534095s 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a1a6a640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 0x7fd89c12b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56714/0 (socket says 10.20.193.102:56714) 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a1a6a640 1 -- 10.20.193.102:0/1999033468 learned_addr learned my addr 10.20.193.102:0/1999033468 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.294+0000 7fd8a226b640 1 --1- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6790/0 conn(0x7fd89c1854b0 0x7fd89c1742d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:46992/0 (socket says 10.20.193.102:46992) 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a226b640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6790/0 conn(0x7fd89c1854b0 legacy=0x7fd89c1742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a1a6a640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 legacy=0x7fd89c12b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2048358998 0 0) 0x7fd89c113150 con 0x7fd89c1854b0 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): my global_id is 4487 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a1269640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.178:6789/0 conn(0x7fd89c091240 legacy=0x7fd89c172bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd86c003620 con 0x7fd89c1854b0 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291745038 0 0) 0x7fd89c1122e0 con 0x7fd89c0909a0 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): my global_id is 34155 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c113150 con 0x7fd89c0909a0 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164294009 0 0) 0x7fd89c12bf00 con 0x7fd89c091240 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): my global_id is 4456 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c1122e0 con 0x7fd89c091240 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 416267409 0 0) 0x7fd86c003620 con 0x7fd89c1854b0 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd89c12bf00 con 0x7fd89c1854b0 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd898004900 con 0x7fd89c1854b0 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): dump: 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.315 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2923391983 0 0) 0x7fd89c113150 con 0x7fd89c0909a0 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd86c003620 con 0x7fd89c0909a0 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd88c003420 con 0x7fd89c0909a0 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): dump: 2026-02-21T22:18:02.316 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3134214959 0 0) 0x7fd89c12bf00 con 0x7fd89c1854b0 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.178:6789/0 conn(0x7fd89c091240 legacy=0x7fd89c172bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c0909a0 legacy=0x7fd89c12b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 monclient: found mon.c 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c110df0 con 0x7fd89c1854b0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 5 monclient: authenticate success, global_id 4487 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 10 monclient: _renew_subs 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c1128b0 con 0x7fd89c1854b0 2026-02-21T22:18:02.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 10 monclient: _renew_subs 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd89c111b20 con 0x7fd89c1854b0 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd898004ed0 con 0x7fd89c1854b0 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd898005270 con 0x7fd89c1854b0 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88bfff640 4 set_mon_vals no callback set 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: dump: 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.319 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd89c0008e0 con 0x7fd89c12e790 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd898005540 con 0x7fd89c1854b0 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7fd89804da00 con 0x7fd89c1854b0 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a1269640 10 monclient: get_auth_request con 0x7fd86c03d830 auth_method 0 2026-02-21T22:18:02.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd8a1a6a640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c12e790 legacy=0x7fd89c08dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782628970 0 0) 0x7fd89c0008e0 con 0x7fd89c12e790 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient(hunting): my global_id is 34158 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c12bf00 con 0x7fd89c12e790 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2120812114 0 0) 0x7fd89c12bf00 con 0x7fd89c12e790 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd89c0008e0 con 0x7fd89c12e790 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd88c003ab0 con 0x7fd89c12e790 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.298+0000 7fd88a7fc640 10 monclient: dump: 2026-02-21T22:18:02.321 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4238042005 0 0) 0x7fd89c0008e0 con 0x7fd89c12e790 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd89c12eb60 con 0x7fd89c12e790 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fd88c0058c0 con 0x7fd89c12e790 2026-02-21T22:18:02.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:02.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c12e790 legacy=0x7fd89c08dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c12e790 legacy=0x7fd89c08dce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.324 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd8a3cf5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.302+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd89c112af0 con 0x7fd89c04d280 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a1a6a640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c04d280 legacy=0x7fd89c176a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240248301 0 0) 0x7fd89c112af0 con 0x7fd89c04d280 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient(hunting): my global_id is 34161 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c0008e0 con 0x7fd89c04d280 2026-02-21T22:18:02.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1213268224 0 0) 0x7fd89c0008e0 con 0x7fd89c04d280 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd89c112af0 con 0x7fd89c04d280 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd88c0041c0 con 0x7fd89c04d280 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient: dump: 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.326 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2244010450 0 0) 0x7fd89c112af0 con 0x7fd89c04d280 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd89c095010 con 0x7fd89c04d280 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 248335319) 0x7fd88c005b20 con 0x7fd89c04d280 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c04d280 legacy=0x7fd89c176a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd88a7fc640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6789/0 conn(0x7fd89c04d280 legacy=0x7fd89c176a00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.327 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 31175, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:02.328 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.329 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.330 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:02.331 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:02.332 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.333 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.334 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.335 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.336 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:02.337 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd86c03d830 legacy=0x7fd86c05dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 10 monclient: shutdown 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 >> v1:10.20.193.102:6790/0 conn(0x7fd89c1854b0 legacy=0x7fd89c1742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a226b640 1 -- 10.20.193.102:0/1999033468 reap_dead start 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a226b640 1 -- 10.20.193.102:0/1999033468 reap_dead start 2026-02-21T22:18:02.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 shutdown_connections 2026-02-21T22:18:02.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 >> 10.20.193.102:0/1999033468 conn(0x7fd89c05e660 msgr2=0x7fd89c089b50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 shutdown_connections 2026-02-21T22:18:02.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.306+0000 7fd8a3cf5640 1 -- 10.20.193.102:0/1999033468 wait complete. 2026-02-21T22:18:02.339 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:02.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:18:02.339 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:18:02.352 INFO:tasks.rados.rados.0.trial178.stdout:460: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:460: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 122 (ObjNum 137 snap 33 seq_num 137) dirty exists 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: finishing write tid 1 to trial17818531-19 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:461: finishing copy_from racing read to trial17818531-30 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: finishing write tid 2 to trial17818531-19 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: finishing write tid 3 to trial17818531-19 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:461: finishing copy_from to trial17818531-30 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 69 (ObjNum 115 snap 27 seq_num 115) dirty exists 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: finishing write tid 4 to trial17818531-19 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: oid 19 updating version 0 to 237 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: oid 19 updating version 237 to 238 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: oid 19 version 238 is already newer than 236 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 238 (ObjNum 177 snap 50 seq_num 177) dirty exists 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:462: left oid 19 (ObjNum 177 snap 50 seq_num 177) 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:463: finishing copy_from racing read to trial17818531-17 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:463: finishing copy_from to trial17818531-17 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 77 (ObjNum 132 snap 31 seq_num 132) dirty exists 2026-02-21T22:18:02.353 INFO:tasks.rados.rados.0.trial178.stdout:459: done (4 left) 2026-02-21T22:18:02.354 INFO:tasks.rados.rados.0.trial178.stdout:460: done (3 left) 2026-02-21T22:18:02.354 INFO:tasks.rados.rados.0.trial178.stdout:461: done (2 left) 2026-02-21T22:18:02.354 INFO:tasks.rados.rados.0.trial178.stdout:462: done (1 left) 2026-02-21T22:18:02.354 INFO:tasks.rados.rados.0.trial178.stdout:463: done (0 left) 2026-02-21T22:18:02.354 INFO:tasks.rados.rados.0.trial178.stdout:464: snap_create 2026-02-21T22:18:02.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: get_monmap_and_config 2026-02-21T22:18:02.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: monmap: 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:02.411259+0000 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:02.411259+0000 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 1 Processor -- start 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 1 -- start start 2026-02-21T22:18:02.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: init 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f80a813f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f80a813fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f80a81401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a8092490 con 0x7f80a81401f0 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a80571f0 con 0x7f80a813fd90 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a80574d0 con 0x7f80a813f9c0 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a7fff640 10 monclient(hunting): authenticate will time out at 1291.647251s 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a6ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 0x7f80a80906e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:46996/0 (socket says 10.20.193.102:46996) 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a77fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f80a81401f0 0x7f80a81a5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56742/0 (socket says 10.20.193.102:56742) 2026-02-21T22:18:02.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.406+0000 7f80a6ffd640 1 -- 10.20.193.102:0/2547147754 learned_addr learned my addr 10.20.193.102:0/2547147754 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a77fe640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.102:6789/0 conn(0x7f80a81401f0 legacy=0x7f80a81a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a6ffd640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 legacy=0x7f80a80906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171069532 0 0) 0x7f80a8092490 con 0x7f80a81401f0 2026-02-21T22:18:02.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): my global_id is 34164 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a67fc640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.178:6789/0 conn(0x7f80a813fd90 legacy=0x7f80a8183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8094003620 con 0x7f80a81401f0 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635572715 0 0) 0x7f80a80574d0 con 0x7f80a813f9c0 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient(hunting): my global_id is 4490 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a8092490 con 0x7f80a813f9c0 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1374690153 0 0) 0x7f80a8092490 con 0x7f80a813f9c0 2026-02-21T22:18:02.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.178:6789/0 conn(0x7f80a813fd90 legacy=0x7f80a8183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.102:6789/0 conn(0x7f80a81401f0 legacy=0x7f80a81a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 monclient: found mon.noname-b 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80577b0 con 0x7f80a813f9c0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f809c003190 con 0x7f80a813f9c0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: dump: 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.422 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 5 monclient: authenticate success, global_id 4490 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f809c003420 con 0x7f80a813f9c0 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 1 -- 10.20.193.102:0/2547147754 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f809c004b50 con 0x7f80a813f9c0 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a5ffb640 10 monclient: dump: 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 4 set_mon_vals no callback set 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: shutdown 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- 10.20.193.102:0/2547147754 >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 legacy=0x7f80a80906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- 10.20.193.102:0/2547147754 shutdown_connections 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- 10.20.193.102:0/2547147754 >> 10.20.193.102:0/2547147754 conn(0x7f80a805e3f0 msgr2=0x7f80a8095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- 10.20.193.102:0/2547147754 shutdown_connections 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- 10.20.193.102:0/2547147754 wait complete. 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: monmap: 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:02.425 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 Processor -- start 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- start start 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: init 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-c con 0x7f80a813f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-b con 0x7f80a813fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient(hunting): picked mon.noname-a con 0x7f80a81401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a812f710 con 0x7f80a81401f0 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a808a450 con 0x7f80a813fd90 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a812d3b0 con 0x7f80a813f9c0 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a7fff640 10 monclient(hunting): authenticate will time out at 1291.651780s 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a6ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 0x7f80a811b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:46998/0 (socket says 10.20.193.102:46998) 2026-02-21T22:18:02.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.410+0000 7f80a6ffd640 1 -- 10.20.193.102:0/2207326912 learned_addr learned my addr 10.20.193.102:0/2207326912 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a6ffd640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 legacy=0x7f80a811b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a77fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6789/0 conn(0x7f80a81401f0 legacy=0x7f80a81a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862805412 0 0) 0x7f80a812d3b0 con 0x7f80a813f9c0 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): my global_id is 4493 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8074003620 con 0x7f80a813f9c0 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a67fc640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.178:6789/0 conn(0x7f80a813fd90 legacy=0x7f80a81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358653470 0 0) 0x7f80a812f710 con 0x7f80a81401f0 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): my global_id is 34167 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a812d3b0 con 0x7f80a81401f0 2026-02-21T22:18:02.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4085158060 0 0) 0x7f8074003620 con 0x7f80a813f9c0 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80a812f710 con 0x7f80a813f9c0 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f809c003190 con 0x7f80a813f9c0 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): dump: 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.428 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219860438 0 0) 0x7f80a808a450 con 0x7f80a813fd90 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient(hunting): my global_id is 4462 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8074003620 con 0x7f80a813fd90 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3764118915 0 0) 0x7f80a812f710 con 0x7f80a813f9c0 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.178:6789/0 conn(0x7f80a813fd90 legacy=0x7f80a81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6789/0 conn(0x7f80a81401f0 legacy=0x7f80a81a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 monclient: found mon.c 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a811bab0 con 0x7f80a813f9c0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 5 monclient: authenticate success, global_id 4493 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 10 monclient: _renew_subs 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f80a812f430 con 0x7f80a813f9c0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 10 monclient: _renew_subs 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f80a812e300 con 0x7f80a813f9c0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f809c003420 con 0x7f80a813f9c0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f809c0052a0 con 0x7f80a813f9c0 2026-02-21T22:18:02.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 10 monclient: dump: 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a57fa640 4 set_mon_vals no callback set 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f80a80008e0 con 0x7f80a811ada0 2026-02-21T22:18:02.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f809c011930 con 0x7f80a813f9c0 2026-02-21T22:18:02.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7f809c04c6b0 con 0x7f80a813f9c0 2026-02-21T22:18:02.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.414+0000 7f80a77fe640 10 monclient: get_auth_request con 0x7f807403d4f0 auth_method 0 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80a67fc640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a811ada0 legacy=0x7f80a808d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727968593 0 0) 0x7f80a80008e0 con 0x7f80a811ada0 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient(hunting): my global_id is 4496 2026-02-21T22:18:02.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a812f710 con 0x7f80a811ada0 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2648241884 0 0) 0x7f80a812f710 con 0x7f80a811ada0 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80a80008e0 con 0x7f80a811ada0 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80900031e0 con 0x7f80a811ada0 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient: dump: 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.435 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1708410422 0 0) 0x7f80a80008e0 con 0x7f80a811ada0 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80a8093ce0 con 0x7f80a811ada0 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f80900058c0 con 0x7f80a811ada0 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a811ada0 legacy=0x7f80a808d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.418+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a811ada0 legacy=0x7f80a808d630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80a7fff640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f80a804d650 con 0x7f80a804d280 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80a67fc640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a804d280 legacy=0x7f80a8132c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269214041 0 0) 0x7f80a804d650 con 0x7f80a804d280 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient(hunting): my global_id is 4499 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80a80008e0 con 0x7f80a804d280 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 757563503 0 0) 0x7f80a80008e0 con 0x7f80a804d280 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80a804d650 con 0x7f80a804d280 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8090003ab0 con 0x7f80a804d280 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.422+0000 7f80877fe640 10 monclient: dump: 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.437 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3375884170 0 0) 0x7f80a804d650 con 0x7f80a804d280 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f80a8135390 con 0x7f80a804d280 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 257230037) 0x7f8090005af0 con 0x7f80a804d280 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a804d280 legacy=0x7f80a8132c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80877fe640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a804d280 legacy=0x7f80a8132c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.438 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 165056, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:02.439 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.440 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.441 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:02.442 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.443 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.444 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.445 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.446 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.447 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:02.448 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:02.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.178:6800/2549949439 conn(0x7f807403d4f0 legacy=0x7f807405d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 10 monclient: shutdown 2026-02-21T22:18:02.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 >> v1:10.20.193.102:6790/0 conn(0x7f80a813f9c0 legacy=0x7f80a811b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a77fe640 1 -- 10.20.193.102:0/2207326912 reap_dead start 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a77fe640 1 -- 10.20.193.102:0/2207326912 reap_dead start 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 shutdown_connections 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 >> 10.20.193.102:0/2207326912 conn(0x7f80a805e3f0 msgr2=0x7f80a808a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 shutdown_connections 2026-02-21T22:18:02.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.426+0000 7f80a7fff640 1 -- 10.20.193.102:0/2207326912 wait complete. 2026-02-21T22:18:02.450 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:02.450 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:18:02.450 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 10 monclient: get_monmap_and_config 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 10 monclient: monmap: 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:02.520329+0000 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:02.520329+0000 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.521 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 1 Processor -- start 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 1 -- start start 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.514+0000 7fee8356c640 10 monclient: init 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-b con 0x7fee7c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-a con 0x7fee7c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-c con 0x7fee7c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c086a10 con 0x7fee7c08b7f0 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c08b130 con 0x7fee7c08d8e0 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c08bd60 con 0x7fee7c08ce00 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient(hunting): authenticate will time out at 1291.756224s 2026-02-21T22:18:02.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee80ae0640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fee7c08b7f0 0x7fee7c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56760/0 (socket says 10.20.193.102:56760) 2026-02-21T22:18:02.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee80ae0640 1 -- 10.20.193.102:0/2380414323 learned_addr learned my addr 10.20.193.102:0/2380414323 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee80ae0640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.102:6789/0 conn(0x7fee7c08b7f0 legacy=0x7fee7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee812e1640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.102:6790/0 conn(0x7fee7c08ce00 legacy=0x7fee7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596607361 0 0) 0x7fee7c086a10 con 0x7fee7c08b7f0 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): my global_id is 34170 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee64003620 con 0x7fee7c08b7f0 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276626032 0 0) 0x7fee7c08bd60 con 0x7fee7c08ce00 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient(hunting): my global_id is 4502 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee7c086a10 con 0x7fee7c08ce00 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee81ae2640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.178:6789/0 conn(0x7fee7c08d8e0 legacy=0x7fee7c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3105381060 0 0) 0x7fee64003620 con 0x7fee7c08b7f0 2026-02-21T22:18:02.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.102:6790/0 conn(0x7fee7c08ce00 legacy=0x7fee7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.178:6789/0 conn(0x7fee7c08d8e0 legacy=0x7fee7c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 monclient: found mon.noname-a 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c08b370 con 0x7fee7c08b7f0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee70003190 con 0x7fee7c08b7f0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: dump: 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.527 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 5 monclient: authenticate success, global_id 34170 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee70003420 con 0x7fee7c08b7f0 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 1 -- 10.20.193.102:0/2380414323 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee70004b50 con 0x7fee7c08b7f0 2026-02-21T22:18:02.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee6bfff640 10 monclient: dump: 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 4 set_mon_vals no callback set 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: shutdown 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- 10.20.193.102:0/2380414323 >> v1:10.20.193.102:6789/0 conn(0x7fee7c08b7f0 legacy=0x7fee7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- 10.20.193.102:0/2380414323 shutdown_connections 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- 10.20.193.102:0/2380414323 >> 10.20.193.102:0/2380414323 conn(0x7fee7c091a50 msgr2=0x7fee7c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- 10.20.193.102:0/2380414323 shutdown_connections 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 1 -- 10.20.193.102:0/2380414323 wait complete. 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.518+0000 7fee8356c640 10 monclient: monmap: 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 Processor -- start 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- start start 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: init 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-b con 0x7fee7c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-a con 0x7fee7c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient(hunting): picked mon.noname-c con 0x7fee7c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c082ea0 con 0x7fee7c08ce00 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c058920 con 0x7fee7c08b7f0 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c05e100 con 0x7fee7c08d8e0 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient(hunting): authenticate will time out at 1291.760662s 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee80ae0640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fee7c08ce00 0x7fee7c084970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56770/0 (socket says 10.20.193.102:56770) 2026-02-21T22:18:02.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee80ae0640 1 -- 10.20.193.102:0/427026352 learned_addr learned my addr 10.20.193.102:0/427026352 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee80ae0640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.102:6789/0 conn(0x7fee7c08ce00 legacy=0x7fee7c084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee81ae2640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.102:6790/0 conn(0x7fee7c08d8e0 legacy=0x7fee7c1955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197733968 0 0) 0x7fee7c082ea0 con 0x7fee7c08ce00 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): my global_id is 34173 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee4c003620 con 0x7fee7c08ce00 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193573179 0 0) 0x7fee7c05e100 con 0x7fee7c08d8e0 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): my global_id is 4505 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee7c082ea0 con 0x7fee7c08d8e0 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c08b7f0 legacy=0x7fee7c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3920822606 0 0) 0x7fee7c082ea0 con 0x7fee7c08d8e0 2026-02-21T22:18:02.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee7c05e100 con 0x7fee7c08d8e0 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3273591663 0 0) 0x7fee4c003620 con 0x7fee7c08ce00 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee7c082ea0 con 0x7fee7c08ce00 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee78004890 con 0x7fee7c08d8e0 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): dump: 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee70003190 con 0x7fee7c08ce00 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): dump: 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054538367 0 0) 0x7fee7c058920 con 0x7fee7c08b7f0 2026-02-21T22:18:02.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient(hunting): my global_id is 4468 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee4c003620 con 0x7fee7c08b7f0 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2973908489 0 0) 0x7fee7c082ea0 con 0x7fee7c08ce00 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.102:6790/0 conn(0x7fee7c08d8e0 legacy=0x7fee7c1955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c08b7f0 legacy=0x7fee7c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 monclient: found mon.a 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c088a70 con 0x7fee7c08ce00 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 5 monclient: authenticate success, global_id 34173 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _renew_subs 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c081c60 con 0x7fee7c08ce00 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _renew_subs 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fee7c085250 con 0x7fee7c08ce00 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee70003330 con 0x7fee7c08ce00 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee70003510 con 0x7fee7c08ce00 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee6b7fe640 4 set_mon_vals no callback set 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee69ffb640 10 monclient: dump: 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.539 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.522+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fee7c0008e0 con 0x7fee7c05b800 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fee70011680 con 0x7fee7c08ce00 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7fee7004c650 con 0x7fee7c08ce00 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee81ae2640 10 monclient: get_auth_request con 0x7fee4c03d820 auth_method 0 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05b800 legacy=0x7fee7c05bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128827192 0 0) 0x7fee7c0008e0 con 0x7fee7c05b800 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient(hunting): my global_id is 4471 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee7c082ea0 con 0x7fee7c05b800 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2010792671 0 0) 0x7fee7c082ea0 con 0x7fee7c05b800 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee7c0008e0 con 0x7fee7c05b800 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee6c003280 con 0x7fee7c05b800 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient: dump: 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.541 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2817578653 0 0) 0x7fee7c0008e0 con 0x7fee7c05b800 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee7c090810 con 0x7fee7c05b800 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fee6c0058c0 con 0x7fee7c05b800 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05b800 legacy=0x7fee7c05bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.526+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05b800 legacy=0x7fee7c05bbd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.530+0000 7fee8356c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.530+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fee7c0909c0 con 0x7fee7c05c2e0 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 legacy=0x7fee7c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 legacy=0x7fee7c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 legacy=0x7fee7c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 --1- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 0x7fee7c197d40 :-1 s=OPENED pgs=204 gs=6 cs=1 l=1).handle_message read tag failed 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 --1- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 0x7fee7c197d40 :-1 s=OPENED pgs=204 gs=6 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 legacy=0x7fee7c197d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee7c05c2e0 legacy=0x7fee7c197d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fee7c0008e0 con 0x7fee4c063b20 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.534+0000 7fee812e1640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee4c063b20 legacy=0x7fee4c062590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300626838 0 0) 0x7fee7c0008e0 con 0x7fee4c063b20 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient(hunting): my global_id is 4474 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee7c082ea0 con 0x7fee4c063b20 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 87583880 0 0) 0x7fee7c082ea0 con 0x7fee4c063b20 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee7c0008e0 con 0x7fee4c063b20 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fee6c003b30 con 0x7fee4c063b20 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient: dump: 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.545 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 706918219 0 0) 0x7fee7c0008e0 con 0x7fee4c063b20 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fee6c0058c0 con 0x7fee4c063b20 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3986962773) 0x7fee6c005b10 con 0x7fee4c063b20 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee4c063b20 legacy=0x7fee4c062590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee69ffb640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6789/0 conn(0x7fee4c063b20 legacy=0x7fee4c062590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee81ae2640 1 -- 10.20.193.102:0/427026352 reap_dead start 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:02.546 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 16, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 165144, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.547 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.548 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.549 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:02.550 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.551 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.552 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.553 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.554 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.555 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:02.556 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.178:6800/2549949439 conn(0x7fee4c03d820 legacy=0x7fee4c05dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 10 monclient: shutdown 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 >> v1:10.20.193.102:6789/0 conn(0x7fee7c08ce00 legacy=0x7fee7c084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 shutdown_connections 2026-02-21T22:18:02.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 >> 10.20.193.102:0/427026352 conn(0x7fee7c091a50 msgr2=0x7fee7c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 shutdown_connections 2026-02-21T22:18:02.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.538+0000 7fee8356c640 1 -- 10.20.193.102:0/427026352 wait complete. 2026-02-21T22:18:02.558 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:18:02.558 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-21T22:18:02.558 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:18:02.558 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-21T22:18:02.558 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-21T22:18:02.558 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:18:02.558 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:18:02.572 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:18:02.572 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:18:02.572 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:18:02.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:18:02.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:18:02.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:18:02.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:18:02.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:18:02.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:18:02.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: get_monmap_and_config 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: monmap: 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.632 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:02.631315+0000 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:02.631315+0000 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 1 Processor -- start 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 1 -- start start 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: init 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-a con 0x7fbfdc08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-b con 0x7fbfdc08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-c con 0x7fbfdc08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc086a10 con 0x7fbfdc08ce00 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc08b130 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc08bd60 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe24d0640 10 monclient(hunting): authenticate will time out at 1291.867228s 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfe0a46640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fbfdc08d8e0 legacy=0x7fbfdc1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfdbfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbfdc08ce00 0x7fbfdc08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56792/0 (socket says 10.20.193.102:56792) 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfdb7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 0x7fbfdc181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:47074/0 (socket says 10.20.193.102:47074) 2026-02-21T22:18:02.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.626+0000 7fbfdbfff640 1 -- 10.20.193.102:0/2464059310 learned_addr learned my addr 10.20.193.102:0/2464059310 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdb7fe640 1 -- 10.20.193.102:0/2464059310 >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 legacy=0x7fbfdc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdbfff640 1 -- 10.20.193.102:0/2464059310 >> v1:10.20.193.102:6789/0 conn(0x7fbfdc08ce00 legacy=0x7fbfdc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437046552 0 0) 0x7fbfdc08bd60 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): my global_id is 4508 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfc0003620 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949076802 0 0) 0x7fbfdc086a10 con 0x7fbfdc08ce00 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient(hunting): my global_id is 34176 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfdc08bd60 con 0x7fbfdc08ce00 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124043891 0 0) 0x7fbfc0003620 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 >> v1:10.20.193.178:6789/0 conn(0x7fbfdc08d8e0 legacy=0x7fbfdc1a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 >> v1:10.20.193.102:6789/0 conn(0x7fbfdc08ce00 legacy=0x7fbfdc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 monclient: found mon.noname-b 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc08b370 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbfcc003110 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 5 monclient: authenticate success, global_id 4508 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: dump: 2026-02-21T22:18:02.639 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbfcc0033e0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 1 -- 10.20.193.102:0/2464059310 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbfcc004a90 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdaffd640 10 monclient: dump: 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 4 set_mon_vals no callback set 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: shutdown 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- 10.20.193.102:0/2464059310 >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 legacy=0x7fbfdc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- 10.20.193.102:0/2464059310 shutdown_connections 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- 10.20.193.102:0/2464059310 >> 10.20.193.102:0/2464059310 conn(0x7fbfdc091a50 msgr2=0x7fbfdc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- 10.20.193.102:0/2464059310 shutdown_connections 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- 10.20.193.102:0/2464059310 wait complete. 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: build_initial_monmap 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: monmap: 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:02.642 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 Processor -- start 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- start start 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: init 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-c con 0x7fbfdc08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-b con 0x7fbfdc08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient(hunting): picked mon.noname-a con 0x7fbfdc08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc088a70 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc080e60 con 0x7fbfdc08ce00 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfdc082920 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfe24d0640 10 monclient(hunting): authenticate will time out at 1291.871338s 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdbfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 0x7fbfdc0846f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:47086/0 (socket says 10.20.193.102:47086) 2026-02-21T22:18:02.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdbfff640 1 -- 10.20.193.102:0/1135079801 learned_addr learned my addr 10.20.193.102:0/1135079801 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:02.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.630+0000 7fbfdb7fe640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.178:6789/0 conn(0x7fbfdc08ce00 legacy=0x7fbfdc194300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:02.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfdbfff640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 legacy=0x7fbfdc0846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe0a46640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.102:6789/0 conn(0x7fbfdc08d8e0 legacy=0x7fbfdc195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:02.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898280398 0 0) 0x7fbfdc082920 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): my global_id is 4511 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfac003620 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596611260 0 0) 0x7fbfdc088a70 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): my global_id is 34179 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfdc082920 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3752774588 0 0) 0x7fbfac003620 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbfdc088a70 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbfc80048e0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): dump: 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3755588145 0 0) 0x7fbfdc082920 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbfac003620 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbfd00058f0 con 0x7fbfdc08d8e0 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient(hunting): dump: 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1999328633 0 0) 0x7fbfdc088a70 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.178:6789/0 conn(0x7fbfdc08ce00 legacy=0x7fbfdc194300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:18:02.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.102:6789/0 conn(0x7fbfdc08d8e0 legacy=0x7fbfdc195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 monclient: found mon.c 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc0831c0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: _finish_auth 0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 5 monclient: authenticate success, global_id 4511 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _renew_subs 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc084e00 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _renew_subs 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbfdc081b90 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbfc8004d30 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbfc80050d0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 10 monclient: dump: 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:02.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfda7fc640 4 set_mon_vals no callback set 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc0008e0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fbfc8005370 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7fbfc804c820 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.634+0000 7fbfdb7fe640 10 monclient: get_auth_request con 0x7fbfac03d520 auth_method 0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.638+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fbfc80177d0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.638+0000 7fbfd8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:02.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.638+0000 7fbfd8ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbfdc090810 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfd8ff9640 1 -- 10.20.193.102:0/1135079801 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 181072971) 0x7fbfc80170b0 con 0x7fbfdc08b7f0 2026-02-21T22:18:02.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfd8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:02.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfd8ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:02.757 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.178:6800/2549949439 conn(0x7fbfac03d520 legacy=0x7fbfac05d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 10 monclient: shutdown 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 >> v1:10.20.193.102:6790/0 conn(0x7fbfdc08b7f0 legacy=0x7fbfdc0846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 shutdown_connections 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.750+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 >> 10.20.193.102:0/1135079801 conn(0x7fbfdc091a50 msgr2=0x7fbfdc088d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:02.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.754+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 shutdown_connections 2026-02-21T22:18:02.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:02.754+0000 7fbfe24d0640 1 -- 10.20.193.102:0/1135079801 wait complete. 2026-02-21T22:18:03.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:03.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:05.767 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:18:05.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: get_monmap_and_config 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: build_initial_monmap 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: monmap: 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:05.838378+0000 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:05.838378+0000 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:05.840 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 1 Processor -- start 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 1 -- start start 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: init 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-a con 0x7fa3b008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-b con 0x7fa3b008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-c con 0x7fa3b008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b0086a10 con 0x7fa3b008ce00 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b008b130 con 0x7fa3b008d8e0 2026-02-21T22:18:05.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b008bd60 con 0x7fa3b008b7f0 2026-02-21T22:18:05.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:05.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3b51ff640 10 monclient(hunting): authenticate will time out at 1295.074274s 2026-02-21T22:18:05.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3aed76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 0x7fa3b008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56796/0 (socket says 10.20.193.102:56796) 2026-02-21T22:18:05.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3aed76640 1 -- 10.20.193.102:0/1403021009 learned_addr learned my addr 10.20.193.102:0/1403021009 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:05.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.834+0000 7fa3af577640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.178:6789/0 conn(0x7fa3b008d8e0 legacy=0x7fa3b01a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:05.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3aed76640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 legacy=0x7fa3b008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:05.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3ae575640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.102:6790/0 conn(0x7fa3b008b7f0 legacy=0x7fa3b0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380645156 0 0) 0x7fa3b0086a10 con 0x7fa3b008ce00 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _negotiate 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): my global_id is 34182 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa39c003620 con 0x7fa3b008ce00 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210473572 0 0) 0x7fa3b008bd60 con 0x7fa3b008b7f0 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _negotiate 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:05.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient(hunting): my global_id is 4514 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3b0086a10 con 0x7fa3b008b7f0 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260034949 0 0) 0x7fa39c003620 con 0x7fa3b008ce00 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.102:6790/0 conn(0x7fa3b008b7f0 legacy=0x7fa3b0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.178:6789/0 conn(0x7fa3b008d8e0 legacy=0x7fa3b01a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: _finish_hunting 0 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 monclient: found mon.noname-a 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b008b370 con 0x7fa3b008ce00 2026-02-21T22:18:05.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: _finish_auth 0 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: _check_auth_tickets 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3a4003110 con 0x7fa3b008ce00 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: dump: 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:05.851 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 5 monclient: authenticate success, global_id 34182 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3a40033e0 con 0x7fa3b008ce00 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 1 -- 10.20.193.102:0/1403021009 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3a4004b50 con 0x7fa3b008ce00 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:05.852 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3add74640 10 monclient: dump: 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:05.853 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 4 set_mon_vals no callback set 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: shutdown 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1403021009 >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 legacy=0x7fa3b008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1403021009 shutdown_connections 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1403021009 >> 10.20.193.102:0/1403021009 conn(0x7fa3b0091a50 msgr2=0x7fa3b0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1403021009 shutdown_connections 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1403021009 wait complete. 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: build_initial_monmap 2026-02-21T22:18:05.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: monmap: 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 Processor -- start 2026-02-21T22:18:05.855 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- start start 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: init 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-b con 0x7fa3b008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-a con 0x7fa3b008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient(hunting): picked mon.noname-c con 0x7fa3b008d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b00829a0 con 0x7fa3b008ce00 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b0088a70 con 0x7fa3b008b7f0 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b0080ee0 con 0x7fa3b008d8e0 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3b51ff640 10 monclient(hunting): authenticate will time out at 1295.078391s 2026-02-21T22:18:05.856 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3aed76640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fa3b008b7f0 legacy=0x7fa3b0153030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3ae575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 0x7fa3b0084770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56802/0 (socket says 10.20.193.102:56802) 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.838+0000 7fa3ae575640 1 -- 10.20.193.102:0/1709913006 learned_addr learned my addr 10.20.193.102:0/1709913006 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3ae575640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 legacy=0x7fa3b0084770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3af577640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.102:6790/0 conn(0x7fa3b008d8e0 legacy=0x7fa3b01953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779217324 0 0) 0x7fa398003450 con 0x7fa3b008ce00 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): my global_id is 34185 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa37c003620 con 0x7fa3b008ce00 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884638091 0 0) 0x7fa3b0080ee0 con 0x7fa3b008d8e0 2026-02-21T22:18:05.857 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): my global_id is 4517 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa398003450 con 0x7fa3b008d8e0 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 344025524 0 0) 0x7fa37c003620 con 0x7fa3b008ce00 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa3b0080ee0 con 0x7fa3b008ce00 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3980038c0 con 0x7fa3b008ce00 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): dump: 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:05.858 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2317487090 0 0) 0x7fa398003450 con 0x7fa3b008d8e0 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa37c003620 con 0x7fa3b008d8e0 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3a00058f0 con 0x7fa3b008d8e0 2026-02-21T22:18:05.859 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient(hunting): dump: 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:05.860 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3983844179 0 0) 0x7fa3b0080ee0 con 0x7fa3b008ce00 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.102:6790/0 conn(0x7fa3b008d8e0 legacy=0x7fa3b01953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.178:6789/0 conn(0x7fa3b008b7f0 legacy=0x7fa3b0153030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 monclient: found mon.a 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b0153f00 con 0x7fa3b008ce00 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:05.861 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 5 monclient: authenticate success, global_id 34185 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _renew_subs 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b0083240 con 0x7fa3b008ce00 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _renew_subs 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa3b0084e80 con 0x7fa3b008ce00 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa398003d90 con 0x7fa3b008ce00 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa398004120 con 0x7fa3b008ce00 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 10 monclient: dump: 2026-02-21T22:18:05.862 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3ad573640 4 set_mon_vals no callback set 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:05.863 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:05.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b00008e0 con 0x7fa3b008ce00 2026-02-21T22:18:05.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa3980043f0 con 0x7fa3b008ce00 2026-02-21T22:18:05.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7fa39804c9a0 con 0x7fa3b008ce00 2026-02-21T22:18:05.864 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.842+0000 7fa3aed76640 10 monclient: get_auth_request con 0x7fa37c03d5c0 auth_method 0 2026-02-21T22:18:05.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.846+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fa3b00829a0 con 0x7fa3b008ce00 2026-02-21T22:18:05.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.846+0000 7fa38f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:05.865 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.846+0000 7fa38f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa3b0090810 con 0x7fa3b008ce00 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa38f7fe640 1 -- 10.20.193.102:0/1709913006 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1138847147) 0x7fa398017950 con 0x7fa3b008ce00 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa38f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:05.967 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa38f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:05.968 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:18:05.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa37c03d5c0 legacy=0x7fa37c05da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 10 monclient: shutdown 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 >> v1:10.20.193.102:6789/0 conn(0x7fa3b008ce00 legacy=0x7fa3b0084770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 shutdown_connections 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 >> 10.20.193.102:0/1709913006 conn(0x7fa3b0091a50 msgr2=0x7fa3b005dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 shutdown_connections 2026-02-21T22:18:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:05.962+0000 7fa3b51ff640 1 -- 10.20.193.102:0/1709913006 wait complete. 2026-02-21T22:18:08.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:08.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:08.978 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: get_monmap_and_config 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: build_initial_monmap 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: monmap: 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:09.042805+0000 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:09.042805+0000 2026-02-21T22:18:09.044 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 1 Processor -- start 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 1 -- start start 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: init 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient(hunting): picked mon.noname-c con 0x7f476013f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:09.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient(hunting): picked mon.noname-a con 0x7f476013fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient(hunting): picked mon.noname-b con 0x7f47601401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4760092490 con 0x7f476013fd90 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47600571f0 con 0x7f476013f9c0 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47600574d0 con 0x7f47601401f0 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f476663b640 10 monclient(hunting): authenticate will time out at 1298.278758s 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f4765639640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f47600906e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f4764e38640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f476013fd90 0x7f4760183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56816/0 (socket says 10.20.193.102:56816) 2026-02-21T22:18:09.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.038+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 learned_addr learned my addr 10.20.193.102:0/1882713162 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:09.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.042+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f476013fd90 legacy=0x7f4760183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:09.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.042+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f47601a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:09.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.242+0000 7f4765639640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f47600906e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:09.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:09.646+0000 7f4765639640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f47600906e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:10.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:10.450+0000 7f4765639640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f47600906e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:12.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): tick 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 monclient(hunting): continuing hunt 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f47601a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f47600906e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f476013fd90 legacy=0x7f4760183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-a con 0x7f47440038c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-b con 0x7f47601401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:12.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-c con 0x7f476013f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:12.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47600577b0 con 0x7f47440038c0 2026-02-21T22:18:12.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47600571f0 con 0x7f476013f9c0 2026-02-21T22:18:12.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474406a5e0 con 0x7f47601401f0 2026-02-21T22:18:12.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f474effd640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:12.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:12.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f4744045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:12.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.042+0000 7f4765639640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f47440038c0 legacy=0x7f4744023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:12.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.246+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:12.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:12.650+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:13.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:13.454+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:13.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:13.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:15.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): tick 2026-02-21T22:18:15.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 monclient(hunting): continuing hunt 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f4744045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f47440038c0 legacy=0x7f4744023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-b con 0x7f47601401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-c con 0x7f476013f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): picked mon.noname-a con 0x7f47440038c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474406b940 con 0x7f47440038c0 2026-02-21T22:18:15.045 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47600571f0 con 0x7f476013f9c0 2026-02-21T22:18:15.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474406a9d0 con 0x7f47601401f0 2026-02-21T22:18:15.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f474effd640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:15.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:15.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f4765639640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f4744045510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:15.046 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.042+0000 7f4765e3a640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f47440038c0 legacy=0x7f4744023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:15.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.246+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:15.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:15.650+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:16.453+0000 7f4764e38640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066d80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064835729 0 0) 0x7f474406a9d0 con 0x7f47601401f0 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 10 monclient(hunting): my global_id is 4520 2026-02-21T22:18:17.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.261+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4748003620 con 0x7f47601401f0 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 853528313 0 0) 0x7f4748003620 con 0x7f47601401f0 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.178:6789/0 conn(0x7f476013f9c0 legacy=0x7f4744066d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6789/0 conn(0x7f47440038c0 legacy=0x7f4744023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 monclient: found mon.noname-b 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474406ac60 con 0x7f47601401f0 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: _finish_auth 0 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 5 monclient: authenticate success, global_id 4520 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4750004550 con 0x7f47601401f0 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: dump: 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.273 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f4765639640 1 --1- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 0x7f4744045510 :-1 s=OPENED pgs=228 gs=15 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f4750004c20 mon_map magic: 0 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4750004890 con 0x7f47601401f0 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:17.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 4 set_mon_vals no callback set 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: shutdown 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- 10.20.193.102:0/1882713162 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f4744045510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 1 -- 10.20.193.102:0/1882713162 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4750004c20 con 0x7f47601401f0 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f474ffff640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- 10.20.193.102:0/1882713162 shutdown_connections 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- 10.20.193.102:0/1882713162 >> 10.20.193.102:0/1882713162 conn(0x7f476005e3f0 msgr2=0x7f4760095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- 10.20.193.102:0/1882713162 shutdown_connections 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- 10.20.193.102:0/1882713162 wait complete. 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: build_initial_monmap 2026-02-21T22:18:17.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: monmap: 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 Processor -- start 2026-02-21T22:18:17.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- start start 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: init 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient(hunting): picked mon.noname-b con 0x7f47440038c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient(hunting): picked mon.noname-a con 0x7f476013f9c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient(hunting): picked mon.noname-c con 0x7f47601401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476012e900 con 0x7f476013f9c0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476012bda0 con 0x7f47440038c0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476012f090 con 0x7f47601401f0 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f476663b640 10 monclient(hunting): authenticate will time out at 1306.506881s 2026-02-21T22:18:17.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f4765639640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f47440038c0 legacy=0x7f476012af80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:17.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f4764e38640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f476013f9c0 0x7f476012b690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:57072/0 (socket says 10.20.193.102:57072) 2026-02-21T22:18:17.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.265+0000 7f4764e38640 1 -- 10.20.193.102:0/3855517576 learned_addr learned my addr 10.20.193.102:0/3855517576 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:17.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f4764e38640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.102:6789/0 conn(0x7f476013f9c0 legacy=0x7f476012b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f4765e3a640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f476011c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111709375 0 0) 0x7f476012f090 con 0x7f47601401f0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): my global_id is 4523 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4730003620 con 0x7f47601401f0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 528845943 0 0) 0x7f4730003620 con 0x7f47601401f0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f476012f090 con 0x7f47601401f0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f475c0047f0 con 0x7f47601401f0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:17.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): dump: 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:17.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4288856330 0 0) 0x7f476012f090 con 0x7f47601401f0 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.178:6789/0 conn(0x7f47440038c0 legacy=0x7f476012af80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.102:6789/0 conn(0x7f476013f9c0 legacy=0x7f476012b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 monclient: found mon.c 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476012ebe0 con 0x7f47601401f0 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: _finish_auth 0 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:17.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 5 monclient: authenticate success, global_id 4523 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _renew_subs 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f47600590d0 con 0x7f47601401f0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _renew_subs 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f476012e620 con 0x7f47601401f0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f475c004a10 con 0x7f47601401f0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f475c005080 con 0x7f47601401f0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474f7fe640 4 set_mon_vals no callback set 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:17.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 10 monclient: dump: 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:17.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47600008e0 con 0x7f47601401f0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f475c005220 con 0x7f47601401f0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7f475c04da00 con 0x7f47601401f0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.269+0000 7f4765639640 10 monclient: get_auth_request con 0x7f473003d370 auth_method 0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.273+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f475c018a10 con 0x7f47601401f0 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.273+0000 7f474dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:17.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.273+0000 7f474dffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:17.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.385+0000 7f476663b640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.385+0000 7f476663b640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.385+0000 7f476663b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.385+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4760093ce0 con 0x7f47601401f0 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f474dffb640 1 -- 10.20.193.102:0/3855517576 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 4020397943) 0x7f475c0182f0 con 0x7f47601401f0 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f474dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:18:17.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f474dffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:17.394 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":18,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:18:17.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.178:6800/2549949439 conn(0x7f473003d370 legacy=0x7f473005d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 10 monclient: shutdown 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 >> v1:10.20.193.102:6790/0 conn(0x7f47601401f0 legacy=0x7f476011c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 shutdown_connections 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 >> 10.20.193.102:0/3855517576 conn(0x7f476005e3f0 msgr2=0x7f4760091630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 shutdown_connections 2026-02-21T22:18:17.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.389+0000 7f476663b640 1 -- 10.20.193.102:0/3855517576 wait complete. 2026-02-21T22:18:17.403 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:18:17.404 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:18:17.404 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: get_monmap_and_config 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: build_initial_monmap 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: monmap: 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.476 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:17.474886+0000 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:17.474886+0000 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 1 Processor -- start 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 1 -- start start 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: init 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:17.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient(hunting): picked mon.noname-b con 0x7f193c08e3a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient(hunting): picked mon.noname-c con 0x7f193c08ee80 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient(hunting): picked mon.noname-a con 0x7f193c058e30 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c08fba0 con 0x7f193c058e30 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c0592b0 con 0x7f193c08ee80 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c057bd0 con 0x7f193c08e3a0 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f194303a640 10 monclient(hunting): authenticate will time out at 1306.710842s 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f1941837640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f193c08ee80 legacy=0x7f193c1842e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f1942038640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f193c08e3a0 0x7f193c08e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42526/0 (socket says 10.20.193.102:42526) 2026-02-21T22:18:17.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.469+0000 7f1942038640 1 -- 10.20.193.102:0/3371511583 learned_addr learned my addr 10.20.193.102:0/3371511583 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1942839640 1 -- 10.20.193.102:0/3371511583 >> v1:10.20.193.102:6789/0 conn(0x7f193c058e30 legacy=0x7f193c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1942038640 1 -- 10.20.193.102:0/3371511583 >> v1:10.20.193.102:6790/0 conn(0x7f193c08e3a0 legacy=0x7f193c08e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149708596 0 0) 0x7f193c057bd0 con 0x7f193c08e3a0 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient(hunting): my global_id is 4526 2026-02-21T22:18:17.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1924003620 con 0x7f193c08e3a0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 260469192 0 0) 0x7f1924003620 con 0x7f193c08e3a0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 >> v1:10.20.193.178:6789/0 conn(0x7f193c08ee80 legacy=0x7f193c1842e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 >> v1:10.20.193.102:6789/0 conn(0x7f193c058e30 legacy=0x7f193c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: _finish_hunting 0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 monclient: found mon.noname-b 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c057eb0 con 0x7f193c08e3a0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: _finish_auth 0 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: _check_auth_tickets 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:17.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f192c003110 con 0x7f193c08e3a0 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 5 monclient: authenticate success, global_id 4526 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: dump: 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.484 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f192c0033e0 con 0x7f193c08e3a0 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 1 -- 10.20.193.102:0/3371511583 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f192c004b50 con 0x7f193c08e3a0 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941036640 10 monclient: dump: 2026-02-21T22:18:17.485 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 4 set_mon_vals no callback set 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: shutdown 2026-02-21T22:18:17.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- 10.20.193.102:0/3371511583 >> v1:10.20.193.102:6790/0 conn(0x7f193c08e3a0 legacy=0x7f193c08e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- 10.20.193.102:0/3371511583 shutdown_connections 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- 10.20.193.102:0/3371511583 >> 10.20.193.102:0/3371511583 conn(0x7f193c056cc0 msgr2=0x7f193c0939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- 10.20.193.102:0/3371511583 shutdown_connections 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- 10.20.193.102:0/3371511583 wait complete. 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: build_initial_monmap 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: monmap: 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:17.487 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 Processor -- start 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- start start 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: init 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient(hunting): picked mon.noname-c con 0x7f193c058e30 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:17.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient(hunting): picked mon.noname-b con 0x7f193c08e3a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient(hunting): picked mon.noname-a con 0x7f193c08ee80 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c08a110 con 0x7f193c08ee80 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c08ab90 con 0x7f193c08e3a0 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c13f610 con 0x7f193c058e30 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f194303a640 10 monclient(hunting): authenticate will time out at 1306.715358s 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1942038640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f193c058e30 0x7f193c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42538/0 (socket says 10.20.193.102:42538) 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1942038640 1 -- 10.20.193.102:0/3581496481 learned_addr learned my addr 10.20.193.102:0/3581496481 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.473+0000 7f1941837640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.178:6789/0 conn(0x7f193c08e3a0 legacy=0x7f193c194820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1942038640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6790/0 conn(0x7f193c058e30 legacy=0x7f193c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1942839640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c08ee80 legacy=0x7f193c195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764877855 0 0) 0x7f193c13f610 con 0x7f193c058e30 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): my global_id is 4529 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f190c003620 con 0x7f193c058e30 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 911499474 0 0) 0x7f190c003620 con 0x7f193c058e30 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f193c13f610 con 0x7f193c058e30 2026-02-21T22:18:17.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f192c0048e0 con 0x7f193c058e30 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): dump: 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.494 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.477+0000 7f1922ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1072676260 0 0) 0x7f193c13f610 con 0x7f193c058e30 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.178:6789/0 conn(0x7f193c08e3a0 legacy=0x7f193c194820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c08ee80 legacy=0x7f193c195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 monclient: found mon.c 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c141970 con 0x7f193c058e30 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: _finish_auth 0 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:18:17.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 5 monclient: authenticate success, global_id 4529 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 10 monclient: _renew_subs 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1410d0 con 0x7f193c058e30 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 10 monclient: _renew_subs 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f193c143dd0 con 0x7f193c058e30 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f192c003380 con 0x7f193c058e30 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f192c005320 con 0x7f193c058e30 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 10 monclient: dump: 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1940835640 4 set_mon_vals no callback set 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f193c0008e0 con 0x7f193c13ec50 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f192c0119b0 con 0x7f193c058e30 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(117..117 src has 1..117) ==== 3915+0+0 (unknown 3275471806 0 0) 0x7f192c04c740 con 0x7f193c058e30 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1942839640 10 monclient: get_auth_request con 0x7f190c03d390 auth_method 0 2026-02-21T22:18:17.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.481+0000 7f1941837640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c13ec50 legacy=0x7f193c139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200840572 0 0) 0x7f193c0008e0 con 0x7f193c13ec50 2026-02-21T22:18:17.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient(hunting): my global_id is 34188 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f193c13f610 con 0x7f193c13ec50 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2315864537 0 0) 0x7f193c13f610 con 0x7f193c13ec50 2026-02-21T22:18:17.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f193c0008e0 con 0x7f193c13ec50 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1930004330 con 0x7f193c13ec50 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 10 monclient: dump: 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.735 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.736 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.737 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.737 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3017125301 0 0) 0x7f193c0008e0 con 0x7f193c13ec50 2026-02-21T22:18:17.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.725+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f193c057440 con 0x7f193c13ec50 2026-02-21T22:18:17.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.729+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f1930006760 con 0x7f193c13ec50 2026-02-21T22:18:17.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.729+0000 7f1922ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:17.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.729+0000 7f1922ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:17.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.729+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c13ec50 legacy=0x7f193c139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.729+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c13ec50 legacy=0x7f193c139bf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:17.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.733+0000 7f194303a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:17.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.733+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f193c1414b0 con 0x7f193c153d20 2026-02-21T22:18:17.740 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.733+0000 7f1941837640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c153d20 legacy=0x7f193c04d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199685565 0 0) 0x7f193c1414b0 con 0x7f193c153d20 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient(hunting): my global_id is 34191 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f193c0008e0 con 0x7f193c153d20 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3467136078 0 0) 0x7f193c0008e0 con 0x7f193c153d20 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f193c1414b0 con 0x7f193c153d20 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1930004b30 con 0x7f193c153d20 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient: dump: 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3955192586 0 0) 0x7f193c1414b0 con 0x7f193c153d20 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f193c051160 con 0x7f193c153d20 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4502 (unknown 0 0 3943669674) 0x7f1930006910 con 0x7f193c153d20 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c153d20 legacy=0x7f193c04d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1922ffd640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6789/0 conn(0x7f193c153d20 legacy=0x7f193c04d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:17.977 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 18, 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 46838, 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:17.978 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:17.979 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:17.980 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:17.981 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:17.982 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:17.983 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:17.984 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:17.985 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:17.986 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:17.987 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:17.988 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:17.989 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:17.990 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:17.991 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.178:6800/2549949439 conn(0x7f190c03d390 legacy=0x7f190c05d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 10 monclient: shutdown 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 >> v1:10.20.193.102:6790/0 conn(0x7f193c058e30 legacy=0x7f193c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1942839640 1 -- 10.20.193.102:0/3581496481 reap_dead start 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f1942839640 1 -- 10.20.193.102:0/3581496481 reap_dead start 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 shutdown_connections 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 >> 10.20.193.102:0/3581496481 conn(0x7f193c056cc0 msgr2=0x7f193c08bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 shutdown_connections 2026-02-21T22:18:17.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:17.969+0000 7f194303a640 1 -- 10.20.193.102:0/3581496481 wait complete. 2026-02-21T22:18:17.994 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:18:17.994 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:18:17.995 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:18:17.997 INFO:tasks.rados.rados.0.trial178.stdout:464: done (0 left) 2026-02-21T22:18:17.997 INFO:tasks.rados.rados.0.trial178.stdout:465: write oid 50 current snap is 51 2026-02-21T22:18:17.997 INFO:tasks.rados.rados.0.trial178.stdout:465: seq_num 178 ranges {591646=470441,1566383=595717,2888822=102824} 2026-02-21T22:18:17.997 INFO:tasks.rados.rados.0.trial178.stdout:465: writing trial17818531-50 from 591646 to 1062087 tid 1 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: writing trial17818531-50 from 1566383 to 2162100 tid 2 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: writing trial17818531-50 from 2888822 to 2991646 tid 3 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:466: read oid 37 snap -1 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:466: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:467: snap_create 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: finishing write tid 1 to trial17818531-50 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: finishing write tid 2 to trial17818531-50 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: finishing write tid 3 to trial17818531-50 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: finishing write tid 4 to trial17818531-50 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: finishing write tid 5 to trial17818531-50 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: oid 50 updating version 0 to 70 2026-02-21T22:18:17.998 INFO:tasks.rados.rados.0.trial178.stdout:465: oid 50 updating version 70 to 72 2026-02-21T22:18:17.999 INFO:tasks.rados.rados.0.trial178.stdout:465: oid 50 version 72 is already newer than 71 2026-02-21T22:18:17.999 INFO:tasks.rados.rados.0.trial178.stdout:465: oid 50 version 72 is already newer than 69 2026-02-21T22:18:17.999 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 72 (ObjNum 178 snap 51 seq_num 178) dirty exists 2026-02-21T22:18:17.999 INFO:tasks.rados.rados.0.trial178.stdout:465: left oid 50 (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:18:17.999 INFO:tasks.rados.rados.0.trial178.stdout:466: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:18.020 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 199224 2026-02-21T22:18:18.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: get_monmap_and_config 2026-02-21T22:18:18.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: build_initial_monmap 2026-02-21T22:18:18.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: monmap: 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:18:18.058835+0000 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:18:18.058835+0000 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:18:18.059 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 1 Processor -- start 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 1 -- start start 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: init 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-a con 0x7f34c808c1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-c con 0x7f34c808cc80 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-b con 0x7f34c8183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c808d780 con 0x7f34c808c1a0 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c8085da0 con 0x7f34c808cc80 2026-02-21T22:18:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c8085420 con 0x7f34c8183480 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34cc8cc640 10 monclient(hunting): authenticate will time out at 1307.294740s 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34c6575640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f34c808cc80 legacy=0x7f34c8181d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34c7577640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 0x7f34c81a3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42546/0 (socket says 10.20.193.102:42546) 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34c7577640 1 -- 10.20.193.102:0/517017069 learned_addr learned my addr 10.20.193.102:0/517017069 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.053+0000 7f34c6d76640 1 --1- 10.20.193.102:0/517017069 >> v1:10.20.193.102:6789/0 conn(0x7f34c808c1a0 0x7f34c808c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:57122/0 (socket says 10.20.193.102:57122) 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c7577640 1 -- 10.20.193.102:0/517017069 >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 legacy=0x7f34c81a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c6d76640 1 -- 10.20.193.102:0/517017069 >> v1:10.20.193.102:6789/0 conn(0x7f34c808c1a0 legacy=0x7f34c808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3071338192 0 0) 0x7f34bc002b90 con 0x7f34c808c1a0 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): my global_id is 34194 2026-02-21T22:18:18.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b4003620 con 0x7f34c808c1a0 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192709901 0 0) 0x7f34c8085420 con 0x7f34c8183480 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient(hunting): my global_id is 4532 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34bc002b90 con 0x7f34c8183480 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1128954172 0 0) 0x7f34b4003620 con 0x7f34c808c1a0 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 legacy=0x7f34c81a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 >> v1:10.20.193.178:6789/0 conn(0x7f34c808cc80 legacy=0x7f34c8181d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: _finish_hunting 0 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 monclient: found mon.noname-a 2026-02-21T22:18:18.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c8086640 con 0x7f34c808c1a0 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: _finish_auth 0 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: _check_auth_tickets 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34bc003210 con 0x7f34c808c1a0 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: dump: 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.072 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 5 monclient: authenticate success, global_id 34194 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f34bc0034e0 con 0x7f34c808c1a0 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:18.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 1 -- 10.20.193.102:0/517017069 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34bc004a90 con 0x7f34c808c1a0 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c5d74640 10 monclient: dump: 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.074 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: get_monmap_and_config success 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 4 set_mon_vals no callback set 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: shutdown 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- 10.20.193.102:0/517017069 >> v1:10.20.193.102:6789/0 conn(0x7f34c808c1a0 legacy=0x7f34c808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- 10.20.193.102:0/517017069 shutdown_connections 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- 10.20.193.102:0/517017069 >> 10.20.193.102:0/517017069 conn(0x7f34c808f6a0 msgr2=0x7f34c808faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- 10.20.193.102:0/517017069 shutdown_connections 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- 10.20.193.102:0/517017069 wait complete. 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: build_initial_monmap 2026-02-21T22:18:18.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: monmap: 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 Processor -- start 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- start start 2026-02-21T22:18:18.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: init 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: _reopen_session rank -1 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-b con 0x7f34c808c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-a con 0x7f34c808cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient(hunting): picked mon.noname-c con 0x7f34c8183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c80830a0 con 0x7f34c808cc80 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c8058980 con 0x7f34c808c1a0 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c805e070 con 0x7f34c8183480 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient(hunting): _renew_subs 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34cc8cc640 10 monclient(hunting): authenticate will time out at 1307.299328s 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c6d76640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f34c808c1a0 legacy=0x7f34c8152fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:18:18.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c7577640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 0x7f34c8175320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42554/0 (socket says 10.20.193.102:42554) 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.057+0000 7f34c7577640 1 -- 10.20.193.102:0/384845264 learned_addr learned my addr 10.20.193.102:0/384845264 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34c7577640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 legacy=0x7f34c8175320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34c6575640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6789/0 conn(0x7f34c808cc80 legacy=0x7f34c80848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452833998 0 0) 0x7f34c805e070 con 0x7f34c8183480 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): my global_id is 4535 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3494003620 con 0x7f34c8183480 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630980093 0 0) 0x7f34c80830a0 con 0x7f34c808cc80 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): my global_id is 34197 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c805e070 con 0x7f34c808cc80 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2976919213 0 0) 0x7f3494003620 con 0x7f34c8183480 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f34c80830a0 con 0x7f34c8183480 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34b80043a0 con 0x7f34c8183480 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): dump: 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.079 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 196743016 0 0) 0x7f34c80830a0 con 0x7f34c8183480 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.178:6789/0 conn(0x7f34c808c1a0 legacy=0x7f34c8152fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6789/0 conn(0x7f34c808cc80 legacy=0x7f34c80848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 monclient: found mon.c 2026-02-21T22:18:18.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c8082dc0 con 0x7f34c8183480 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: _finish_auth 0 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 5 monclient: authenticate success, global_id 4535 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 10 monclient: _renew_subs 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f34c8083380 con 0x7f34c8183480 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 10 monclient: _renew_subs 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f34c8084fc0 con 0x7f34c8183480 2026-02-21T22:18:18.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f34b80060a0 con 0x7f34c8183480 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34b8006440 con 0x7f34c8183480 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 10 monclient: dump: 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.082 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34c5573640 4 set_mon_vals no callback set 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f34b8006620 con 0x7f34c8183480 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f34c80008e0 con 0x7f34c805b890 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(118..118 src has 1..118) ==== 3915+0+0 (unknown 1160747031 0 0) 0x7f34b804da50 con 0x7f34c8183480 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.061+0000 7f34c6575640 10 monclient: get_auth_request con 0x7f349403d240 auth_method 0 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34c6d76640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805b890 legacy=0x7f34c804d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 746356133 0 0) 0x7f34c80008e0 con 0x7f34c805b890 2026-02-21T22:18:18.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient(hunting): my global_id is 4538 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c80830a0 con 0x7f34c805b890 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 937550217 0 0) 0x7f34c80830a0 con 0x7f34c805b890 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f34c80008e0 con 0x7f34c805b890 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34bc002c10 con 0x7f34c805b890 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient: dump: 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.084 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3702336039 0 0) 0x7f34c80008e0 con 0x7f34c805b890 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34c804d990 con 0x7f34c805b890 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f34bc003200 con 0x7f34c805b890 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:18:18.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805b890 legacy=0x7f34c804d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.065+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805b890 legacy=0x7f34c804d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:18.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:18:18.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f34c80857a0 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34c6d76640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805bc60 legacy=0x7f34c817f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721947655 0 0) 0x7f34c80857a0 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient(hunting): my global_id is 4541 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34c80008e0 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4003726869 0 0) 0x7f34c80008e0 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f34c80857a0 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f34bc004d10 con 0x7f34c805bc60 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:18:18.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient: dump: 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3680400396 0 0) 0x7f34c80857a0 con 0x7f34c805bc60 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f34c8054480 con 0x7f34c805bc60 2026-02-21T22:18:18.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 127496553) 0x7f34bc007090 con 0x7f34c805bc60 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805bc60 legacy=0x7f34c817f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34a77fe640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c805bc60 legacy=0x7f34c817f040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:18:18.090 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 18, 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 180699, 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:18.091 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:18.092 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:18:18.093 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:18:18.094 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.095 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:18:18.096 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.097 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.098 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.099 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.100 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:18:18.101 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.178:6800/2549949439 conn(0x7f349403d240 legacy=0x7f349405d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 10 monclient: shutdown 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 >> v1:10.20.193.102:6790/0 conn(0x7f34c8183480 legacy=0x7f34c8175320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34c7577640 1 -- 10.20.193.102:0/384845264 reap_dead start 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 shutdown_connections 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.069+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 >> 10.20.193.102:0/384845264 conn(0x7f34c808f6a0 msgr2=0x7f34c805d600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:18:18.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.073+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 shutdown_connections 2026-02-21T22:18:18.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:18:18.073+0000 7f34cc8cc640 1 -- 10.20.193.102:0/384845264 wait complete. 2026-02-21T22:18:18.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:18:18.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:18:18.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:18.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:465: done (2 left) 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:466: done (1 left) 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:467: done (0 left) 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:468: rollback oid 40 current snap is 52 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 45 2026-02-21T22:18:18.978 INFO:tasks.rados.rados.0.trial178.stdout:469: copy_from oid 23 from oid 14 current snap is 52 2026-02-21T22:18:18.979 INFO:tasks.rados.rados.0.trial178.stdout:470: delete oid 1 current snap is 52 2026-02-21T22:18:18.980 INFO:tasks.rados.rados.0.trial178.stdout:470: done (2 left) 2026-02-21T22:18:18.980 INFO:tasks.rados.rados.0.trial178.stdout:471: write (excl) oid 36 current snap is 52 2026-02-21T22:18:18.980 INFO:tasks.rados.rados.0.trial178.stdout:471: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2026-02-21T22:18:18.988 INFO:tasks.rados.rados.0.trial178.stdout:471: writing trial17818531-36 from 625260 to 1246199 tid 1 2026-02-21T22:18:18.995 INFO:tasks.rados.rados.0.trial178.stdout:471: writing trial17818531-36 from 1683347 to 2282733 tid 2 2026-02-21T22:18:19.000 INFO:tasks.rados.rados.0.trial178.stdout:471: writing trial17818531-36 from 2943668 to 3025260 tid 3 2026-02-21T22:18:19.000 INFO:tasks.rados.rados.0.trial178.stdout:472: snap_remove snap 49 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:472: done (3 left) 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:473: snap_create 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:468: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:468: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:468: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 76 (ObjNum 14 snap 0 seq_num 14) dirty exists 2026-02-21T22:18:19.982 INFO:tasks.rados.rados.0.trial178.stdout:469: finishing copy_from racing read to trial17818531-23 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:469: finishing copy_from to trial17818531-23 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 40 (ObjNum 72 snap 9 seq_num 72) dirty exists 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: finishing write tid 1 to trial17818531-36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: finishing write tid 2 to trial17818531-36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: finishing write tid 3 to trial17818531-36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: finishing write tid 4 to trial17818531-36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: finishing write tid 5 to trial17818531-36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: oid 36 updating version 0 to 38 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: oid 36 version 38 is already newer than 36 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: oid 36 updating version 38 to 39 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: oid 36 version 39 is already newer than 37 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 39 (ObjNum 180 snap 52 seq_num 180) dirty exists 2026-02-21T22:18:19.983 INFO:tasks.rados.rados.0.trial178.stdout:471: left oid 36 (ObjNum 180 snap 52 seq_num 180) 2026-02-21T22:18:20.982 INFO:tasks.rados.rados.0.trial178.stdout:468: done (3 left) 2026-02-21T22:18:20.983 INFO:tasks.rados.rados.0.trial178.stdout:469: done (2 left) 2026-02-21T22:18:20.983 INFO:tasks.rados.rados.0.trial178.stdout:471: done (1 left) 2026-02-21T22:18:20.983 INFO:tasks.rados.rados.0.trial178.stdout:473: done (0 left) 2026-02-21T22:18:20.983 INFO:tasks.rados.rados.0.trial178.stdout:474: snap_remove snap 50 2026-02-21T22:18:21.988 INFO:tasks.rados.rados.0.trial178.stdout:474: done (0 left) 2026-02-21T22:18:21.988 INFO:tasks.rados.rados.0.trial178.stdout:475: delete oid 3 current snap is 53 2026-02-21T22:18:21.992 INFO:tasks.rados.rados.0.trial178.stdout:475: done (0 left) 2026-02-21T22:18:21.992 INFO:tasks.rados.rados.0.trial178.stdout:476: snap_create 2026-02-21T22:18:22.388 INFO:tasks.rados.rados.0.trial178.stdout:476: done (0 left) 2026-02-21T22:18:22.389 INFO:tasks.rados.rados.0.trial178.stdout:477: delete oid 2 current snap is 54 2026-02-21T22:18:22.393 INFO:tasks.rados.rados.0.trial178.stdout:477: done (0 left) 2026-02-21T22:18:22.393 INFO:tasks.rados.rados.0.trial178.stdout:478: write oid 43 current snap is 54 2026-02-21T22:18:22.394 INFO:tasks.rados.rados.0.trial178.stdout:478: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2026-02-21T22:18:22.401 INFO:tasks.rados.rados.0.trial178.stdout:478: writing trial17818531-43 from 675681 to 1238720 tid 1 2026-02-21T22:18:22.411 INFO:tasks.rados.rados.0.trial178.stdout:478: writing trial17818531-43 from 1858793 to 2605506 tid 2 2026-02-21T22:18:22.414 INFO:tasks.rados.rados.0.trial178.stdout:478: writing trial17818531-43 from 3025937 to 3075681 tid 3 2026-02-21T22:18:22.414 INFO:tasks.rados.rados.0.trial178.stdout:479: read oid 50 snap -1 2026-02-21T22:18:22.414 INFO:tasks.rados.rados.0.trial178.stdout:479: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:18:22.414 INFO:tasks.rados.rados.0.trial178.stdout:480: snap_remove snap 41 2026-02-21T22:18:23.392 INFO:tasks.rados.rados.0.trial178.stdout:480: done (2 left) 2026-02-21T22:18:23.392 INFO:tasks.rados.rados.0.trial178.stdout:481: write (excl) oid 3 current snap is 54 2026-02-21T22:18:23.393 INFO:tasks.rados.rados.0.trial178.stdout:481: seq_num 184 ranges {692488=638288,1917275=606724} 2026-02-21T22:18:23.401 INFO:tasks.rados.rados.0.trial178.stdout:481: writing trial17818531-3 from 692488 to 1330776 tid 1 2026-02-21T22:18:23.410 INFO:tasks.rados.rados.0.trial178.stdout:481: writing trial17818531-3 from 1917275 to 2523999 tid 2 2026-02-21T22:18:23.410 INFO:tasks.rados.rados.0.trial178.stdout:482: rollback oid 22 current snap is 54 2026-02-21T22:18:23.410 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 47 2026-02-21T22:18:23.410 INFO:tasks.rados.rados.0.trial178.stdout:478: finishing write tid 1 to trial17818531-43 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: finishing write tid 2 to trial17818531-43 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: finishing write tid 3 to trial17818531-43 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: finishing write tid 4 to trial17818531-43 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: finishing write tid 5 to trial17818531-43 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: oid 43 updating version 0 to 28 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: oid 43 version 28 is already newer than 26 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: oid 43 version 28 is already newer than 27 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: oid 43 version 28 is already newer than 25 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 28 (ObjNum 183 snap 54 seq_num 183) dirty exists 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:478: left oid 43 (ObjNum 183 snap 54 seq_num 183) 2026-02-21T22:18:23.411 INFO:tasks.rados.rados.0.trial178.stdout:479: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:18:23.438 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2297075 2026-02-21T22:18:23.498 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:18:23] 2026-02-21T22:18:23.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:18:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:18:23, mode upmap, max misplaced 0.050000 2026-02-21T22:18:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:18:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:18:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', '.mgr', 'unique_pool_0'] 2026-02-21T22:18:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:18:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:18:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:18:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:18:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:18:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:18:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:18:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:18:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:18:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:18:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:18:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:18:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:18:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:18:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:18:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:18:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:18:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:18:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "124", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:18:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:18:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:18:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 124, 2026-02-21T22:18:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 56, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:18:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00036409872549551505 of space, bias 1.0, pg target 0.10922961764865452 quantized to 32 (current 16) 2026-02-21T22:18:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:18:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:23.556 INFO:tasks.rados.rados.0.trial178.stdout:481: finishing write tid 1 to trial17818531-3 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:481: finishing write tid 2 to trial17818531-3 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:478: done (3 left) 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:479: done (2 left) 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:483: rollback oid 20 current snap is 54 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 39 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:482: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:481: finishing write tid 3 to trial17818531-3 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:482: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:481: finishing write tid 4 to trial17818531-3 2026-02-21T22:18:23.557 INFO:tasks.rados.rados.0.trial178.stdout:481: oid 3 updating version 0 to 231 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:481: oid 3 version 231 is already newer than 230 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:481: oid 3 version 231 is already newer than 229 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 231 (ObjNum 184 snap 54 seq_num 184) dirty exists 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:481: left oid 3 (ObjNum 184 snap 54 seq_num 184) 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:482: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 92 (ObjNum 154 snap 41 seq_num 154) dirty exists 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:481: done (2 left) 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:482: done (1 left) 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:484: snap_create 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:483: finishing rollback tid 0 to trial17818531-20 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:483: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:18:23.558 INFO:tasks.rados.rados.0.trial178.stdout:483: finishing rollback tid 2 to trial17818531-20 2026-02-21T22:18:23.559 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 136 (ObjNum 132 snap 31 seq_num 132) dirty exists 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:18:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:18:24.395 INFO:tasks.rados.rados.0.trial178.stdout:483: done (1 left) 2026-02-21T22:18:24.395 INFO:tasks.rados.rados.0.trial178.stdout:484: done (0 left) 2026-02-21T22:18:24.395 INFO:tasks.rados.rados.0.trial178.stdout:485: write oid 18 current snap is 55 2026-02-21T22:18:24.395 INFO:tasks.rados.rados.0.trial178.stdout:485: seq_num 185 ranges {709295=713537,1859404=750382} 2026-02-21T22:18:24.400 INFO:tasks.rados.rados.0.trial178.stdout:485: writing trial17818531-18 from 709295 to 1422832 tid 1 2026-02-21T22:18:24.407 INFO:tasks.rados.rados.0.trial178.stdout:485: writing trial17818531-18 from 1859404 to 2609786 tid 2 2026-02-21T22:18:24.407 INFO:tasks.rados.rados.0.trial178.stdout:486: rollback oid 11 current snap is 55 2026-02-21T22:18:24.408 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 45 2026-02-21T22:18:24.408 INFO:tasks.rados.rados.0.trial178.stdout:485: finishing write tid 1 to trial17818531-18 2026-02-21T22:18:24.408 INFO:tasks.rados.rados.0.trial178.stdout:487: snap_remove snap 44 2026-02-21T22:18:25.396 INFO:tasks.rados.rados.0.trial178.stdout:487: done (2 left) 2026-02-21T22:18:25.396 INFO:tasks.rados.rados.0.trial178.stdout:488: snap_create 2026-02-21T22:18:25.396 INFO:tasks.rados.rados.0.trial178.stdout:486: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:18:25.396 INFO:tasks.rados.rados.0.trial178.stdout:486: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:486: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 235 (ObjNum 114 snap 27 seq_num 114) dirty exists 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: finishing write tid 2 to trial17818531-18 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: finishing write tid 3 to trial17818531-18 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: finishing write tid 4 to trial17818531-18 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: oid 18 updating version 0 to 132 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: oid 18 version 132 is already newer than 131 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: oid 18 version 132 is already newer than 130 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 132 (ObjNum 185 snap 55 seq_num 185) dirty exists 2026-02-21T22:18:25.397 INFO:tasks.rados.rados.0.trial178.stdout:485: left oid 18 (ObjNum 185 snap 55 seq_num 185) 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:485: done (2 left) 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:486: done (1 left) 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:488: done (0 left) 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:489: read oid 44 snap -1 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:489: expect (ObjNum 119 snap 29 seq_num 119) 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:490: rollback oid 47 current snap is 56 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 45 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:491: rollback oid 6 current snap is 56 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 39 2026-02-21T22:18:26.399 INFO:tasks.rados.rados.0.trial178.stdout:492: delete oid 40 current snap is 56 2026-02-21T22:18:26.402 INFO:tasks.rados.rados.0.trial178.stdout:491: finishing rollback tid 0 to trial17818531-6 2026-02-21T22:18:26.402 INFO:tasks.rados.rados.0.trial178.stdout:491: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:18:26.402 INFO:tasks.rados.rados.0.trial178.stdout:491: finishing rollback tid 2 to trial17818531-6 2026-02-21T22:18:26.402 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 126 (ObjNum 138 snap 34 seq_num 138) dirty exists 2026-02-21T22:18:26.403 INFO:tasks.rados.rados.0.trial178.stdout:491: done (3 left) 2026-02-21T22:18:26.403 INFO:tasks.rados.rados.0.trial178.stdout:492: done (2 left) 2026-02-21T22:18:26.403 INFO:tasks.rados.rados.0.trial178.stdout:493: snap_remove snap 39 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:493: done (2 left) 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:494: read oid 46 snap -1 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:494: expect deleted 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:495: delete oid 28 current snap is 56 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:489: expect (ObjNum 119 snap 29 seq_num 119) 2026-02-21T22:18:27.390 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:27.429 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1044024 2026-02-21T22:18:27.459 INFO:tasks.rados.rados.0.trial178.stdout:490: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:18:27.459 INFO:tasks.rados.rados.0.trial178.stdout:490: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:18:27.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 260 (ObjNum 123 snap 29 seq_num 123) dirty exists 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:489: done (3 left) 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:490: done (2 left) 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:494: done (1 left) 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:495: done (0 left) 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:496: write (excl) oid 4 current snap is 56 2026-02-21T22:18:27.460 INFO:tasks.rados.rados.0.trial178.stdout:496: seq_num 188 ranges {759716=539284,2034850=614062} 2026-02-21T22:18:27.468 INFO:tasks.rados.rados.0.trial178.stdout:496: writing trial17818531-4 from 759716 to 1299000 tid 1 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:496: writing trial17818531-4 from 2034850 to 2648912 tid 2 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:497: rollback oid 25 current snap is 56 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 25 to 51 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:496: finishing write tid 1 to trial17818531-4 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:498: read oid 36 snap 52 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:498: expect (ObjNum 180 snap 52 seq_num 180) 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:499: read oid 32 snap -1 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:499: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:500: copy_from oid 49 from oid 35 current snap is 56 2026-02-21T22:18:27.478 INFO:tasks.rados.rados.0.trial178.stdout:501: read oid 3 snap -1 2026-02-21T22:18:27.479 INFO:tasks.rados.rados.0.trial178.stdout:501: expect (ObjNum 184 snap 54 seq_num 184) 2026-02-21T22:18:27.479 INFO:tasks.rados.rados.0.trial178.stdout:502: snap_remove snap 45 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:502: done (6 left) 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:503: rollback oid 5 current snap is 56 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 52 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: finishing write tid 2 to trial17818531-4 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: finishing write tid 3 to trial17818531-4 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: finishing write tid 4 to trial17818531-4 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: oid 4 updating version 0 to 73 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: oid 4 updating version 73 to 74 2026-02-21T22:18:28.407 INFO:tasks.rados.rados.0.trial178.stdout:496: oid 4 version 74 is already newer than 72 2026-02-21T22:18:28.408 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 74 (ObjNum 188 snap 56 seq_num 188) dirty exists 2026-02-21T22:18:28.408 INFO:tasks.rados.rados.0.trial178.stdout:496: left oid 4 (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:18:28.408 INFO:tasks.rados.rados.0.trial178.stdout:499: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:28.440 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2046065 2026-02-21T22:18:28.453 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1121536 2026-02-21T22:18:28.501 INFO:tasks.rados.rados.0.trial178.stdout:498: expect (ObjNum 180 snap 52 seq_num 180) 2026-02-21T22:18:28.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:28.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:28.552 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 343346 2026-02-21T22:18:28.653 INFO:tasks.rados.rados.0.trial178.stdout:497: finishing rollback tid 0 to trial17818531-25 2026-02-21T22:18:28.653 INFO:tasks.rados.rados.0.trial178.stdout:497: finishing rollback tid 1 to trial17818531-25 2026-02-21T22:18:28.653 INFO:tasks.rados.rados.0.trial178.stdout:497: finishing rollback tid 2 to trial17818531-25 2026-02-21T22:18:28.654 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 78 (ObjNum 160 snap 47 seq_num 160) dirty exists 2026-02-21T22:18:28.654 INFO:tasks.rados.rados.0.trial178.stdout:501: expect (ObjNum 184 snap 54 seq_num 184) 2026-02-21T22:18:28.700 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:500: finishing copy_from racing read to trial17818531-49 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:500: finishing copy_from to trial17818531-49 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 216 (ObjNum 174 snap 49 seq_num 174) dirty exists 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:503: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:503: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:503: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:18:28.725 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 152 (ObjNum 170 snap 48 seq_num 170) dirty exists 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:496: done (6 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:497: done (5 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:498: done (4 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:499: done (3 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:500: done (2 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:501: done (1 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:503: done (0 left) 2026-02-21T22:18:28.726 INFO:tasks.rados.rados.0.trial178.stdout:504: snap_create 2026-02-21T22:18:29.408 INFO:tasks.rados.rados.0.trial178.stdout:504: done (0 left) 2026-02-21T22:18:29.408 INFO:tasks.rados.rados.0.trial178.stdout:505: copy_from oid 2 from oid 45 current snap is 57 2026-02-21T22:18:29.408 INFO:tasks.rados.rados.0.trial178.stdout:506: copy_from oid 36 from oid 21 current snap is 57 2026-02-21T22:18:29.408 INFO:tasks.rados.rados.0.trial178.stdout:507: rollback oid 47 current snap is 57 2026-02-21T22:18:29.409 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 52 2026-02-21T22:18:29.409 INFO:tasks.rados.rados.0.trial178.stdout:508: write (excl) oid 7 current snap is 57 2026-02-21T22:18:29.409 INFO:tasks.rados.rados.0.trial178.stdout:508: seq_num 189 ranges {776523=614533,1976979=757720} 2026-02-21T22:18:29.417 INFO:tasks.rados.rados.0.trial178.stdout:508: writing trial17818531-7 from 776523 to 1391056 tid 1 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:508: writing trial17818531-7 from 1976979 to 2734699 tid 2 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:509: rollback oid 16 current snap is 57 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 48 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:506: finishing copy_from to trial17818531-36 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:506: got expected ENOENT (src dne) 2026-02-21T22:18:29.427 INFO:tasks.rados.rados.0.trial178.stdout:507: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:506: finishing copy_from racing read to trial17818531-36 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:507: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 279 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:508: finishing write tid 1 to trial17818531-7 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:505: finishing copy_from racing read to trial17818531-2 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:505: finishing copy_from to trial17818531-2 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 82 (ObjNum 165 snap 47 seq_num 165) dirty exists 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:505: done (4 left) 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:506: done (3 left) 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:507: done (2 left) 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:510: copy_from oid 36 from oid 31 current snap is 57 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:511: read oid 4 snap -1 2026-02-21T22:18:29.428 INFO:tasks.rados.rados.0.trial178.stdout:511: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:18:29.429 INFO:tasks.rados.rados.0.trial178.stdout:512: delete oid 35 current snap is 57 2026-02-21T22:18:29.429 INFO:tasks.rados.rados.0.trial178.stdout:510: finishing copy_from to trial17818531-36 2026-02-21T22:18:29.429 INFO:tasks.rados.rados.0.trial178.stdout:510: got expected ENOENT (src dne) 2026-02-21T22:18:29.429 INFO:tasks.rados.rados.0.trial178.stdout:510: finishing copy_from racing read to trial17818531-36 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:510: done (4 left) 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:512: done (3 left) 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:513: snap_create 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:509: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:509: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:508: finishing write tid 2 to trial17818531-7 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:508: finishing write tid 3 to trial17818531-7 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:508: finishing write tid 4 to trial17818531-7 2026-02-21T22:18:29.430 INFO:tasks.rados.rados.0.trial178.stdout:508: oid 7 updating version 0 to 280 2026-02-21T22:18:29.431 INFO:tasks.rados.rados.0.trial178.stdout:508: oid 7 updating version 280 to 282 2026-02-21T22:18:29.431 INFO:tasks.rados.rados.0.trial178.stdout:508: oid 7 version 282 is already newer than 281 2026-02-21T22:18:29.431 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 282 (ObjNum 189 snap 57 seq_num 189) dirty exists 2026-02-21T22:18:29.431 INFO:tasks.rados.rados.0.trial178.stdout:508: left oid 7 (ObjNum 189 snap 57 seq_num 189) 2026-02-21T22:18:29.432 INFO:tasks.rados.rados.0.trial178.stdout:509: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:18:29.432 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 80 (ObjNum 15 snap 0 seq_num 15) dirty exists 2026-02-21T22:18:29.433 INFO:tasks.rados.rados.0.trial178.stdout:511: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:18:29.433 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 595624 2026-02-21T22:18:29.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 878672 2026-02-21T22:18:29.520 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 820758 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:508: done (3 left) 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:509: done (2 left) 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:511: done (1 left) 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:513: done (0 left) 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:514: read oid 39 snap -1 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:514: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:515: read oid 16 snap 53 2026-02-21T22:18:30.415 INFO:tasks.rados.rados.0.trial178.stdout:515: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:516: copy_from oid 20 from oid 12 current snap is 58 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:517: copy_from oid 21 from oid 1 current snap is 58 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:518: copy_from oid 45 from oid 2 current snap is 58 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:519: read oid 42 snap -1 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:519: expect (ObjNum 159 snap 47 seq_num 159) 2026-02-21T22:18:30.416 INFO:tasks.rados.rados.0.trial178.stdout:520: snap_remove snap 52 2026-02-21T22:18:31.424 INFO:tasks.rados.rados.0.trial178.stdout:520: done (6 left) 2026-02-21T22:18:31.424 INFO:tasks.rados.rados.0.trial178.stdout:521: write (excl) oid 43 current snap is 58 2026-02-21T22:18:31.424 INFO:tasks.rados.rados.0.trial178.stdout:521: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2026-02-21T22:18:31.428 INFO:tasks.rados.rados.0.trial178.stdout:521: writing trial17818531-43 from 410137 to 891521 tid 1 2026-02-21T22:18:31.436 INFO:tasks.rados.rados.0.trial178.stdout:521: writing trial17818531-43 from 1410296 to 2171685 tid 2 2026-02-21T22:18:31.443 INFO:tasks.rados.rados.0.trial178.stdout:521: writing trial17818531-43 from 2728968 to 3210137 tid 3 2026-02-21T22:18:31.443 INFO:tasks.rados.rados.0.trial178.stdout:522: snap_remove snap 56 2026-02-21T22:18:32.399 INFO:tasks.rados.rados.0.trial178.stdout:522: done (7 left) 2026-02-21T22:18:32.399 INFO:tasks.rados.rados.0.trial178.stdout:523: rollback oid 34 current snap is 58 2026-02-21T22:18:32.399 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 34 to 54 2026-02-21T22:18:32.400 INFO:tasks.rados.rados.0.trial178.stdout:515: expect (ObjNum 15 snap 0 seq_num 15) 2026-02-21T22:18:32.430 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:32.470 INFO:tasks.rados.rados.0.trial178.stdout:519: expect (ObjNum 159 snap 47 seq_num 159) 2026-02-21T22:18:32.473 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 774289 2026-02-21T22:18:32.495 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:32.553 INFO:tasks.rados.rados.0.trial178.stdout:516: finishing copy_from to trial17818531-20 2026-02-21T22:18:32.554 INFO:tasks.rados.rados.0.trial178.stdout:516: got expected ENOENT (src dne) 2026-02-21T22:18:32.554 INFO:tasks.rados.rados.0.trial178.stdout:516: finishing copy_from racing read to trial17818531-20 2026-02-21T22:18:32.554 INFO:tasks.rados.rados.0.trial178.stdout:517: finishing copy_from to trial17818531-21 2026-02-21T22:18:32.554 INFO:tasks.rados.rados.0.trial178.stdout:517: got expected ENOENT (src dne) 2026-02-21T22:18:32.554 INFO:tasks.rados.rados.0.trial178.stdout:517: finishing copy_from racing read to trial17818531-21 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:515: done (7 left) 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:516: done (6 left) 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:517: done (5 left) 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:519: done (4 left) 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:524: snap_create 2026-02-21T22:18:32.555 INFO:tasks.rados.rados.0.trial178.stdout:514: expect (ObjNum 68 snap 7 seq_num 68) 2026-02-21T22:18:32.594 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1018731 2026-02-21T22:18:32.635 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2977092 2026-02-21T22:18:32.641 INFO:tasks.rados.rados.0.trial178.stdout:518: finishing copy_from racing read to trial17818531-45 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:518: finishing copy_from to trial17818531-45 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 82 (ObjNum 165 snap 47 seq_num 165) dirty exists 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: finishing write tid 1 to trial17818531-43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: finishing write tid 2 to trial17818531-43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: finishing write tid 3 to trial17818531-43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: finishing write tid 4 to trial17818531-43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: finishing write tid 5 to trial17818531-43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: oid 43 updating version 0 to 40 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: oid 43 updating version 40 to 42 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: oid 43 updating version 42 to 43 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: oid 43 version 43 is already newer than 41 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 43 (ObjNum 191 snap 58 seq_num 191) dirty exists 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:521: left oid 43 (ObjNum 191 snap 58 seq_num 191) 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:523: finishing rollback tid 0 to trial17818531-34 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:523: finishing rollback tid 1 to trial17818531-34 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:523: finishing rollback tid 2 to trial17818531-34 2026-02-21T22:18:32.642 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 65 (ObjNum 156 snap 43 seq_num 156) dirty exists 2026-02-21T22:18:33.420 INFO:tasks.rados.rados.0.trial178.stdout:514: done (4 left) 2026-02-21T22:18:33.420 INFO:tasks.rados.rados.0.trial178.stdout:518: done (3 left) 2026-02-21T22:18:33.420 INFO:tasks.rados.rados.0.trial178.stdout:521: done (2 left) 2026-02-21T22:18:33.420 INFO:tasks.rados.rados.0.trial178.stdout:523: done (1 left) 2026-02-21T22:18:33.421 INFO:tasks.rados.rados.0.trial178.stdout:524: done (0 left) 2026-02-21T22:18:33.421 INFO:tasks.rados.rados.0.trial178.stdout:525: snap_create 2026-02-21T22:18:33.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:33.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:34.423 INFO:tasks.rados.rados.0.trial178.stdout:525: done (0 left) 2026-02-21T22:18:34.423 INFO:tasks.rados.rados.0.trial178.stdout:526: snap_create 2026-02-21T22:18:35.424 INFO:tasks.rados.rados.0.trial178.stdout:526: done (0 left) 2026-02-21T22:18:35.424 INFO:tasks.rados.rados.0.trial178.stdout:527: delete oid 8 current snap is 61 2026-02-21T22:18:35.427 INFO:tasks.rados.rados.0.trial178.stdout:527: done (0 left) 2026-02-21T22:18:35.427 INFO:tasks.rados.rados.0.trial178.stdout:528: snap_remove snap 48 2026-02-21T22:18:36.432 INFO:tasks.rados.rados.0.trial178.stdout:528: done (0 left) 2026-02-21T22:18:36.432 INFO:tasks.rados.rados.0.trial178.stdout:529: write oid 11 current snap is 61 2026-02-21T22:18:36.432 INFO:tasks.rados.rados.0.trial178.stdout:529: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2026-02-21T22:18:36.438 INFO:tasks.rados.rados.0.trial178.stdout:529: writing trial17818531-11 from 443751 to 1075633 tid 1 2026-02-21T22:18:36.445 INFO:tasks.rados.rados.0.trial178.stdout:529: writing trial17818531-11 from 1810907 to 2575965 tid 2 2026-02-21T22:18:36.447 INFO:tasks.rados.rados.0.trial178.stdout:529: writing trial17818531-11 from 3067461 to 3243751 tid 3 2026-02-21T22:18:36.447 INFO:tasks.rados.rados.0.trial178.stdout:530: snap_remove snap 58 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:530: done (1 left) 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:531: copy_from oid 49 from oid 37 current snap is 61 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: finishing write tid 1 to trial17818531-11 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: finishing write tid 2 to trial17818531-11 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: finishing write tid 3 to trial17818531-11 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: finishing write tid 4 to trial17818531-11 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: finishing write tid 5 to trial17818531-11 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: oid 11 updating version 0 to 287 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: oid 11 version 287 is already newer than 285 2026-02-21T22:18:37.392 INFO:tasks.rados.rados.0.trial178.stdout:529: oid 11 version 287 is already newer than 286 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:529: oid 11 updating version 287 to 288 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 288 (ObjNum 193 snap 61 seq_num 193) dirty exists 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:529: left oid 11 (ObjNum 193 snap 61 seq_num 193) 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:529: done (1 left) 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:532: read oid 46 snap -1 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:532: expect deleted 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:533: rollback oid 44 current snap is 61 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 47 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:534: delete oid 40 current snap is 61 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:534: done (3 left) 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:535: copy_from oid 12 from oid 26 current snap is 61 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:536: copy_from oid 23 from oid 17 current snap is 61 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:537: read oid 6 snap -1 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:537: expect (ObjNum 138 snap 34 seq_num 138) 2026-02-21T22:18:37.393 INFO:tasks.rados.rados.0.trial178.stdout:538: snap_remove snap 60 2026-02-21T22:18:38.396 INFO:tasks.rados.rados.0.trial178.stdout:538: done (6 left) 2026-02-21T22:18:38.396 INFO:tasks.rados.rados.0.trial178.stdout:539: write (excl) oid 3 current snap is 61 2026-02-21T22:18:38.396 INFO:tasks.rados.rados.0.trial178.stdout:539: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2026-02-21T22:18:38.405 INFO:tasks.rados.rados.0.trial178.stdout:539: writing trial17818531-3 from 477365 to 1259745 tid 1 2026-02-21T22:18:38.415 INFO:tasks.rados.rados.0.trial178.stdout:539: writing trial17818531-3 from 1927871 to 2696598 tid 2 2026-02-21T22:18:38.419 INFO:tasks.rados.rados.0.trial178.stdout:539: writing trial17818531-3 from 3122307 to 3277365 tid 3 2026-02-21T22:18:38.419 INFO:tasks.rados.rados.0.trial178.stdout:540: write oid 2 current snap is 61 2026-02-21T22:18:38.419 INFO:tasks.rados.rados.0.trial178.stdout:540: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2026-02-21T22:18:38.425 INFO:tasks.rados.rados.0.trial178.stdout:540: writing trial17818531-2 from 494172 to 951801 tid 1 2026-02-21T22:18:38.433 INFO:tasks.rados.rados.0.trial178.stdout:540: writing trial17818531-2 from 1586353 to 2098738 tid 2 2026-02-21T22:18:38.441 INFO:tasks.rados.rados.0.trial178.stdout:540: writing trial17818531-2 from 2749730 to 3294172 tid 3 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:541: copy_from oid 16 from oid 34 current snap is 61 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:533: finishing rollback tid 0 to trial17818531-44 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:532: done (8 left) 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:542: read oid 38 snap -1 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:542: expect deleted 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:543: rollback oid 8 current snap is 61 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 59 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:535: finishing copy_from to trial17818531-12 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:535: got expected ENOENT (src dne) 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:535: finishing copy_from racing read to trial17818531-12 2026-02-21T22:18:38.442 INFO:tasks.rados.rados.0.trial178.stdout:533: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:18:38.443 INFO:tasks.rados.rados.0.trial178.stdout:533: finishing rollback tid 2 to trial17818531-44 2026-02-21T22:18:38.443 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 94 (ObjNum 119 snap 29 seq_num 119) dirty exists 2026-02-21T22:18:38.443 INFO:tasks.rados.rados.0.trial178.stdout:537: expect (ObjNum 138 snap 34 seq_num 138) 2026-02-21T22:18:38.445 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 690801 2026-02-21T22:18:38.486 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1465111 2026-02-21T22:18:38.507 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1884693 2026-02-21T22:18:38.510 INFO:tasks.rados.rados.0.trial178.stdout:531: finishing copy_from racing read to trial17818531-49 2026-02-21T22:18:38.510 INFO:tasks.rados.rados.0.trial178.stdout:531: finishing copy_from to trial17818531-49 2026-02-21T22:18:38.510 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 52 (ObjNum 131 snap 31 seq_num 131) dirty exists 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:536: finishing copy_from racing read to trial17818531-23 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:536: finishing copy_from to trial17818531-23 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 77 (ObjNum 132 snap 31 seq_num 132) dirty exists 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:531: done (9 left) 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:533: done (8 left) 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:535: done (7 left) 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:536: done (6 left) 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:537: done (5 left) 2026-02-21T22:18:38.511 INFO:tasks.rados.rados.0.trial178.stdout:544: snap_remove snap 47 2026-02-21T22:18:38.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:38.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:544: done (5 left) 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:545: snap_create 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: finishing write tid 1 to trial17818531-3 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: finishing write tid 2 to trial17818531-3 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: finishing write tid 3 to trial17818531-3 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: finishing write tid 4 to trial17818531-3 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: finishing write tid 5 to trial17818531-3 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: oid 3 updating version 0 to 294 2026-02-21T22:18:39.436 INFO:tasks.rados.rados.0.trial178.stdout:539: oid 3 updating version 294 to 295 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:539: oid 3 updating version 295 to 296 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:539: oid 3 updating version 296 to 297 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 297 (ObjNum 195 snap 61 seq_num 195) dirty exists 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:539: left oid 3 (ObjNum 195 snap 61 seq_num 195) 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: finishing write tid 1 to trial17818531-2 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: finishing write tid 2 to trial17818531-2 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:543: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: finishing write tid 3 to trial17818531-2 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: finishing write tid 4 to trial17818531-2 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: finishing write tid 5 to trial17818531-2 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: oid 2 updating version 0 to 144 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: oid 2 version 144 is already newer than 143 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: oid 2 updating version 144 to 145 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: oid 2 updating version 145 to 146 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 146 (ObjNum 196 snap 61 seq_num 196) dirty exists 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:540: left oid 2 (ObjNum 196 snap 61 seq_num 196) 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:543: finishing rollback tid 2 to trial17818531-8 2026-02-21T22:18:39.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 296 (ObjNum 167 snap 48 seq_num 167) dirty exists 2026-02-21T22:18:39.438 INFO:tasks.rados.rados.0.trial178.stdout:541: finishing copy_from racing read to trial17818531-16 2026-02-21T22:18:39.438 INFO:tasks.rados.rados.0.trial178.stdout:541: finishing copy_from to trial17818531-16 2026-02-21T22:18:39.438 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 65 (ObjNum 156 snap 43 seq_num 156) dirty exists 2026-02-21T22:18:40.441 INFO:tasks.rados.rados.0.trial178.stdout:539: done (5 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:540: done (4 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:541: done (3 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:542: done (2 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:543: done (1 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:545: done (0 left) 2026-02-21T22:18:40.442 INFO:tasks.rados.rados.0.trial178.stdout:546: delete oid 30 current snap is 62 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:546: done (0 left) 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:547: copy_from oid 10 from oid 46 current snap is 62 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:548: copy_from oid 39 from oid 3 current snap is 62 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:549: read oid 37 snap 55 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:549: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:550: write oid 19 current snap is 62 2026-02-21T22:18:40.445 INFO:tasks.rados.rados.0.trial178.stdout:550: seq_num 198 ranges {527786=724480,1703317=516054,2804576=435035} 2026-02-21T22:18:40.453 INFO:tasks.rados.rados.0.trial178.stdout:550: writing trial17818531-19 from 527786 to 1252266 tid 1 2026-02-21T22:18:40.460 INFO:tasks.rados.rados.0.trial178.stdout:550: writing trial17818531-19 from 1703317 to 2219371 tid 2 2026-02-21T22:18:40.467 INFO:tasks.rados.rados.0.trial178.stdout:550: writing trial17818531-19 from 2804576 to 3239611 tid 3 2026-02-21T22:18:40.468 INFO:tasks.rados.rados.0.trial178.stdout:551: snap_remove snap 59 2026-02-21T22:18:41.446 INFO:tasks.rados.rados.0.trial178.stdout:551: done (4 left) 2026-02-21T22:18:41.446 INFO:tasks.rados.rados.0.trial178.stdout:552: snap_create 2026-02-21T22:18:41.446 INFO:tasks.rados.rados.0.trial178.stdout:547: finishing copy_from to trial17818531-10 2026-02-21T22:18:41.446 INFO:tasks.rados.rados.0.trial178.stdout:547: got expected ENOENT (src dne) 2026-02-21T22:18:41.447 INFO:tasks.rados.rados.0.trial178.stdout:547: finishing copy_from racing read to trial17818531-10 2026-02-21T22:18:41.447 INFO:tasks.rados.rados.0.trial178.stdout:549: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:41.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 508544 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:550: finishing write tid 1 to trial17818531-19 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:548: finishing copy_from racing read to trial17818531-39 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:550: finishing write tid 2 to trial17818531-19 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:548: finishing copy_from to trial17818531-39 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 297 (ObjNum 195 snap 61 seq_num 195) dirty exists 2026-02-21T22:18:41.506 INFO:tasks.rados.rados.0.trial178.stdout:550: finishing write tid 3 to trial17818531-19 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: finishing write tid 4 to trial17818531-19 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: finishing write tid 5 to trial17818531-19 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: oid 19 updating version 0 to 309 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: oid 19 version 309 is already newer than 307 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: oid 19 version 309 is already newer than 308 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: oid 19 version 309 is already newer than 306 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 309 (ObjNum 198 snap 62 seq_num 198) dirty exists 2026-02-21T22:18:41.507 INFO:tasks.rados.rados.0.trial178.stdout:550: left oid 19 (ObjNum 198 snap 62 seq_num 198) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:547: done (4 left) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:548: done (3 left) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:549: done (2 left) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:550: done (1 left) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:552: done (0 left) 2026-02-21T22:18:42.393 INFO:tasks.rados.rados.0.trial178.stdout:553: read oid 34 snap -1 2026-02-21T22:18:42.394 INFO:tasks.rados.rados.0.trial178.stdout:553: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:18:42.394 INFO:tasks.rados.rados.0.trial178.stdout:554: write oid 10 current snap is 63 2026-02-21T22:18:42.394 INFO:tasks.rados.rados.0.trial178.stdout:554: seq_num 199 ranges {544593=799729,1761799=776065,2948352=396241} 2026-02-21T22:18:42.402 INFO:tasks.rados.rados.0.trial178.stdout:554: writing trial17818531-10 from 544593 to 1344322 tid 1 2026-02-21T22:18:42.410 INFO:tasks.rados.rados.0.trial178.stdout:554: writing trial17818531-10 from 1761799 to 2537864 tid 2 2026-02-21T22:18:42.416 INFO:tasks.rados.rados.0.trial178.stdout:554: writing trial17818531-10 from 2948352 to 3344593 tid 3 2026-02-21T22:18:42.416 INFO:tasks.rados.rados.0.trial178.stdout:555: read oid 22 snap -1 2026-02-21T22:18:42.416 INFO:tasks.rados.rados.0.trial178.stdout:555: expect (ObjNum 154 snap 41 seq_num 154) 2026-02-21T22:18:42.416 INFO:tasks.rados.rados.0.trial178.stdout:556: delete oid 28 current snap is 63 2026-02-21T22:18:42.416 INFO:tasks.rados.rados.0.trial178.stdout:553: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:18:42.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:42.484 INFO:tasks.rados.rados.0.trial178.stdout:554: finishing write tid 1 to trial17818531-10 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:554: finishing write tid 2 to trial17818531-10 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:554: finishing write tid 3 to trial17818531-10 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:553: done (3 left) 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:556: done (2 left) 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:557: write (excl) oid 25 current snap is 63 2026-02-21T22:18:42.485 INFO:tasks.rados.rados.0.trial178.stdout:557: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2026-02-21T22:18:42.493 INFO:tasks.rados.rados.0.trial178.stdout:557: writing trial17818531-25 from 578207 to 1128434 tid 1 2026-02-21T22:18:42.502 INFO:tasks.rados.rados.0.trial178.stdout:557: writing trial17818531-25 from 1762410 to 2542144 tid 2 2026-02-21T22:18:42.508 INFO:tasks.rados.rados.0.trial178.stdout:557: writing trial17818531-25 from 3286845 to 3378207 tid 3 2026-02-21T22:18:42.508 INFO:tasks.rados.rados.0.trial178.stdout:558: copy_from oid 1 from oid 28 current snap is 63 2026-02-21T22:18:42.508 INFO:tasks.rados.rados.0.trial178.stdout:554: finishing write tid 4 to trial17818531-10 2026-02-21T22:18:42.508 INFO:tasks.rados.rados.0.trial178.stdout:554: finishing write tid 5 to trial17818531-10 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:554: oid 10 updating version 0 to 155 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:554: oid 10 version 155 is already newer than 154 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:554: oid 10 updating version 155 to 156 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:554: oid 10 updating version 156 to 157 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 157 (ObjNum 199 snap 63 seq_num 199) dirty exists 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:554: left oid 10 (ObjNum 199 snap 63 seq_num 199) 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:555: expect (ObjNum 154 snap 41 seq_num 154) 2026-02-21T22:18:42.509 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:42.542 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:42.560 INFO:tasks.rados.rados.0.trial178.stdout:557: finishing write tid 1 to trial17818531-25 2026-02-21T22:18:42.560 INFO:tasks.rados.rados.0.trial178.stdout:557: finishing write tid 2 to trial17818531-25 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: finishing write tid 3 to trial17818531-25 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: finishing write tid 4 to trial17818531-25 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: finishing write tid 5 to trial17818531-25 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: oid 25 updating version 0 to 219 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: oid 25 version 219 is already newer than 218 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: oid 25 version 219 is already newer than 217 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: oid 25 updating version 219 to 220 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 220 (ObjNum 201 snap 63 seq_num 201) dirty exists 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:557: left oid 25 (ObjNum 201 snap 63 seq_num 201) 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:558: finishing copy_from to trial17818531-1 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:558: got expected ENOENT (src dne) 2026-02-21T22:18:42.561 INFO:tasks.rados.rados.0.trial178.stdout:558: finishing copy_from racing read to trial17818531-1 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:554: done (3 left) 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:555: done (2 left) 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:557: done (1 left) 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:558: done (0 left) 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:559: write (excl) oid 20 current snap is 63 2026-02-21T22:18:42.562 INFO:tasks.rados.rados.0.trial178.stdout:559: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2026-02-21T22:18:42.569 INFO:tasks.rados.rados.0.trial178.stdout:559: writing trial17818531-20 from 595014 to 1220490 tid 1 2026-02-21T22:18:42.576 INFO:tasks.rados.rados.0.trial178.stdout:559: writing trial17818531-20 from 1820892 to 2344284 tid 2 2026-02-21T22:18:42.584 INFO:tasks.rados.rados.0.trial178.stdout:559: writing trial17818531-20 from 2797915 to 3394391 tid 3 2026-02-21T22:18:42.584 INFO:tasks.rados.rados.0.trial178.stdout:560: write (excl) oid 23 current snap is 63 2026-02-21T22:18:42.584 INFO:tasks.rados.rados.0.trial178.stdout:560: seq_num 203 ranges {611821=700725,1879374=783403,3341691=70130} 2026-02-21T22:18:42.593 INFO:tasks.rados.rados.0.trial178.stdout:560: writing trial17818531-23 from 611821 to 1312546 tid 1 2026-02-21T22:18:42.601 INFO:tasks.rados.rados.0.trial178.stdout:560: writing trial17818531-23 from 1879374 to 2662777 tid 2 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:560: writing trial17818531-23 from 3341691 to 3411821 tid 3 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:561: copy_from oid 8 from oid 41 current snap is 63 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:559: finishing write tid 1 to trial17818531-20 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:562: read oid 30 snap -1 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:562: expect deleted 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:563: read oid 11 snap -1 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:563: expect (ObjNum 193 snap 61 seq_num 193) 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:564: snap_create 2026-02-21T22:18:42.605 INFO:tasks.rados.rados.0.trial178.stdout:559: finishing write tid 2 to trial17818531-20 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: finishing write tid 3 to trial17818531-20 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: finishing write tid 4 to trial17818531-20 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: finishing write tid 5 to trial17818531-20 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: oid 20 updating version 0 to 173 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: oid 20 updating version 173 to 174 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: oid 20 updating version 174 to 175 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: oid 20 version 175 is already newer than 172 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 175 (ObjNum 202 snap 63 seq_num 202) dirty exists 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:559: left oid 20 (ObjNum 202 snap 63 seq_num 202) 2026-02-21T22:18:42.606 INFO:tasks.rados.rados.0.trial178.stdout:560: finishing write tid 1 to trial17818531-23 2026-02-21T22:18:42.607 INFO:tasks.rados.rados.0.trial178.stdout:560: finishing write tid 2 to trial17818531-23 2026-02-21T22:18:42.607 INFO:tasks.rados.rados.0.trial178.stdout:560: finishing write tid 3 to trial17818531-23 2026-02-21T22:18:42.607 INFO:tasks.rados.rados.0.trial178.stdout:560: finishing write tid 4 to trial17818531-23 2026-02-21T22:18:42.607 INFO:tasks.rados.rados.0.trial178.stdout:560: finishing write tid 5 to trial17818531-23 2026-02-21T22:18:42.607 INFO:tasks.rados.rados.0.trial178.stdout:560: oid 23 updating version 0 to 165 2026-02-21T22:18:42.608 INFO:tasks.rados.rados.0.trial178.stdout:560: oid 23 version 165 is already newer than 164 2026-02-21T22:18:42.608 INFO:tasks.rados.rados.0.trial178.stdout:560: oid 23 version 165 is already newer than 163 2026-02-21T22:18:42.608 INFO:tasks.rados.rados.0.trial178.stdout:560: oid 23 version 165 is already newer than 162 2026-02-21T22:18:42.608 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 165 (ObjNum 203 snap 63 seq_num 203) dirty exists 2026-02-21T22:18:42.608 INFO:tasks.rados.rados.0.trial178.stdout:560: left oid 23 (ObjNum 203 snap 63 seq_num 203) 2026-02-21T22:18:42.610 INFO:tasks.rados.rados.0.trial178.stdout:563: expect (ObjNum 193 snap 61 seq_num 193) 2026-02-21T22:18:42.610 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:42.748 INFO:tasks.rados.rados.0.trial178.stdout:561: finishing copy_from racing read to trial17818531-8 2026-02-21T22:18:42.749 INFO:tasks.rados.rados.0.trial178.stdout:561: finishing copy_from to trial17818531-8 2026-02-21T22:18:42.749 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 122 (ObjNum 137 snap 33 seq_num 137) dirty exists 2026-02-21T22:18:43.450 INFO:tasks.rados.rados.0.trial178.stdout:559: done (5 left) 2026-02-21T22:18:43.450 INFO:tasks.rados.rados.0.trial178.stdout:560: done (4 left) 2026-02-21T22:18:43.450 INFO:tasks.rados.rados.0.trial178.stdout:561: done (3 left) 2026-02-21T22:18:43.450 INFO:tasks.rados.rados.0.trial178.stdout:562: done (2 left) 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:563: done (1 left) 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:564: done (0 left) 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:565: read oid 24 snap -1 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:565: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:566: read oid 10 snap -1 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:566: expect (ObjNum 199 snap 63 seq_num 199) 2026-02-21T22:18:43.451 INFO:tasks.rados.rados.0.trial178.stdout:567: snap_remove snap 55 2026-02-21T22:18:43.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:43.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:44.675 INFO:tasks.rados.rados.0.trial178.stdout:567: done (2 left) 2026-02-21T22:18:44.675 INFO:tasks.rados.rados.0.trial178.stdout:568: snap_create 2026-02-21T22:18:44.675 INFO:tasks.rados.rados.0.trial178.stdout:565: expect (ObjNum 127 snap 30 seq_num 127) 2026-02-21T22:18:44.675 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:44.752 INFO:tasks.rados.rados.0.trial178.stdout:566: expect (ObjNum 199 snap 63 seq_num 199) 2026-02-21T22:18:45.687 INFO:tasks.rados.rados.0.trial178.stdout:565: done (2 left) 2026-02-21T22:18:45.687 INFO:tasks.rados.rados.0.trial178.stdout:566: done (1 left) 2026-02-21T22:18:45.687 INFO:tasks.rados.rados.0.trial178.stdout:568: done (0 left) 2026-02-21T22:18:45.688 INFO:tasks.rados.rados.0.trial178.stdout:569: write oid 11 current snap is 65 2026-02-21T22:18:45.688 INFO:tasks.rados.rados.0.trial178.stdout:569: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2026-02-21T22:18:45.697 INFO:tasks.rados.rados.0.trial178.stdout:569: writing trial17818531-11 from 628628 to 1404602 tid 1 2026-02-21T22:18:45.704 INFO:tasks.rados.rados.0.trial178.stdout:569: writing trial17818531-11 from 1937856 to 2464917 tid 2 2026-02-21T22:18:45.711 INFO:tasks.rados.rados.0.trial178.stdout:569: writing trial17818531-11 from 3252761 to 3428628 tid 3 2026-02-21T22:18:45.711 INFO:tasks.rados.rados.0.trial178.stdout:570: delete oid 38 current snap is 65 2026-02-21T22:18:45.711 INFO:tasks.rados.rados.0.trial178.stdout:569: finishing write tid 1 to trial17818531-11 2026-02-21T22:18:45.711 INFO:tasks.rados.rados.0.trial178.stdout:569: finishing write tid 2 to trial17818531-11 2026-02-21T22:18:45.712 INFO:tasks.rados.rados.0.trial178.stdout:569: finishing write tid 3 to trial17818531-11 2026-02-21T22:18:45.712 INFO:tasks.rados.rados.0.trial178.stdout:569: finishing write tid 4 to trial17818531-11 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: finishing write tid 5 to trial17818531-11 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: oid 11 updating version 0 to 318 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: oid 11 version 318 is already newer than 317 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: oid 11 updating version 318 to 320 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: oid 11 version 320 is already newer than 319 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 320 (ObjNum 204 snap 65 seq_num 204) dirty exists 2026-02-21T22:18:45.713 INFO:tasks.rados.rados.0.trial178.stdout:569: left oid 11 (ObjNum 204 snap 65 seq_num 204) 2026-02-21T22:18:45.714 INFO:tasks.rados.rados.0.trial178.stdout:569: done (1 left) 2026-02-21T22:18:45.714 INFO:tasks.rados.rados.0.trial178.stdout:570: done (0 left) 2026-02-21T22:18:45.714 INFO:tasks.rados.rados.0.trial178.stdout:571: read oid 18 snap 62 2026-02-21T22:18:45.714 INFO:tasks.rados.rados.0.trial178.stdout:571: expect (ObjNum 185 snap 55 seq_num 185) 2026-02-21T22:18:45.714 INFO:tasks.rados.rados.0.trial178.stdout:572: snap_create 2026-02-21T22:18:45.715 INFO:tasks.rados.rados.0.trial178.stdout:571: expect (ObjNum 185 snap 55 seq_num 185) 2026-02-21T22:18:45.757 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2853836 2026-02-21T22:18:46.689 INFO:tasks.rados.rados.0.trial178.stdout:571: done (1 left) 2026-02-21T22:18:46.689 INFO:tasks.rados.rados.0.trial178.stdout:572: done (0 left) 2026-02-21T22:18:46.689 INFO:tasks.rados.rados.0.trial178.stdout:573: read oid 6 snap 51 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:573: expect (ObjNum 138 snap 34 seq_num 138) 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:574: rollback oid 36 current snap is 66 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 53 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:575: rollback oid 28 current snap is 66 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 64 2026-02-21T22:18:46.690 INFO:tasks.rados.rados.0.trial178.stdout:576: delete oid 2 current snap is 66 2026-02-21T22:18:46.693 INFO:tasks.rados.rados.0.trial178.stdout:574: finishing rollback tid 0 to trial17818531-36 2026-02-21T22:18:46.693 INFO:tasks.rados.rados.0.trial178.stdout:574: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:18:46.693 INFO:tasks.rados.rados.0.trial178.stdout:574: finishing rollback tid 2 to trial17818531-36 2026-02-21T22:18:46.693 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 59 (ObjNum 180 snap 52 seq_num 180) dirty exists 2026-02-21T22:18:46.697 INFO:tasks.rados.rados.0.trial178.stdout:573: expect (ObjNum 138 snap 34 seq_num 138) 2026-02-21T22:18:46.746 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1607202 2026-02-21T22:18:46.754 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:46.776 INFO:tasks.rados.rados.0.trial178.stdout:575: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:18:46.776 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 117 (ObjNum 99265 snap 0 seq_num 2885559408) dirty dne 2026-02-21T22:18:46.776 INFO:tasks.rados.rados.0.trial178.stdout:573: done (3 left) 2026-02-21T22:18:46.776 INFO:tasks.rados.rados.0.trial178.stdout:574: done (2 left) 2026-02-21T22:18:46.776 INFO:tasks.rados.rados.0.trial178.stdout:575: done (1 left) 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:576: done (0 left) 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:577: read oid 32 snap -1 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:577: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:578: read oid 37 snap -1 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:578: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:579: write (excl) oid 31 current snap is 66 2026-02-21T22:18:46.777 INFO:tasks.rados.rados.0.trial178.stdout:579: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2026-02-21T22:18:46.785 INFO:tasks.rados.rados.0.trial178.stdout:579: writing trial17818531-31 from 679049 to 1397123 tid 1 2026-02-21T22:18:46.795 INFO:tasks.rados.rados.0.trial178.stdout:579: writing trial17818531-31 from 2113302 to 2787690 tid 2 2026-02-21T22:18:46.800 INFO:tasks.rados.rados.0.trial178.stdout:579: writing trial17818531-31 from 3335030 to 3479049 tid 3 2026-02-21T22:18:46.800 INFO:tasks.rados.rados.0.trial178.stdout:580: snap_remove snap 63 2026-02-21T22:18:47.393 INFO:tasks.rados.rados.0.trial178.stdout:580: done (3 left) 2026-02-21T22:18:47.393 INFO:tasks.rados.rados.0.trial178.stdout:581: snap_remove snap 61 2026-02-21T22:18:48.399 INFO:tasks.rados.rados.0.trial178.stdout:581: done (3 left) 2026-02-21T22:18:48.399 INFO:tasks.rados.rados.0.trial178.stdout:582: rollback oid 43 current snap is 66 2026-02-21T22:18:48.399 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 53 2026-02-21T22:18:48.399 INFO:tasks.rados.rados.0.trial178.stdout:578: expect (ObjNum 131 snap 31 seq_num 131) 2026-02-21T22:18:48.414 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:48.444 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:48.460 INFO:tasks.rados.rados.0.trial178.stdout:577: expect (ObjNum 136 snap 32 seq_num 136) 2026-02-21T22:18:48.460 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:48.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: finishing write tid 1 to trial17818531-31 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: finishing write tid 2 to trial17818531-31 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: finishing write tid 3 to trial17818531-31 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: finishing write tid 4 to trial17818531-31 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: finishing write tid 5 to trial17818531-31 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: oid 31 updating version 0 to 223 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: oid 31 version 223 is already newer than 222 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: oid 31 version 223 is already newer than 221 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: oid 31 updating version 223 to 224 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 224 (ObjNum 207 snap 66 seq_num 207) dirty exists 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: left oid 31 (ObjNum 207 snap 66 seq_num 207) 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:582: finishing rollback tid 0 to trial17818531-43 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:582: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:582: finishing rollback tid 2 to trial17818531-43 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 55 (ObjNum 144 snap 37 seq_num 144) dirty exists 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:577: done (3 left) 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:578: done (2 left) 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:579: done (1 left) 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:582: done (0 left) 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:583: write oid 22 current snap is 66 2026-02-21T22:18:48.543 INFO:tasks.rados.rados.0.trial178.stdout:583: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2026-02-21T22:18:48.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:48.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:48.553 INFO:tasks.rados.rados.0.trial178.stdout:583: writing trial17818531-22 from 695856 to 1489179 tid 1 2026-02-21T22:18:48.561 INFO:tasks.rados.rados.0.trial178.stdout:583: writing trial17818531-22 from 2171784 to 2706183 tid 2 2026-02-21T22:18:48.567 INFO:tasks.rados.rados.0.trial178.stdout:583: writing trial17818531-22 from 3362453 to 3495856 tid 3 2026-02-21T22:18:48.567 INFO:tasks.rados.rados.0.trial178.stdout:584: write (excl) oid 40 current snap is 66 2026-02-21T22:18:48.567 INFO:tasks.rados.rados.0.trial178.stdout:584: seq_num 209 ranges {712663=468572,1713913=678057,2873523=591733} 2026-02-21T22:18:48.574 INFO:tasks.rados.rados.0.trial178.stdout:584: writing trial17818531-40 from 712663 to 1181235 tid 1 2026-02-21T22:18:48.582 INFO:tasks.rados.rados.0.trial178.stdout:584: writing trial17818531-40 from 1713913 to 2391970 tid 2 2026-02-21T22:18:48.589 INFO:tasks.rados.rados.0.trial178.stdout:584: writing trial17818531-40 from 2873523 to 3465256 tid 3 2026-02-21T22:18:48.589 INFO:tasks.rados.rados.0.trial178.stdout:585: snap_create 2026-02-21T22:18:48.589 INFO:tasks.rados.rados.0.trial178.stdout:583: finishing write tid 1 to trial17818531-22 2026-02-21T22:18:48.589 INFO:tasks.rados.rados.0.trial178.stdout:583: finishing write tid 2 to trial17818531-22 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: finishing write tid 3 to trial17818531-22 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: finishing write tid 4 to trial17818531-22 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: finishing write tid 5 to trial17818531-22 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: oid 22 updating version 0 to 136 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: oid 22 version 136 is already newer than 135 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: oid 22 version 136 is already newer than 133 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: oid 22 version 136 is already newer than 134 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 136 (ObjNum 208 snap 66 seq_num 208) dirty exists 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:583: left oid 22 (ObjNum 208 snap 66 seq_num 208) 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:584: finishing write tid 1 to trial17818531-40 2026-02-21T22:18:48.590 INFO:tasks.rados.rados.0.trial178.stdout:584: finishing write tid 2 to trial17818531-40 2026-02-21T22:18:48.591 INFO:tasks.rados.rados.0.trial178.stdout:584: finishing write tid 3 to trial17818531-40 2026-02-21T22:18:48.591 INFO:tasks.rados.rados.0.trial178.stdout:584: finishing write tid 4 to trial17818531-40 2026-02-21T22:18:48.591 INFO:tasks.rados.rados.0.trial178.stdout:584: finishing write tid 5 to trial17818531-40 2026-02-21T22:18:48.591 INFO:tasks.rados.rados.0.trial178.stdout:584: oid 40 updating version 0 to 140 2026-02-21T22:18:48.592 INFO:tasks.rados.rados.0.trial178.stdout:584: oid 40 version 140 is already newer than 137 2026-02-21T22:18:48.592 INFO:tasks.rados.rados.0.trial178.stdout:584: oid 40 version 140 is already newer than 139 2026-02-21T22:18:48.592 INFO:tasks.rados.rados.0.trial178.stdout:584: oid 40 version 140 is already newer than 138 2026-02-21T22:18:48.592 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 140 (ObjNum 209 snap 66 seq_num 209) dirty exists 2026-02-21T22:18:48.592 INFO:tasks.rados.rados.0.trial178.stdout:584: left oid 40 (ObjNum 209 snap 66 seq_num 209) 2026-02-21T22:18:49.399 INFO:tasks.rados.rados.0.trial178.stdout:583: done (2 left) 2026-02-21T22:18:49.400 INFO:tasks.rados.rados.0.trial178.stdout:584: done (1 left) 2026-02-21T22:18:49.400 INFO:tasks.rados.rados.0.trial178.stdout:585: done (0 left) 2026-02-21T22:18:49.400 INFO:tasks.rados.rados.0.trial178.stdout:586: snap_remove snap 62 2026-02-21T22:18:50.612 INFO:tasks.rados.rados.0.trial178.stdout:586: done (0 left) 2026-02-21T22:18:50.612 INFO:tasks.rados.rados.0.trial178.stdout:587: delete oid 9 current snap is 67 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:587: done (0 left) 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:588: read oid 33 snap -1 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:588: expect deleted 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:589: rollback oid 34 current snap is 67 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 34 to 54 2026-02-21T22:18:50.616 INFO:tasks.rados.rados.0.trial178.stdout:590: write oid 2 current snap is 67 2026-02-21T22:18:50.617 INFO:tasks.rados.rados.0.trial178.stdout:590: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2026-02-21T22:18:50.625 INFO:tasks.rados.rados.0.trial178.stdout:590: writing trial17818531-2 from 746277 to 1365347 tid 1 2026-02-21T22:18:50.633 INFO:tasks.rados.rados.0.trial178.stdout:590: writing trial17818531-2 from 1830877 to 2512603 tid 2 2026-02-21T22:18:50.640 INFO:tasks.rados.rados.0.trial178.stdout:590: writing trial17818531-2 from 2928369 to 3546277 tid 3 2026-02-21T22:18:50.640 INFO:tasks.rados.rados.0.trial178.stdout:591: snap_remove snap 64 2026-02-21T22:18:51.617 INFO:tasks.rados.rados.0.trial178.stdout:591: done (3 left) 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:592: delete oid 17 current snap is 67 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:589: finishing rollback tid 0 to trial17818531-34 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:589: finishing rollback tid 1 to trial17818531-34 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:589: finishing rollback tid 2 to trial17818531-34 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 75 (ObjNum 156 snap 43 seq_num 156) dirty exists 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:590: finishing write tid 1 to trial17818531-2 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:590: finishing write tid 2 to trial17818531-2 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:590: finishing write tid 3 to trial17818531-2 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:590: finishing write tid 4 to trial17818531-2 2026-02-21T22:18:51.618 INFO:tasks.rados.rados.0.trial178.stdout:590: finishing write tid 5 to trial17818531-2 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:590: oid 2 updating version 0 to 177 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:590: oid 2 version 177 is already newer than 176 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:590: oid 2 version 177 is already newer than 174 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:590: oid 2 version 177 is already newer than 175 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 177 (ObjNum 211 snap 67 seq_num 211) dirty exists 2026-02-21T22:18:51.619 INFO:tasks.rados.rados.0.trial178.stdout:590: left oid 2 (ObjNum 211 snap 67 seq_num 211) 2026-02-21T22:18:51.621 INFO:tasks.rados.rados.0.trial178.stdout:588: done (3 left) 2026-02-21T22:18:51.622 INFO:tasks.rados.rados.0.trial178.stdout:589: done (2 left) 2026-02-21T22:18:51.622 INFO:tasks.rados.rados.0.trial178.stdout:590: done (1 left) 2026-02-21T22:18:51.622 INFO:tasks.rados.rados.0.trial178.stdout:592: done (0 left) 2026-02-21T22:18:51.622 INFO:tasks.rados.rados.0.trial178.stdout:593: write oid 21 current snap is 67 2026-02-21T22:18:51.622 INFO:tasks.rados.rados.0.trial178.stdout:593: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2026-02-21T22:18:51.630 INFO:tasks.rados.rados.0.trial178.stdout:593: writing trial17818531-21 from 779891 to 1265812 tid 1 2026-02-21T22:18:51.639 INFO:tasks.rados.rados.0.trial178.stdout:593: writing trial17818531-21 from 1947841 to 2633236 tid 2 2026-02-21T22:18:51.645 INFO:tasks.rados.rados.0.trial178.stdout:593: writing trial17818531-21 from 3383215 to 3579891 tid 3 2026-02-21T22:18:51.645 INFO:tasks.rados.rados.0.trial178.stdout:594: snap_remove snap 57 2026-02-21T22:18:52.394 INFO:tasks.rados.rados.0.trial178.stdout:594: done (1 left) 2026-02-21T22:18:52.394 INFO:tasks.rados.rados.0.trial178.stdout:595: write (excl) oid 31 current snap is 67 2026-02-21T22:18:52.395 INFO:tasks.rados.rados.0.trial178.stdout:595: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2026-02-21T22:18:52.403 INFO:tasks.rados.rados.0.trial178.stdout:595: writing trial17818531-31 from 796698 to 1357868 tid 1 2026-02-21T22:18:52.411 INFO:tasks.rados.rados.0.trial178.stdout:595: writing trial17818531-31 from 2006323 to 2551729 tid 2 2026-02-21T22:18:52.419 INFO:tasks.rados.rados.0.trial178.stdout:595: writing trial17818531-31 from 3126991 to 3596698 tid 3 2026-02-21T22:18:52.419 INFO:tasks.rados.rados.0.trial178.stdout:596: read oid 25 snap 53 2026-02-21T22:18:52.419 INFO:tasks.rados.rados.0.trial178.stdout:596: expect (ObjNum 160 snap 47 seq_num 160) 2026-02-21T22:18:52.419 INFO:tasks.rados.rados.0.trial178.stdout:597: snap_remove snap 65 2026-02-21T22:18:53.395 INFO:tasks.rados.rados.0.trial178.stdout:597: done (3 left) 2026-02-21T22:18:53.395 INFO:tasks.rados.rados.0.trial178.stdout:598: write (excl) oid 48 current snap is 67 2026-02-21T22:18:53.395 INFO:tasks.rados.rados.0.trial178.stdout:598: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2026-02-21T22:18:53.403 INFO:tasks.rados.rados.0.trial178.stdout:598: writing trial17818531-48 from 413505 to 1049924 tid 1 2026-02-21T22:18:53.412 INFO:tasks.rados.rados.0.trial178.stdout:598: writing trial17818531-48 from 1664805 to 2353869 tid 2 2026-02-21T22:18:53.420 INFO:tasks.rados.rados.0.trial178.stdout:598: writing trial17818531-48 from 3038061 to 3530132 tid 3 2026-02-21T22:18:53.420 INFO:tasks.rados.rados.0.trial178.stdout:599: snap_create 2026-02-21T22:18:53.420 INFO:tasks.rados.rados.0.trial178.stdout:593: finishing write tid 1 to trial17818531-21 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: finishing write tid 2 to trial17818531-21 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: finishing write tid 3 to trial17818531-21 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: finishing write tid 4 to trial17818531-21 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: finishing write tid 5 to trial17818531-21 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: oid 21 updating version 0 to 328 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: oid 21 version 328 is already newer than 327 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: oid 21 version 328 is already newer than 326 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: oid 21 updating version 328 to 329 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 329 (ObjNum 213 snap 67 seq_num 213) dirty exists 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:593: left oid 21 (ObjNum 213 snap 67 seq_num 213) 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:595: finishing write tid 1 to trial17818531-31 2026-02-21T22:18:53.421 INFO:tasks.rados.rados.0.trial178.stdout:595: finishing write tid 2 to trial17818531-31 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: finishing write tid 3 to trial17818531-31 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: finishing write tid 4 to trial17818531-31 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: finishing write tid 5 to trial17818531-31 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: oid 31 updating version 0 to 228 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: oid 31 version 228 is already newer than 226 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: oid 31 version 228 is already newer than 227 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: oid 31 version 228 is already newer than 225 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 228 (ObjNum 214 snap 67 seq_num 214) dirty exists 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:595: left oid 31 (ObjNum 214 snap 67 seq_num 214) 2026-02-21T22:18:53.422 INFO:tasks.rados.rados.0.trial178.stdout:596: expect (ObjNum 160 snap 47 seq_num 160) 2026-02-21T22:18:53.455 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:53.497 INFO:tasks.rados.rados.0.trial178.stdout:598: finishing write tid 1 to trial17818531-48 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: finishing write tid 2 to trial17818531-48 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: finishing write tid 3 to trial17818531-48 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: finishing write tid 4 to trial17818531-48 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: finishing write tid 5 to trial17818531-48 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: oid 48 updating version 0 to 135 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: oid 48 version 135 is already newer than 133 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: oid 48 updating version 135 to 136 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: oid 48 version 136 is already newer than 134 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 136 (ObjNum 215 snap 67 seq_num 215) dirty exists 2026-02-21T22:18:53.498 INFO:tasks.rados.rados.0.trial178.stdout:598: left oid 48 (ObjNum 215 snap 67 seq_num 215) 2026-02-21T22:18:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:18:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:18:54.398 INFO:tasks.rados.rados.0.trial178.stdout:593: done (4 left) 2026-02-21T22:18:54.398 INFO:tasks.rados.rados.0.trial178.stdout:595: done (3 left) 2026-02-21T22:18:54.398 INFO:tasks.rados.rados.0.trial178.stdout:596: done (2 left) 2026-02-21T22:18:54.398 INFO:tasks.rados.rados.0.trial178.stdout:598: done (1 left) 2026-02-21T22:18:54.399 INFO:tasks.rados.rados.0.trial178.stdout:599: done (0 left) 2026-02-21T22:18:54.399 INFO:tasks.rados.rados.0.trial178.stdout:600: snap_create 2026-02-21T22:18:55.401 INFO:tasks.rados.rados.0.trial178.stdout:600: done (0 left) 2026-02-21T22:18:55.401 INFO:tasks.rados.rados.0.trial178.stdout:601: snap_remove snap 51 2026-02-21T22:18:56.404 INFO:tasks.rados.rados.0.trial178.stdout:601: done (0 left) 2026-02-21T22:18:56.404 INFO:tasks.rados.rados.0.trial178.stdout:602: delete oid 25 current snap is 69 2026-02-21T22:18:56.410 INFO:tasks.rados.rados.0.trial178.stdout:602: done (0 left) 2026-02-21T22:18:56.410 INFO:tasks.rados.rados.0.trial178.stdout:603: copy_from oid 32 from oid 28 current snap is 69 2026-02-21T22:18:56.410 INFO:tasks.rados.rados.0.trial178.stdout:604: snap_remove snap 67 2026-02-21T22:18:57.395 INFO:tasks.rados.rados.0.trial178.stdout:604: done (1 left) 2026-02-21T22:18:57.395 INFO:tasks.rados.rados.0.trial178.stdout:605: delete oid 43 current snap is 69 2026-02-21T22:18:57.395 INFO:tasks.rados.rados.0.trial178.stdout:603: finishing copy_from to trial17818531-32 2026-02-21T22:18:57.395 INFO:tasks.rados.rados.0.trial178.stdout:603: got expected ENOENT (src dne) 2026-02-21T22:18:57.395 INFO:tasks.rados.rados.0.trial178.stdout:603: finishing copy_from racing read to trial17818531-32 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:603: done (1 left) 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:605: done (0 left) 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:606: copy_from oid 39 from oid 45 current snap is 69 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:607: read oid 8 snap 54 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:607: expect (ObjNum 167 snap 48 seq_num 167) 2026-02-21T22:18:57.400 INFO:tasks.rados.rados.0.trial178.stdout:608: delete oid 46 current snap is 69 2026-02-21T22:18:57.401 INFO:tasks.rados.rados.0.trial178.stdout:608: done (2 left) 2026-02-21T22:18:57.401 INFO:tasks.rados.rados.0.trial178.stdout:609: delete oid 6 current snap is 69 2026-02-21T22:18:57.403 INFO:tasks.rados.rados.0.trial178.stdout:609: done (2 left) 2026-02-21T22:18:57.403 INFO:tasks.rados.rados.0.trial178.stdout:610: read oid 16 snap -1 2026-02-21T22:18:57.403 INFO:tasks.rados.rados.0.trial178.stdout:610: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:18:57.403 INFO:tasks.rados.rados.0.trial178.stdout:611: snap_remove snap 53 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:611: done (3 left) 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:612: read oid 2 snap 68 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:612: expect (ObjNum 211 snap 67 seq_num 211) 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:613: rollback oid 43 current snap is 69 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 66 2026-02-21T22:18:58.409 INFO:tasks.rados.rados.0.trial178.stdout:610: expect (ObjNum 156 snap 43 seq_num 156) 2026-02-21T22:18:58.443 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1443604 2026-02-21T22:18:58.479 INFO:tasks.rados.rados.0.trial178.stdout:607: expect (ObjNum 167 snap 48 seq_num 167) 2026-02-21T22:18:58.538 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:58.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:18:58.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:18:58.568 INFO:tasks.rados.rados.0.trial178.stdout:606: finishing copy_from racing read to trial17818531-39 2026-02-21T22:18:58.568 INFO:tasks.rados.rados.0.trial178.stdout:606: finishing copy_from to trial17818531-39 2026-02-21T22:18:58.568 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 82 (ObjNum 165 snap 47 seq_num 165) dirty exists 2026-02-21T22:18:58.568 INFO:tasks.rados.rados.0.trial178.stdout:613: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:18:58.569 INFO:tasks.rados.rados.0.trial178.stdout:613: finishing rollback tid 2 to trial17818531-43 2026-02-21T22:18:58.569 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 71 (ObjNum 144 snap 37 seq_num 144) dirty exists 2026-02-21T22:18:58.569 INFO:tasks.rados.rados.0.trial178.stdout:612: expect (ObjNum 211 snap 67 seq_num 211) 2026-02-21T22:18:58.575 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1862818 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:606: done (4 left) 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:607: done (3 left) 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:610: done (2 left) 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:612: done (1 left) 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:613: done (0 left) 2026-02-21T22:18:58.631 INFO:tasks.rados.rados.0.trial178.stdout:614: delete oid 10 current snap is 69 2026-02-21T22:18:58.634 INFO:tasks.rados.rados.0.trial178.stdout:614: done (0 left) 2026-02-21T22:18:58.634 INFO:tasks.rados.rados.0.trial178.stdout:615: copy_from oid 27 from oid 9 current snap is 69 2026-02-21T22:18:58.634 INFO:tasks.rados.rados.0.trial178.stdout:616: copy_from oid 16 from oid 12 current snap is 69 2026-02-21T22:18:58.635 INFO:tasks.rados.rados.0.trial178.stdout:617: copy_from oid 48 from oid 29 current snap is 69 2026-02-21T22:18:58.635 INFO:tasks.rados.rados.0.trial178.stdout:618: delete oid 35 current snap is 69 2026-02-21T22:18:58.635 INFO:tasks.rados.rados.0.trial178.stdout:615: finishing copy_from to trial17818531-27 2026-02-21T22:18:58.635 INFO:tasks.rados.rados.0.trial178.stdout:615: got expected ENOENT (src dne) 2026-02-21T22:18:58.635 INFO:tasks.rados.rados.0.trial178.stdout:615: finishing copy_from racing read to trial17818531-27 2026-02-21T22:18:58.638 INFO:tasks.rados.rados.0.trial178.stdout:616: finishing copy_from to trial17818531-16 2026-02-21T22:18:58.638 INFO:tasks.rados.rados.0.trial178.stdout:616: got expected ENOENT (src dne) 2026-02-21T22:18:58.638 INFO:tasks.rados.rados.0.trial178.stdout:616: finishing copy_from racing read to trial17818531-16 2026-02-21T22:18:58.639 INFO:tasks.rados.rados.0.trial178.stdout:615: done (3 left) 2026-02-21T22:18:58.639 INFO:tasks.rados.rados.0.trial178.stdout:616: done (2 left) 2026-02-21T22:18:58.639 INFO:tasks.rados.rados.0.trial178.stdout:618: done (1 left) 2026-02-21T22:18:58.639 INFO:tasks.rados.rados.0.trial178.stdout:619: read oid 2 snap -1 2026-02-21T22:18:58.640 INFO:tasks.rados.rados.0.trial178.stdout:619: expect (ObjNum 211 snap 67 seq_num 211) 2026-02-21T22:18:58.640 INFO:tasks.rados.rados.0.trial178.stdout:620: write oid 22 current snap is 69 2026-02-21T22:18:58.640 INFO:tasks.rados.rados.0.trial178.stdout:620: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2026-02-21T22:18:58.646 INFO:tasks.rados.rados.0.trial178.stdout:620: writing trial17818531-22 from 531154 to 1010669 tid 1 2026-02-21T22:18:58.652 INFO:tasks.rados.rados.0.trial178.stdout:620: writing trial17818531-22 from 1557826 to 2001555 tid 2 2026-02-21T22:18:58.661 INFO:tasks.rados.rados.0.trial178.stdout:620: writing trial17818531-22 from 2713669 to 3200997 tid 3 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:620: writing trial17818531-22 from 3679295 to 3731154 tid 4 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:621: read oid 8 snap 68 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:621: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:622: snap_create 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:619: expect (ObjNum 211 snap 67 seq_num 211) 2026-02-21T22:18:58.665 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:58.723 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:58.747 INFO:tasks.rados.rados.0.trial178.stdout:617: finishing copy_from racing read to trial17818531-48 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 1 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:617: finishing copy_from to trial17818531-48 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 2 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 3 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 4 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 5 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: finishing write tid 6 to trial17818531-22 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: oid 22 updating version 0 to 160 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: oid 22 version 160 is already newer than 159 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: oid 22 updating version 160 to 162 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: oid 22 version 162 is already newer than 161 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: oid 22 version 162 is already newer than 158 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 162 (ObjNum 222 snap 69 seq_num 222) dirty exists 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:620: left oid 22 (ObjNum 222 snap 69 seq_num 222) 2026-02-21T22:18:58.748 INFO:tasks.rados.rados.0.trial178.stdout:621: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:18:58.790 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:18:59.413 INFO:tasks.rados.rados.0.trial178.stdout:617: done (4 left) 2026-02-21T22:18:59.414 INFO:tasks.rados.rados.0.trial178.stdout:619: done (3 left) 2026-02-21T22:18:59.414 INFO:tasks.rados.rados.0.trial178.stdout:620: done (2 left) 2026-02-21T22:18:59.414 INFO:tasks.rados.rados.0.trial178.stdout:621: done (1 left) 2026-02-21T22:18:59.414 INFO:tasks.rados.rados.0.trial178.stdout:622: done (0 left) 2026-02-21T22:18:59.414 INFO:tasks.rados.rados.0.trial178.stdout:623: snap_remove snap 68 2026-02-21T22:19:00.416 INFO:tasks.rados.rados.0.trial178.stdout:623: done (0 left) 2026-02-21T22:19:00.417 INFO:tasks.rados.rados.0.trial178.stdout:624: delete oid 40 current snap is 70 2026-02-21T22:19:00.427 INFO:tasks.rados.rados.0.trial178.stdout:624: done (0 left) 2026-02-21T22:19:00.427 INFO:tasks.rados.rados.0.trial178.stdout:625: read oid 45 snap -1 2026-02-21T22:19:00.427 INFO:tasks.rados.rados.0.trial178.stdout:625: expect (ObjNum 165 snap 47 seq_num 165) 2026-02-21T22:19:00.427 INFO:tasks.rados.rados.0.trial178.stdout:626: snap_create 2026-02-21T22:19:00.433 INFO:tasks.rados.rados.0.trial178.stdout:625: expect (ObjNum 165 snap 47 seq_num 165) 2026-02-21T22:19:00.445 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:01.875 INFO:tasks.rados.rados.0.trial178.stdout:625: done (1 left) 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:626: done (0 left) 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:627: copy_from oid 6 from oid 33 current snap is 71 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:628: read oid 43 snap -1 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:628: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:629: read oid 50 snap -1 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:629: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:630: write (excl) oid 41 current snap is 71 2026-02-21T22:19:01.876 INFO:tasks.rados.rados.0.trial178.stdout:630: seq_num 224 ranges {564768=630013,1674790=447398,2768515=626225} 2026-02-21T22:19:01.883 INFO:tasks.rados.rados.0.trial178.stdout:630: writing trial17818531-41 from 564768 to 1194781 tid 1 2026-02-21T22:19:01.890 INFO:tasks.rados.rados.0.trial178.stdout:630: writing trial17818531-41 from 1674790 to 2122188 tid 2 2026-02-21T22:19:01.898 INFO:tasks.rados.rados.0.trial178.stdout:630: writing trial17818531-41 from 2768515 to 3394740 tid 3 2026-02-21T22:19:01.899 INFO:tasks.rados.rados.0.trial178.stdout:631: snap_remove snap 54 2026-02-21T22:19:02.396 INFO:tasks.rados.rados.0.trial178.stdout:631: done (4 left) 2026-02-21T22:19:02.396 INFO:tasks.rados.rados.0.trial178.stdout:632: write oid 32 current snap is 71 2026-02-21T22:19:02.396 INFO:tasks.rados.rados.0.trial178.stdout:632: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2026-02-21T22:19:02.403 INFO:tasks.rados.rados.0.trial178.stdout:632: writing trial17818531-32 from 581575 to 1286837 tid 1 2026-02-21T22:19:02.407 INFO:tasks.rados.rados.0.trial178.stdout:632: writing trial17818531-32 from 1733272 to 2440681 tid 2 2026-02-21T22:19:02.412 INFO:tasks.rados.rados.0.trial178.stdout:632: writing trial17818531-32 from 3195938 to 3633435 tid 3 2026-02-21T22:19:02.412 INFO:tasks.rados.rados.0.trial178.stdout:633: read oid 5 snap 69 2026-02-21T22:19:02.412 INFO:tasks.rados.rados.0.trial178.stdout:633: expect (ObjNum 170 snap 48 seq_num 170) 2026-02-21T22:19:02.412 INFO:tasks.rados.rados.0.trial178.stdout:634: snap_remove snap 70 2026-02-21T22:19:03.396 INFO:tasks.rados.rados.0.trial178.stdout:634: done (6 left) 2026-02-21T22:19:03.397 INFO:tasks.rados.rados.0.trial178.stdout:635: snap_remove snap 66 2026-02-21T22:19:03.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:03.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:635: done (6 left) 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:636: read oid 11 snap -1 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:636: expect (ObjNum 204 snap 65 seq_num 204) 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:637: rollback oid 40 current snap is 71 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 69 2026-02-21T22:19:04.400 INFO:tasks.rados.rados.0.trial178.stdout:628: expect (ObjNum 144 snap 37 seq_num 144) 2026-02-21T22:19:04.482 INFO:tasks.rados.rados.0.trial178.stdout:627: finishing copy_from to trial17818531-6 2026-02-21T22:19:04.483 INFO:tasks.rados.rados.0.trial178.stdout:627: got expected ENOENT (src dne) 2026-02-21T22:19:04.483 INFO:tasks.rados.rados.0.trial178.stdout:627: finishing copy_from racing read to trial17818531-6 2026-02-21T22:19:04.483 INFO:tasks.rados.rados.0.trial178.stdout:629: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:04.483 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:04.521 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:04.601 INFO:tasks.rados.rados.0.trial178.stdout:630: finishing write tid 1 to trial17818531-41 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: finishing write tid 2 to trial17818531-41 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: finishing write tid 3 to trial17818531-41 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: finishing write tid 4 to trial17818531-41 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: finishing write tid 5 to trial17818531-41 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: oid 41 updating version 0 to 200 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: oid 41 updating version 200 to 201 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: oid 41 updating version 201 to 202 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: oid 41 version 202 is already newer than 199 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 202 (ObjNum 224 snap 71 seq_num 224) dirty exists 2026-02-21T22:19:04.602 INFO:tasks.rados.rados.0.trial178.stdout:630: left oid 41 (ObjNum 224 snap 71 seq_num 224) 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: finishing write tid 1 to trial17818531-32 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: finishing write tid 2 to trial17818531-32 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: finishing write tid 3 to trial17818531-32 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: finishing write tid 4 to trial17818531-32 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: finishing write tid 5 to trial17818531-32 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: oid 32 updating version 0 to 352 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: oid 32 updating version 352 to 354 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: oid 32 version 354 is already newer than 353 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: oid 32 version 354 is already newer than 349 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 354 (ObjNum 225 snap 71 seq_num 225) dirty exists 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:632: left oid 32 (ObjNum 225 snap 71 seq_num 225) 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:633: expect (ObjNum 170 snap 48 seq_num 170) 2026-02-21T22:19:04.603 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:04.675 INFO:tasks.rados.rados.0.trial178.stdout:636: expect (ObjNum 204 snap 65 seq_num 204) 2026-02-21T22:19:04.675 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:04.819 INFO:tasks.rados.rados.0.trial178.stdout:637: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:637: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 176 (ObjNum 209 snap 66 seq_num 209) dirty exists 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:627: done (7 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:628: done (6 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:629: done (5 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:630: done (4 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:632: done (3 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:633: done (2 left) 2026-02-21T22:19:04.820 INFO:tasks.rados.rados.0.trial178.stdout:636: done (1 left) 2026-02-21T22:19:04.821 INFO:tasks.rados.rados.0.trial178.stdout:637: done (0 left) 2026-02-21T22:19:04.821 INFO:tasks.rados.rados.0.trial178.stdout:638: write (excl) oid 14 current snap is 71 2026-02-21T22:19:04.821 INFO:tasks.rados.rados.0.trial178.stdout:638: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2026-02-21T22:19:04.828 INFO:tasks.rados.rados.0.trial178.stdout:638: writing trial17818531-14 from 598382 to 1378893 tid 1 2026-02-21T22:19:04.837 INFO:tasks.rados.rados.0.trial178.stdout:638: writing trial17818531-14 from 2075401 to 2526468 tid 2 2026-02-21T22:19:04.845 INFO:tasks.rados.rados.0.trial178.stdout:638: writing trial17818531-14 from 3107008 to 3755777 tid 3 2026-02-21T22:19:04.846 INFO:tasks.rados.rados.0.trial178.stdout:639: write oid 18 current snap is 71 2026-02-21T22:19:04.846 INFO:tasks.rados.rados.0.trial178.stdout:638: finishing write tid 1 to trial17818531-14 2026-02-21T22:19:04.846 INFO:tasks.rados.rados.0.trial178.stdout:638: finishing write tid 2 to trial17818531-14 2026-02-21T22:19:04.846 INFO:tasks.rados.rados.0.trial178.stdout:639: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2026-02-21T22:19:04.852 INFO:tasks.rados.rados.0.trial178.stdout:639: writing trial17818531-18 from 615189 to 1070949 tid 1 2026-02-21T22:19:04.862 INFO:tasks.rados.rados.0.trial178.stdout:639: writing trial17818531-18 from 1733883 to 2444961 tid 2 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:639: writing trial17818531-18 from 3134431 to 3594472 tid 3 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:640: delete oid 6 current snap is 71 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: finishing write tid 3 to trial17818531-14 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: finishing write tid 4 to trial17818531-14 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: finishing write tid 5 to trial17818531-14 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: oid 14 updating version 0 to 300 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: oid 14 updating version 300 to 302 2026-02-21T22:19:04.871 INFO:tasks.rados.rados.0.trial178.stdout:638: oid 14 version 302 is already newer than 299 2026-02-21T22:19:04.872 INFO:tasks.rados.rados.0.trial178.stdout:638: oid 14 version 302 is already newer than 301 2026-02-21T22:19:04.872 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 302 (ObjNum 226 snap 71 seq_num 226) dirty exists 2026-02-21T22:19:04.872 INFO:tasks.rados.rados.0.trial178.stdout:638: left oid 14 (ObjNum 226 snap 71 seq_num 226) 2026-02-21T22:19:04.872 INFO:tasks.rados.rados.0.trial178.stdout:639: finishing write tid 1 to trial17818531-18 2026-02-21T22:19:04.872 INFO:tasks.rados.rados.0.trial178.stdout:639: finishing write tid 2 to trial17818531-18 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:638: done (2 left) 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:640: done (1 left) 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:641: copy_from oid 23 from oid 44 current snap is 71 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:642: read oid 4 snap -1 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:642: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:643: write oid 11 current snap is 71 2026-02-21T22:19:04.873 INFO:tasks.rados.rados.0.trial178.stdout:643: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2026-02-21T22:19:04.876 INFO:tasks.rados.rados.0.trial178.stdout:643: writing trial17818531-11 from 648803 to 1371414 tid 1 2026-02-21T22:19:04.881 INFO:tasks.rados.rados.0.trial178.stdout:643: writing trial17818531-11 from 1967200 to 2681947 tid 2 2026-02-21T22:19:04.886 INFO:tasks.rados.rados.0.trial178.stdout:643: writing trial17818531-11 from 3421983 to 3848803 tid 3 2026-02-21T22:19:04.886 INFO:tasks.rados.rados.0.trial178.stdout:644: delete oid 43 current snap is 71 2026-02-21T22:19:04.886 INFO:tasks.rados.rados.0.trial178.stdout:639: finishing write tid 3 to trial17818531-18 2026-02-21T22:19:04.886 INFO:tasks.rados.rados.0.trial178.stdout:639: finishing write tid 4 to trial17818531-18 2026-02-21T22:19:04.886 INFO:tasks.rados.rados.0.trial178.stdout:639: finishing write tid 5 to trial17818531-18 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:639: oid 18 updating version 0 to 205 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:639: oid 18 version 205 is already newer than 203 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:639: oid 18 version 205 is already newer than 204 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:639: oid 18 version 205 is already newer than 202 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 205 (ObjNum 227 snap 71 seq_num 227) dirty exists 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:639: left oid 18 (ObjNum 227 snap 71 seq_num 227) 2026-02-21T22:19:04.887 INFO:tasks.rados.rados.0.trial178.stdout:642: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:04.945 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1303244 2026-02-21T22:19:04.968 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:05.001 INFO:tasks.rados.rados.0.trial178.stdout:643: finishing write tid 1 to trial17818531-11 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:641: finishing copy_from racing read to trial17818531-23 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:641: finishing copy_from to trial17818531-23 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 94 (ObjNum 119 snap 29 seq_num 119) dirty exists 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: finishing write tid 2 to trial17818531-11 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: finishing write tid 3 to trial17818531-11 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: finishing write tid 4 to trial17818531-11 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: finishing write tid 5 to trial17818531-11 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: oid 11 updating version 0 to 363 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: oid 11 version 363 is already newer than 362 2026-02-21T22:19:05.002 INFO:tasks.rados.rados.0.trial178.stdout:643: oid 11 version 363 is already newer than 360 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:643: oid 11 version 363 is already newer than 361 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 363 (ObjNum 229 snap 71 seq_num 229) dirty exists 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:643: left oid 11 (ObjNum 229 snap 71 seq_num 229) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:639: done (4 left) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:641: done (3 left) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:642: done (2 left) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:643: done (1 left) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:644: done (0 left) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:645: read oid 48 snap -1 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:645: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:05.003 INFO:tasks.rados.rados.0.trial178.stdout:646: delete oid 41 current snap is 71 2026-02-21T22:19:05.006 INFO:tasks.rados.rados.0.trial178.stdout:646: done (1 left) 2026-02-21T22:19:05.006 INFO:tasks.rados.rados.0.trial178.stdout:647: copy_from oid 16 from oid 39 current snap is 71 2026-02-21T22:19:05.006 INFO:tasks.rados.rados.0.trial178.stdout:648: snap_create 2026-02-21T22:19:05.007 INFO:tasks.rados.rados.0.trial178.stdout:645: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:05.143 INFO:tasks.rados.rados.0.trial178.stdout:647: finishing copy_from racing read to trial17818531-16 2026-02-21T22:19:05.144 INFO:tasks.rados.rados.0.trial178.stdout:647: finishing copy_from to trial17818531-16 2026-02-21T22:19:05.144 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 82 (ObjNum 165 snap 47 seq_num 165) dirty exists 2026-02-21T22:19:05.405 INFO:tasks.rados.rados.0.trial178.stdout:645: done (2 left) 2026-02-21T22:19:05.405 INFO:tasks.rados.rados.0.trial178.stdout:647: done (1 left) 2026-02-21T22:19:05.405 INFO:tasks.rados.rados.0.trial178.stdout:648: done (0 left) 2026-02-21T22:19:05.405 INFO:tasks.rados.rados.0.trial178.stdout:649: read oid 14 snap 71 2026-02-21T22:19:05.405 INFO:tasks.rados.rados.0.trial178.stdout:649: expect (ObjNum 226 snap 71 seq_num 226) 2026-02-21T22:19:05.406 INFO:tasks.rados.rados.0.trial178.stdout:650: write oid 47 current snap is 72 2026-02-21T22:19:05.406 INFO:tasks.rados.rados.0.trial178.stdout:650: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2026-02-21T22:19:05.413 INFO:tasks.rados.rados.0.trial178.stdout:650: writing trial17818531-47 from 699224 to 1247582 tid 1 2026-02-21T22:19:05.422 INFO:tasks.rados.rados.0.trial178.stdout:650: writing trial17818531-47 from 2026293 to 2488367 tid 2 2026-02-21T22:19:05.431 INFO:tasks.rados.rados.0.trial178.stdout:650: writing trial17818531-47 from 3271546 to 3704300 tid 3 2026-02-21T22:19:05.431 INFO:tasks.rados.rados.0.trial178.stdout:651: read oid 15 snap -1 2026-02-21T22:19:05.431 INFO:tasks.rados.rados.0.trial178.stdout:651: expect (ObjNum 14 snap 0 seq_num 14) 2026-02-21T22:19:05.431 INFO:tasks.rados.rados.0.trial178.stdout:652: snap_remove snap 69 2026-02-21T22:19:06.409 INFO:tasks.rados.rados.0.trial178.stdout:652: done (3 left) 2026-02-21T22:19:06.409 INFO:tasks.rados.rados.0.trial178.stdout:653: copy_from oid 3 from oid 33 current snap is 72 2026-02-21T22:19:06.409 INFO:tasks.rados.rados.0.trial178.stdout:649: expect (ObjNum 226 snap 71 seq_num 226) 2026-02-21T22:19:06.492 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: finishing write tid 1 to trial17818531-47 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: finishing write tid 2 to trial17818531-47 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: finishing write tid 3 to trial17818531-47 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: finishing write tid 4 to trial17818531-47 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: finishing write tid 5 to trial17818531-47 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: oid 47 updating version 0 to 346 2026-02-21T22:19:06.539 INFO:tasks.rados.rados.0.trial178.stdout:650: oid 47 updating version 346 to 349 2026-02-21T22:19:06.540 INFO:tasks.rados.rados.0.trial178.stdout:650: oid 47 version 349 is already newer than 347 2026-02-21T22:19:06.540 INFO:tasks.rados.rados.0.trial178.stdout:650: oid 47 version 349 is already newer than 348 2026-02-21T22:19:06.540 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 349 (ObjNum 232 snap 72 seq_num 232) dirty exists 2026-02-21T22:19:06.540 INFO:tasks.rados.rados.0.trial178.stdout:650: left oid 47 (ObjNum 232 snap 72 seq_num 232) 2026-02-21T22:19:06.540 INFO:tasks.rados.rados.0.trial178.stdout:651: expect (ObjNum 14 snap 0 seq_num 14) 2026-02-21T22:19:06.573 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2018715 2026-02-21T22:19:06.580 INFO:tasks.rados.rados.0.trial178.stdout:653: finishing copy_from to trial17818531-3 2026-02-21T22:19:06.580 INFO:tasks.rados.rados.0.trial178.stdout:653: got expected ENOENT (src dne) 2026-02-21T22:19:06.580 INFO:tasks.rados.rados.0.trial178.stdout:653: finishing copy_from racing read to trial17818531-3 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:649: done (3 left) 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:650: done (2 left) 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:651: done (1 left) 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:653: done (0 left) 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:654: write (excl) oid 43 current snap is 72 2026-02-21T22:19:06.581 INFO:tasks.rados.rados.0.trial178.stdout:654: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2026-02-21T22:19:06.589 INFO:tasks.rados.rados.0.trial178.stdout:654: writing trial17818531-43 from 716031 to 1339638 tid 1 2026-02-21T22:19:06.599 INFO:tasks.rados.rados.0.trial178.stdout:654: writing trial17818531-43 from 2084775 to 2690507 tid 2 2026-02-21T22:19:06.607 INFO:tasks.rados.rados.0.trial178.stdout:654: writing trial17818531-43 from 3298969 to 3916031 tid 3 2026-02-21T22:19:06.608 INFO:tasks.rados.rados.0.trial178.stdout:655: write (excl) oid 15 current snap is 72 2026-02-21T22:19:06.608 INFO:tasks.rados.rados.0.trial178.stdout:654: finishing write tid 1 to trial17818531-43 2026-02-21T22:19:06.608 INFO:tasks.rados.rados.0.trial178.stdout:654: finishing write tid 2 to trial17818531-43 2026-02-21T22:19:06.608 INFO:tasks.rados.rados.0.trial178.stdout:655: seq_num 234 ranges {732838=698856,2026904=465743,3210039=571651} 2026-02-21T22:19:06.617 INFO:tasks.rados.rados.0.trial178.stdout:655: writing trial17818531-15 from 732838 to 1431694 tid 1 2026-02-21T22:19:06.624 INFO:tasks.rados.rados.0.trial178.stdout:655: writing trial17818531-15 from 2026904 to 2492647 tid 2 2026-02-21T22:19:06.633 INFO:tasks.rados.rados.0.trial178.stdout:655: writing trial17818531-15 from 3210039 to 3781690 tid 3 2026-02-21T22:19:06.633 INFO:tasks.rados.rados.0.trial178.stdout:656: write (excl) oid 9 current snap is 72 2026-02-21T22:19:06.633 INFO:tasks.rados.rados.0.trial178.stdout:656: seq_num 235 ranges {749645=774105,2085386=609401,3237462=712183} 2026-02-21T22:19:06.638 INFO:tasks.rados.rados.0.trial178.stdout:656: writing trial17818531-9 from 749645 to 1523750 tid 1 2026-02-21T22:19:06.642 INFO:tasks.rados.rados.0.trial178.stdout:656: writing trial17818531-9 from 2085386 to 2694787 tid 2 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:656: writing trial17818531-9 from 3237462 to 3949645 tid 3 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:657: delete oid 13 current snap is 72 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:654: finishing write tid 3 to trial17818531-43 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:654: finishing write tid 4 to trial17818531-43 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:654: finishing write tid 5 to trial17818531-43 2026-02-21T22:19:06.648 INFO:tasks.rados.rados.0.trial178.stdout:654: oid 43 updating version 0 to 82 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:654: oid 43 updating version 82 to 85 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:654: oid 43 version 85 is already newer than 84 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:654: oid 43 version 85 is already newer than 83 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 85 (ObjNum 233 snap 72 seq_num 233) dirty exists 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:654: left oid 43 (ObjNum 233 snap 72 seq_num 233) 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: finishing write tid 1 to trial17818531-15 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: finishing write tid 2 to trial17818531-15 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: finishing write tid 3 to trial17818531-15 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: finishing write tid 4 to trial17818531-15 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: finishing write tid 5 to trial17818531-15 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: oid 15 updating version 0 to 219 2026-02-21T22:19:06.649 INFO:tasks.rados.rados.0.trial178.stdout:655: oid 15 updating version 219 to 222 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:655: oid 15 version 222 is already newer than 221 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:655: oid 15 version 222 is already newer than 220 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 222 (ObjNum 234 snap 72 seq_num 234) dirty exists 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:655: left oid 15 (ObjNum 234 snap 72 seq_num 234) 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:656: finishing write tid 1 to trial17818531-9 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:656: finishing write tid 2 to trial17818531-9 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:654: done (3 left) 2026-02-21T22:19:06.650 INFO:tasks.rados.rados.0.trial178.stdout:655: done (2 left) 2026-02-21T22:19:06.651 INFO:tasks.rados.rados.0.trial178.stdout:657: done (1 left) 2026-02-21T22:19:06.651 INFO:tasks.rados.rados.0.trial178.stdout:658: snap_remove snap 71 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:658: done (1 left) 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:659: snap_create 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: finishing write tid 3 to trial17818531-9 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: finishing write tid 4 to trial17818531-9 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: finishing write tid 5 to trial17818531-9 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: oid 9 updating version 0 to 303 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: oid 9 updating version 303 to 305 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: oid 9 updating version 305 to 306 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:656: oid 9 version 306 is already newer than 304 2026-02-21T22:19:07.398 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 306 (ObjNum 235 snap 72 seq_num 235) dirty exists 2026-02-21T22:19:07.399 INFO:tasks.rados.rados.0.trial178.stdout:656: left oid 9 (ObjNum 235 snap 72 seq_num 235) 2026-02-21T22:19:08.401 INFO:tasks.rados.rados.0.trial178.stdout:656: done (1 left) 2026-02-21T22:19:08.401 INFO:tasks.rados.rados.0.trial178.stdout:659: done (0 left) 2026-02-21T22:19:08.401 INFO:tasks.rados.rados.0.trial178.stdout:660: read oid 3 snap -1 2026-02-21T22:19:08.402 INFO:tasks.rados.rados.0.trial178.stdout:660: expect (ObjNum 195 snap 61 seq_num 195) 2026-02-21T22:19:08.402 INFO:tasks.rados.rados.0.trial178.stdout:661: copy_from oid 40 from oid 19 current snap is 73 2026-02-21T22:19:08.402 INFO:tasks.rados.rados.0.trial178.stdout:662: copy_from oid 1 from oid 32 current snap is 73 2026-02-21T22:19:08.402 INFO:tasks.rados.rados.0.trial178.stdout:663: delete oid 24 current snap is 73 2026-02-21T22:19:08.405 INFO:tasks.rados.rados.0.trial178.stdout:663: done (3 left) 2026-02-21T22:19:08.405 INFO:tasks.rados.rados.0.trial178.stdout:664: rollback oid 5 current snap is 73 2026-02-21T22:19:08.405 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 72 2026-02-21T22:19:08.405 INFO:tasks.rados.rados.0.trial178.stdout:665: write oid 36 current snap is 73 2026-02-21T22:19:08.405 INFO:tasks.rados.rados.0.trial178.stdout:665: seq_num 238 ranges {400066=716205,1860832=139234} 2026-02-21T22:19:08.412 INFO:tasks.rados.rados.0.trial178.stdout:665: writing trial17818531-36 from 400066 to 1116271 tid 1 2026-02-21T22:19:08.418 INFO:tasks.rados.rados.0.trial178.stdout:665: writing trial17818531-36 from 1860832 to 2000066 tid 2 2026-02-21T22:19:08.419 INFO:tasks.rados.rados.0.trial178.stdout:666: snap_create 2026-02-21T22:19:08.419 INFO:tasks.rados.rados.0.trial178.stdout:660: expect (ObjNum 195 snap 61 seq_num 195) 2026-02-21T22:19:08.479 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1721609 2026-02-21T22:19:08.505 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1114262 2026-02-21T22:19:08.531 INFO:tasks.rados.rados.0.trial178.stdout:664: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:19:08.531 INFO:tasks.rados.rados.0.trial178.stdout:664: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:19:08.531 INFO:tasks.rados.rados.0.trial178.stdout:664: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:19:08.531 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 211 (ObjNum 170 snap 48 seq_num 170) dirty exists 2026-02-21T22:19:08.531 INFO:tasks.rados.rados.0.trial178.stdout:665: finishing write tid 1 to trial17818531-36 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:661: finishing copy_from racing read to trial17818531-40 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: finishing write tid 2 to trial17818531-36 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: finishing write tid 3 to trial17818531-36 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: finishing write tid 4 to trial17818531-36 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: oid 36 updating version 0 to 80 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: oid 36 version 80 is already newer than 79 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: oid 36 updating version 80 to 81 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 81 (ObjNum 238 snap 73 seq_num 238) dirty exists 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:665: left oid 36 (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:661: finishing copy_from to trial17818531-40 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 309 (ObjNum 198 snap 62 seq_num 198) dirty exists 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:662: finishing copy_from racing read to trial17818531-1 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:662: finishing copy_from to trial17818531-1 2026-02-21T22:19:08.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 354 (ObjNum 225 snap 71 seq_num 225) dirty exists 2026-02-21T22:19:08.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:08.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:660: done (5 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:661: done (4 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:662: done (3 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:664: done (2 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:665: done (1 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:666: done (0 left) 2026-02-21T22:19:09.412 INFO:tasks.rados.rados.0.trial178.stdout:667: snap_remove snap 73 2026-02-21T22:19:10.414 INFO:tasks.rados.rados.0.trial178.stdout:667: done (0 left) 2026-02-21T22:19:10.414 INFO:tasks.rados.rados.0.trial178.stdout:668: rollback oid 11 current snap is 74 2026-02-21T22:19:10.414 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 72 2026-02-21T22:19:10.414 INFO:tasks.rados.rados.0.trial178.stdout:669: snap_create 2026-02-21T22:19:10.427 INFO:tasks.rados.rados.0.trial178.stdout:668: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:19:10.427 INFO:tasks.rados.rados.0.trial178.stdout:668: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:19:10.427 INFO:tasks.rados.rados.0.trial178.stdout:668: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:19:10.427 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 375 (ObjNum 229 snap 71 seq_num 229) dirty exists 2026-02-21T22:19:11.417 INFO:tasks.rados.rados.0.trial178.stdout:668: done (1 left) 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:669: done (0 left) 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:670: rollback oid 17 current snap is 75 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 74 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:671: copy_from oid 7 from oid 50 current snap is 75 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:672: write oid 31 current snap is 75 2026-02-21T22:19:11.418 INFO:tasks.rados.rados.0.trial178.stdout:672: seq_num 239 ranges {416873=791454,1919314=97559} 2026-02-21T22:19:11.426 INFO:tasks.rados.rados.0.trial178.stdout:672: writing trial17818531-31 from 416873 to 1208327 tid 1 2026-02-21T22:19:11.431 INFO:tasks.rados.rados.0.trial178.stdout:672: writing trial17818531-31 from 1919314 to 2016873 tid 2 2026-02-21T22:19:11.432 INFO:tasks.rados.rados.0.trial178.stdout:673: snap_remove snap 72 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:673: done (3 left) 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:674: copy_from oid 35 from oid 29 current snap is 75 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:670: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 0 (ObjNum 321 snap 0 seq_num 1453350989) dirty dne 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:671: finishing copy_from racing read to trial17818531-7 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:671: finishing copy_from to trial17818531-7 2026-02-21T22:19:12.399 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 72 (ObjNum 178 snap 51 seq_num 178) dirty exists 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: finishing write tid 1 to trial17818531-31 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: finishing write tid 2 to trial17818531-31 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: finishing write tid 3 to trial17818531-31 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: finishing write tid 4 to trial17818531-31 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: oid 31 updating version 0 to 230 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: oid 31 updating version 230 to 232 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: oid 31 version 232 is already newer than 231 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 232 (ObjNum 239 snap 75 seq_num 239) dirty exists 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: left oid 31 (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:670: done (3 left) 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:671: done (2 left) 2026-02-21T22:19:12.400 INFO:tasks.rados.rados.0.trial178.stdout:672: done (1 left) 2026-02-21T22:19:12.401 INFO:tasks.rados.rados.0.trial178.stdout:675: copy_from oid 6 from oid 14 current snap is 75 2026-02-21T22:19:12.401 INFO:tasks.rados.rados.0.trial178.stdout:676: read oid 43 snap 74 2026-02-21T22:19:12.401 INFO:tasks.rados.rados.0.trial178.stdout:676: expect (ObjNum 233 snap 72 seq_num 233) 2026-02-21T22:19:12.401 INFO:tasks.rados.rados.0.trial178.stdout:677: write oid 34 current snap is 75 2026-02-21T22:19:12.401 INFO:tasks.rados.rados.0.trial178.stdout:677: seq_num 240 ranges {433680=466703,1577796=455884} 2026-02-21T22:19:12.405 INFO:tasks.rados.rados.0.trial178.stdout:677: writing trial17818531-34 from 433680 to 900383 tid 1 2026-02-21T22:19:12.413 INFO:tasks.rados.rados.0.trial178.stdout:677: writing trial17818531-34 from 1577796 to 2033680 tid 2 2026-02-21T22:19:12.413 INFO:tasks.rados.rados.0.trial178.stdout:678: read oid 7 snap -1 2026-02-21T22:19:12.413 INFO:tasks.rados.rados.0.trial178.stdout:678: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:12.414 INFO:tasks.rados.rados.0.trial178.stdout:679: delete oid 18 current snap is 75 2026-02-21T22:19:12.414 INFO:tasks.rados.rados.0.trial178.stdout:676: expect (ObjNum 233 snap 72 seq_num 233) 2026-02-21T22:19:12.500 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3787341 2026-02-21T22:19:12.501 INFO:tasks.rados.rados.0.trial178.stdout:677: finishing write tid 1 to trial17818531-34 2026-02-21T22:19:12.501 INFO:tasks.rados.rados.0.trial178.stdout:675: finishing copy_from racing read to trial17818531-6 2026-02-21T22:19:12.501 INFO:tasks.rados.rados.0.trial178.stdout:674: finishing copy_from racing read to trial17818531-35 2026-02-21T22:19:12.501 INFO:tasks.rados.rados.0.trial178.stdout:676: done (5 left) 2026-02-21T22:19:12.502 INFO:tasks.rados.rados.0.trial178.stdout:679: done (4 left) 2026-02-21T22:19:12.502 INFO:tasks.rados.rados.0.trial178.stdout:680: read oid 36 snap 74 2026-02-21T22:19:12.502 INFO:tasks.rados.rados.0.trial178.stdout:680: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:12.502 INFO:tasks.rados.rados.0.trial178.stdout:681: write oid 23 current snap is 75 2026-02-21T22:19:12.502 INFO:tasks.rados.rados.0.trial178.stdout:681: seq_num 242 ranges {467294=617201,1578407=488887} 2026-02-21T22:19:12.507 INFO:tasks.rados.rados.0.trial178.stdout:681: writing trial17818531-23 from 467294 to 1084495 tid 1 2026-02-21T22:19:12.515 INFO:tasks.rados.rados.0.trial178.stdout:681: writing trial17818531-23 from 1578407 to 2067294 tid 2 2026-02-21T22:19:12.515 INFO:tasks.rados.rados.0.trial178.stdout:682: read oid 50 snap -1 2026-02-21T22:19:12.515 INFO:tasks.rados.rados.0.trial178.stdout:682: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:12.515 INFO:tasks.rados.rados.0.trial178.stdout:683: write (excl) oid 9 current snap is 75 2026-02-21T22:19:12.515 INFO:tasks.rados.rados.0.trial178.stdout:683: seq_num 243 ranges {484101=692450,1636889=447212} 2026-02-21T22:19:12.522 INFO:tasks.rados.rados.0.trial178.stdout:683: writing trial17818531-9 from 484101 to 1176551 tid 1 2026-02-21T22:19:12.528 INFO:tasks.rados.rados.0.trial178.stdout:683: writing trial17818531-9 from 1636889 to 2084101 tid 2 2026-02-21T22:19:12.528 INFO:tasks.rados.rados.0.trial178.stdout:684: read oid 30 snap 74 2026-02-21T22:19:12.528 INFO:tasks.rados.rados.0.trial178.stdout:684: expect deleted 2026-02-21T22:19:12.528 INFO:tasks.rados.rados.0.trial178.stdout:685: write (excl) oid 45 current snap is 75 2026-02-21T22:19:12.528 INFO:tasks.rados.rados.0.trial178.stdout:685: seq_num 244 ranges {500908=484052,1411724=689184} 2026-02-21T22:19:12.535 INFO:tasks.rados.rados.0.trial178.stdout:685: writing trial17818531-45 from 500908 to 984960 tid 1 2026-02-21T22:19:12.543 INFO:tasks.rados.rados.0.trial178.stdout:685: writing trial17818531-45 from 1411724 to 2100908 tid 2 2026-02-21T22:19:12.543 INFO:tasks.rados.rados.0.trial178.stdout:686: write oid 11 current snap is 75 2026-02-21T22:19:12.543 INFO:tasks.rados.rados.0.trial178.stdout:686: seq_num 245 ranges {517715=559301,1870206=247509} 2026-02-21T22:19:12.547 INFO:tasks.rados.rados.0.trial178.stdout:686: writing trial17818531-11 from 517715 to 1077016 tid 1 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:686: writing trial17818531-11 from 1870206 to 2117715 tid 2 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:687: read oid 25 snap -1 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:687: expect deleted 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:688: delete oid 44 current snap is 75 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:674: finishing copy_from to trial17818531-35 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:675: finishing copy_from to trial17818531-6 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 302 (ObjNum 226 snap 71 seq_num 226) dirty exists 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:677: finishing write tid 2 to trial17818531-34 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:677: finishing write tid 3 to trial17818531-34 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:677: finishing write tid 4 to trial17818531-34 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:677: oid 34 updating version 0 to 91 2026-02-21T22:19:12.551 INFO:tasks.rados.rados.0.trial178.stdout:677: oid 34 updating version 91 to 93 2026-02-21T22:19:12.552 INFO:tasks.rados.rados.0.trial178.stdout:677: oid 34 version 93 is already newer than 92 2026-02-21T22:19:12.552 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 93 (ObjNum 240 snap 75 seq_num 240) dirty exists 2026-02-21T22:19:12.552 INFO:tasks.rados.rados.0.trial178.stdout:677: left oid 34 (ObjNum 240 snap 75 seq_num 240) 2026-02-21T22:19:12.552 INFO:tasks.rados.rados.0.trial178.stdout:678: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:12.641 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:12.680 INFO:tasks.rados.rados.0.trial178.stdout:680: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:12.736 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:12.768 INFO:tasks.rados.rados.0.trial178.stdout:681: finishing write tid 1 to trial17818531-23 2026-02-21T22:19:12.768 INFO:tasks.rados.rados.0.trial178.stdout:681: finishing write tid 2 to trial17818531-23 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:681: finishing write tid 3 to trial17818531-23 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:674: done (12 left) 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:675: done (11 left) 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:677: done (10 left) 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:678: done (9 left) 2026-02-21T22:19:12.769 INFO:tasks.rados.rados.0.trial178.stdout:680: done (8 left) 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:688: done (7 left) 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:689: delete oid 12 current snap is 75 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:681: finishing write tid 4 to trial17818531-23 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:681: oid 23 updating version 0 to 167 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:681: oid 23 version 167 is already newer than 166 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:681: oid 23 updating version 167 to 168 2026-02-21T22:19:12.770 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 168 (ObjNum 242 snap 75 seq_num 242) dirty exists 2026-02-21T22:19:12.771 INFO:tasks.rados.rados.0.trial178.stdout:681: left oid 23 (ObjNum 242 snap 75 seq_num 242) 2026-02-21T22:19:12.771 INFO:tasks.rados.rados.0.trial178.stdout:682: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:12.888 INFO:tasks.rados.rados.0.trial178.stdout:683: finishing write tid 1 to trial17818531-9 2026-02-21T22:19:12.888 INFO:tasks.rados.rados.0.trial178.stdout:683: finishing write tid 2 to trial17818531-9 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:683: finishing write tid 3 to trial17818531-9 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:681: done (7 left) 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:682: done (6 left) 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:684: done (5 left) 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:689: done (4 left) 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:690: write oid 41 current snap is 75 2026-02-21T22:19:12.889 INFO:tasks.rados.rados.0.trial178.stdout:690: seq_num 248 ranges {568136=785048,1929299=238837} 2026-02-21T22:19:12.897 INFO:tasks.rados.rados.0.trial178.stdout:690: writing trial17818531-41 from 568136 to 1353184 tid 1 2026-02-21T22:19:12.903 INFO:tasks.rados.rados.0.trial178.stdout:690: writing trial17818531-41 from 1929299 to 2168136 tid 2 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:691: snap_remove snap 74 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:683: finishing write tid 4 to trial17818531-9 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:683: oid 9 updating version 0 to 308 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:683: oid 9 version 308 is already newer than 307 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:683: oid 9 updating version 308 to 309 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 309 (ObjNum 243 snap 75 seq_num 243) dirty exists 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:683: left oid 9 (ObjNum 243 snap 75 seq_num 243) 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:685: finishing write tid 1 to trial17818531-45 2026-02-21T22:19:12.904 INFO:tasks.rados.rados.0.trial178.stdout:685: finishing write tid 2 to trial17818531-45 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: finishing write tid 3 to trial17818531-45 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: finishing write tid 4 to trial17818531-45 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: oid 45 updating version 0 to 356 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: oid 45 version 356 is already newer than 355 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: oid 45 updating version 356 to 357 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 357 (ObjNum 244 snap 75 seq_num 244) dirty exists 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:685: left oid 45 (ObjNum 244 snap 75 seq_num 244) 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: finishing write tid 1 to trial17818531-11 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: finishing write tid 2 to trial17818531-11 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: finishing write tid 3 to trial17818531-11 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: finishing write tid 4 to trial17818531-11 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: oid 11 updating version 0 to 382 2026-02-21T22:19:12.905 INFO:tasks.rados.rados.0.trial178.stdout:686: oid 11 version 382 is already newer than 381 2026-02-21T22:19:12.906 INFO:tasks.rados.rados.0.trial178.stdout:686: oid 11 version 382 is already newer than 380 2026-02-21T22:19:12.906 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 382 (ObjNum 245 snap 75 seq_num 245) dirty exists 2026-02-21T22:19:12.906 INFO:tasks.rados.rados.0.trial178.stdout:686: left oid 11 (ObjNum 245 snap 75 seq_num 245) 2026-02-21T22:19:12.906 INFO:tasks.rados.rados.0.trial178.stdout:690: finishing write tid 1 to trial17818531-41 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:683: done (5 left) 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:685: done (4 left) 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:686: done (3 left) 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:687: done (2 left) 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:691: done (1 left) 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:692: write oid 20 current snap is 75 2026-02-21T22:19:13.424 INFO:tasks.rados.rados.0.trial178.stdout:692: seq_num 249 ranges {584943=460297,1587781=597162} 2026-02-21T22:19:13.430 INFO:tasks.rados.rados.0.trial178.stdout:692: writing trial17818531-20 from 584943 to 1045240 tid 1 2026-02-21T22:19:13.439 INFO:tasks.rados.rados.0.trial178.stdout:692: writing trial17818531-20 from 1587781 to 2184943 tid 2 2026-02-21T22:19:13.439 INFO:tasks.rados.rados.0.trial178.stdout:693: write (excl) oid 32 current snap is 75 2026-02-21T22:19:13.439 INFO:tasks.rados.rados.0.trial178.stdout:690: finishing write tid 2 to trial17818531-41 2026-02-21T22:19:13.439 INFO:tasks.rados.rados.0.trial178.stdout:690: finishing write tid 3 to trial17818531-41 2026-02-21T22:19:13.439 INFO:tasks.rados.rados.0.trial178.stdout:693: seq_num 250 ranges {601750=535546,1929910=271840} 2026-02-21T22:19:13.446 INFO:tasks.rados.rados.0.trial178.stdout:693: writing trial17818531-32 from 601750 to 1137296 tid 1 2026-02-21T22:19:13.454 INFO:tasks.rados.rados.0.trial178.stdout:693: writing trial17818531-32 from 1929910 to 2201750 tid 2 2026-02-21T22:19:13.454 INFO:tasks.rados.rados.0.trial178.stdout:694: write (excl) oid 13 current snap is 75 2026-02-21T22:19:13.454 INFO:tasks.rados.rados.0.trial178.stdout:694: seq_num 251 ranges {618557=727148,2104745=113812} 2026-02-21T22:19:13.463 INFO:tasks.rados.rados.0.trial178.stdout:694: writing trial17818531-13 from 618557 to 1345705 tid 1 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:694: writing trial17818531-13 from 2104745 to 2218557 tid 2 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:695: rollback oid 45 current snap is 75 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:695: done (4 left) 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:696: read oid 35 snap -1 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:696: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:697: rollback oid 29 current snap is 75 2026-02-21T22:19:13.471 INFO:tasks.rados.rados.0.trial178.stdout:697: done (5 left) 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:698: delete oid 14 current snap is 75 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:690: finishing write tid 4 to trial17818531-41 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:690: oid 41 updating version 0 to 225 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:690: oid 41 updating version 225 to 227 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:690: oid 41 version 227 is already newer than 226 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 227 (ObjNum 248 snap 75 seq_num 248) dirty exists 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:690: left oid 41 (ObjNum 248 snap 75 seq_num 248) 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:692: finishing write tid 1 to trial17818531-20 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:692: finishing write tid 2 to trial17818531-20 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:692: finishing write tid 3 to trial17818531-20 2026-02-21T22:19:13.472 INFO:tasks.rados.rados.0.trial178.stdout:692: finishing write tid 4 to trial17818531-20 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:692: oid 20 updating version 0 to 228 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:692: oid 20 updating version 228 to 229 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:692: oid 20 updating version 229 to 230 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 230 (ObjNum 249 snap 75 seq_num 249) dirty exists 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:692: left oid 20 (ObjNum 249 snap 75 seq_num 249) 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: finishing write tid 1 to trial17818531-32 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: finishing write tid 2 to trial17818531-32 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: finishing write tid 3 to trial17818531-32 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: finishing write tid 4 to trial17818531-32 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: oid 32 updating version 0 to 386 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: oid 32 version 386 is already newer than 385 2026-02-21T22:19:13.473 INFO:tasks.rados.rados.0.trial178.stdout:693: oid 32 updating version 386 to 387 2026-02-21T22:19:13.474 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 387 (ObjNum 250 snap 75 seq_num 250) dirty exists 2026-02-21T22:19:13.474 INFO:tasks.rados.rados.0.trial178.stdout:693: left oid 32 (ObjNum 250 snap 75 seq_num 250) 2026-02-21T22:19:13.474 INFO:tasks.rados.rados.0.trial178.stdout:694: finishing write tid 1 to trial17818531-13 2026-02-21T22:19:13.474 INFO:tasks.rados.rados.0.trial178.stdout:694: finishing write tid 2 to trial17818531-13 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:690: done (5 left) 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:692: done (4 left) 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:693: done (3 left) 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:698: done (2 left) 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:699: delete oid 33 current snap is 75 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:694: finishing write tid 3 to trial17818531-13 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:694: finishing write tid 4 to trial17818531-13 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:694: oid 13 updating version 0 to 389 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:694: oid 13 updating version 389 to 390 2026-02-21T22:19:13.475 INFO:tasks.rados.rados.0.trial178.stdout:694: oid 13 version 390 is already newer than 388 2026-02-21T22:19:13.476 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 390 (ObjNum 251 snap 75 seq_num 251) dirty exists 2026-02-21T22:19:13.476 INFO:tasks.rados.rados.0.trial178.stdout:694: left oid 13 (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:13.476 INFO:tasks.rados.rados.0.trial178.stdout:694: done (2 left) 2026-02-21T22:19:13.476 INFO:tasks.rados.rados.0.trial178.stdout:699: done (1 left) 2026-02-21T22:19:13.476 INFO:tasks.rados.rados.0.trial178.stdout:700: delete oid 2 current snap is 75 2026-02-21T22:19:13.479 INFO:tasks.rados.rados.0.trial178.stdout:700: done (1 left) 2026-02-21T22:19:13.479 INFO:tasks.rados.rados.0.trial178.stdout:701: delete oid 15 current snap is 75 2026-02-21T22:19:13.480 INFO:tasks.rados.rados.0.trial178.stdout:696: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:13.518 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2298282 2026-02-21T22:19:13.542 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:13.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:13.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:696: done (1 left) 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:701: done (0 left) 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:702: read oid 13 snap -1 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:702: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:703: write (excl) oid 10 current snap is 75 2026-02-21T22:19:13.634 INFO:tasks.rados.rados.0.trial178.stdout:703: seq_num 256 ranges {702592=703393,1880802=421790} 2026-02-21T22:19:13.644 INFO:tasks.rados.rados.0.trial178.stdout:703: writing trial17818531-10 from 702592 to 1405985 tid 1 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:703: writing trial17818531-10 from 1880802 to 2302592 tid 2 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:704: read oid 19 snap -1 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:704: expect (ObjNum 198 snap 62 seq_num 198) 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:705: read oid 11 snap -1 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:705: expect (ObjNum 245 snap 75 seq_num 245) 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:706: snap_create 2026-02-21T22:19:13.650 INFO:tasks.rados.rados.0.trial178.stdout:702: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:13.706 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: finishing write tid 1 to trial17818531-10 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: finishing write tid 2 to trial17818531-10 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: finishing write tid 3 to trial17818531-10 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: finishing write tid 4 to trial17818531-10 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: oid 10 updating version 0 to 306 2026-02-21T22:19:13.723 INFO:tasks.rados.rados.0.trial178.stdout:703: oid 10 version 306 is already newer than 305 2026-02-21T22:19:13.724 INFO:tasks.rados.rados.0.trial178.stdout:703: oid 10 version 306 is already newer than 304 2026-02-21T22:19:13.724 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 306 (ObjNum 256 snap 75 seq_num 256) dirty exists 2026-02-21T22:19:13.724 INFO:tasks.rados.rados.0.trial178.stdout:703: left oid 10 (ObjNum 256 snap 75 seq_num 256) 2026-02-21T22:19:13.724 INFO:tasks.rados.rados.0.trial178.stdout:705: expect (ObjNum 245 snap 75 seq_num 245) 2026-02-21T22:19:13.831 INFO:tasks.rados.rados.0.trial178.stdout:704: expect (ObjNum 198 snap 62 seq_num 198) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:702: done (4 left) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:703: done (3 left) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:704: done (2 left) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:705: done (1 left) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:706: done (0 left) 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:707: copy_from oid 25 from oid 35 current snap is 76 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:708: write oid 27 current snap is 76 2026-02-21T22:19:14.426 INFO:tasks.rados.rados.0.trial178.stdout:708: seq_num 257 ranges {719399=778642,1939284=380115} 2026-02-21T22:19:14.436 INFO:tasks.rados.rados.0.trial178.stdout:708: writing trial17818531-27 from 719399 to 1498041 tid 1 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:708: writing trial17818531-27 from 1939284 to 2319399 tid 2 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:709: write oid 16 current snap is 76 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:707: finishing copy_from racing read to trial17818531-25 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:707: finishing copy_from to trial17818531-25 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:14.442 INFO:tasks.rados.rados.0.trial178.stdout:709: seq_num 258 ranges {736206=453891,1881413=454793} 2026-02-21T22:19:14.450 INFO:tasks.rados.rados.0.trial178.stdout:709: writing trial17818531-16 from 736206 to 1190097 tid 1 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:709: writing trial17818531-16 from 1881413 to 2336206 tid 2 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:707: done (2 left) 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:710: write oid 29 current snap is 76 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: finishing write tid 1 to trial17818531-27 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: finishing write tid 2 to trial17818531-27 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: finishing write tid 3 to trial17818531-27 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: finishing write tid 4 to trial17818531-27 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: oid 27 updating version 0 to 312 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: oid 27 version 312 is already newer than 311 2026-02-21T22:19:14.458 INFO:tasks.rados.rados.0.trial178.stdout:708: oid 27 updating version 312 to 313 2026-02-21T22:19:14.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 313 (ObjNum 257 snap 76 seq_num 257) dirty exists 2026-02-21T22:19:14.459 INFO:tasks.rados.rados.0.trial178.stdout:708: left oid 27 (ObjNum 257 snap 76 seq_num 257) 2026-02-21T22:19:14.459 INFO:tasks.rados.rados.0.trial178.stdout:709: finishing write tid 1 to trial17818531-16 2026-02-21T22:19:14.459 INFO:tasks.rados.rados.0.trial178.stdout:710: seq_num 259 ranges {753013=645493,2056248=296765} 2026-02-21T22:19:14.468 INFO:tasks.rados.rados.0.trial178.stdout:710: writing trial17818531-29 from 753013 to 1398506 tid 1 2026-02-21T22:19:14.474 INFO:tasks.rados.rados.0.trial178.stdout:710: writing trial17818531-29 from 2056248 to 2353013 tid 2 2026-02-21T22:19:14.474 INFO:tasks.rados.rados.0.trial178.stdout:708: done (2 left) 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:711: snap_create 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: finishing write tid 2 to trial17818531-16 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: finishing write tid 3 to trial17818531-16 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: finishing write tid 4 to trial17818531-16 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: oid 16 updating version 0 to 147 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: oid 16 updating version 147 to 149 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: oid 16 version 149 is already newer than 148 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 149 (ObjNum 258 snap 76 seq_num 258) dirty exists 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:709: left oid 16 (ObjNum 258 snap 76 seq_num 258) 2026-02-21T22:19:14.475 INFO:tasks.rados.rados.0.trial178.stdout:710: finishing write tid 1 to trial17818531-29 2026-02-21T22:19:14.476 INFO:tasks.rados.rados.0.trial178.stdout:710: finishing write tid 2 to trial17818531-29 2026-02-21T22:19:14.476 INFO:tasks.rados.rados.0.trial178.stdout:710: finishing write tid 3 to trial17818531-29 2026-02-21T22:19:14.476 INFO:tasks.rados.rados.0.trial178.stdout:710: finishing write tid 4 to trial17818531-29 2026-02-21T22:19:14.477 INFO:tasks.rados.rados.0.trial178.stdout:710: oid 29 updating version 0 to 97 2026-02-21T22:19:14.477 INFO:tasks.rados.rados.0.trial178.stdout:710: oid 29 updating version 97 to 98 2026-02-21T22:19:14.477 INFO:tasks.rados.rados.0.trial178.stdout:710: oid 29 updating version 98 to 99 2026-02-21T22:19:14.477 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 99 (ObjNum 259 snap 76 seq_num 259) dirty exists 2026-02-21T22:19:14.477 INFO:tasks.rados.rados.0.trial178.stdout:710: left oid 29 (ObjNum 259 snap 76 seq_num 259) 2026-02-21T22:19:15.429 INFO:tasks.rados.rados.0.trial178.stdout:709: done (2 left) 2026-02-21T22:19:15.430 INFO:tasks.rados.rados.0.trial178.stdout:710: done (1 left) 2026-02-21T22:19:15.430 INFO:tasks.rados.rados.0.trial178.stdout:711: done (0 left) 2026-02-21T22:19:15.430 INFO:tasks.rados.rados.0.trial178.stdout:712: write oid 42 current snap is 77 2026-02-21T22:19:15.430 INFO:tasks.rados.rados.0.trial178.stdout:712: seq_num 260 ranges {769820=720742,2114730=255090} 2026-02-21T22:19:15.439 INFO:tasks.rados.rados.0.trial178.stdout:712: writing trial17818531-42 from 769820 to 1490562 tid 1 2026-02-21T22:19:15.445 INFO:tasks.rados.rados.0.trial178.stdout:712: writing trial17818531-42 from 2114730 to 2369820 tid 2 2026-02-21T22:19:15.445 INFO:tasks.rados.rados.0.trial178.stdout:713: snap_remove snap 75 2026-02-21T22:19:16.434 INFO:tasks.rados.rados.0.trial178.stdout:713: done (1 left) 2026-02-21T22:19:16.434 INFO:tasks.rados.rados.0.trial178.stdout:714: rollback oid 28 current snap is 77 2026-02-21T22:19:16.434 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 76 2026-02-21T22:19:16.434 INFO:tasks.rados.rados.0.trial178.stdout:712: finishing write tid 1 to trial17818531-42 2026-02-21T22:19:16.434 INFO:tasks.rados.rados.0.trial178.stdout:712: finishing write tid 2 to trial17818531-42 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: finishing write tid 3 to trial17818531-42 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: finishing write tid 4 to trial17818531-42 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: oid 42 updating version 0 to 361 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: oid 42 updating version 361 to 363 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: oid 42 version 363 is already newer than 362 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 363 (ObjNum 260 snap 77 seq_num 260) dirty exists 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: left oid 42 (ObjNum 260 snap 77 seq_num 260) 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:712: done (1 left) 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:715: read oid 44 snap -1 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:715: expect deleted 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:716: read oid 34 snap -1 2026-02-21T22:19:16.435 INFO:tasks.rados.rados.0.trial178.stdout:716: expect (ObjNum 240 snap 75 seq_num 240) 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:717: read oid 38 snap -1 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:717: expect deleted 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:718: copy_from oid 43 from oid 48 current snap is 77 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:719: snap_create 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:714: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:19:16.436 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 0 (ObjNum 129 snap 0 seq_num 100990112) dirty dne 2026-02-21T22:19:16.439 INFO:tasks.rados.rados.0.trial178.stdout:716: expect (ObjNum 240 snap 75 seq_num 240) 2026-02-21T22:19:16.491 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 515713 2026-02-21T22:19:16.505 INFO:tasks.rados.rados.0.trial178.stdout:718: finishing copy_from racing read to trial17818531-43 2026-02-21T22:19:16.505 INFO:tasks.rados.rados.0.trial178.stdout:718: finishing copy_from to trial17818531-43 2026-02-21T22:19:16.505 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:714: done (5 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:715: done (4 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:716: done (3 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:717: done (2 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:718: done (1 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:719: done (0 left) 2026-02-21T22:19:17.399 INFO:tasks.rados.rados.0.trial178.stdout:720: snap_create 2026-02-21T22:19:18.402 INFO:tasks.rados.rados.0.trial178.stdout:720: done (0 left) 2026-02-21T22:19:18.403 INFO:tasks.rados.rados.0.trial178.stdout:721: copy_from oid 41 from oid 37 current snap is 79 2026-02-21T22:19:18.403 INFO:tasks.rados.rados.0.trial178.stdout:722: snap_create 2026-02-21T22:19:18.415 INFO:tasks.rados.rados.0.trial178.stdout:721: finishing copy_from racing read to trial17818531-41 2026-02-21T22:19:18.416 INFO:tasks.rados.rados.0.trial178.stdout:721: finishing copy_from to trial17818531-41 2026-02-21T22:19:18.416 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 52 (ObjNum 131 snap 31 seq_num 131) dirty exists 2026-02-21T22:19:18.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:18.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:19.403 INFO:tasks.rados.rados.0.trial178.stdout:721: done (1 left) 2026-02-21T22:19:19.404 INFO:tasks.rados.rados.0.trial178.stdout:722: done (0 left) 2026-02-21T22:19:19.404 INFO:tasks.rados.rados.0.trial178.stdout:723: read oid 39 snap -1 2026-02-21T22:19:19.404 INFO:tasks.rados.rados.0.trial178.stdout:723: expect (ObjNum 165 snap 47 seq_num 165) 2026-02-21T22:19:19.404 INFO:tasks.rados.rados.0.trial178.stdout:724: write oid 19 current snap is 80 2026-02-21T22:19:19.404 INFO:tasks.rados.rados.0.trial178.stdout:724: seq_num 261 ranges {786627=795991,2173212=213415} 2026-02-21T22:19:19.414 INFO:tasks.rados.rados.0.trial178.stdout:724: writing trial17818531-19 from 786627 to 1582618 tid 1 2026-02-21T22:19:19.419 INFO:tasks.rados.rados.0.trial178.stdout:724: writing trial17818531-19 from 2173212 to 2386627 tid 2 2026-02-21T22:19:19.420 INFO:tasks.rados.rados.0.trial178.stdout:725: snap_create 2026-02-21T22:19:19.466 INFO:tasks.rados.rados.0.trial178.stdout:724: finishing write tid 1 to trial17818531-19 2026-02-21T22:19:19.471 INFO:tasks.rados.rados.0.trial178.stdout:724: finishing write tid 2 to trial17818531-19 2026-02-21T22:19:19.472 INFO:tasks.rados.rados.0.trial178.stdout:724: finishing write tid 3 to trial17818531-19 2026-02-21T22:19:19.472 INFO:tasks.rados.rados.0.trial178.stdout:723: expect (ObjNum 165 snap 47 seq_num 165) 2026-02-21T22:19:19.472 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:19.510 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:19.542 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2076206 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:724: finishing write tid 4 to trial17818531-19 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:724: oid 19 updating version 0 to 367 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:724: oid 19 updating version 367 to 368 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:724: oid 19 updating version 368 to 369 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 369 (ObjNum 261 snap 80 seq_num 261) dirty exists 2026-02-21T22:19:19.548 INFO:tasks.rados.rados.0.trial178.stdout:724: left oid 19 (ObjNum 261 snap 80 seq_num 261) 2026-02-21T22:19:20.405 INFO:tasks.rados.rados.0.trial178.stdout:723: done (2 left) 2026-02-21T22:19:20.405 INFO:tasks.rados.rados.0.trial178.stdout:724: done (1 left) 2026-02-21T22:19:20.405 INFO:tasks.rados.rados.0.trial178.stdout:725: done (0 left) 2026-02-21T22:19:20.405 INFO:tasks.rados.rados.0.trial178.stdout:726: snap_remove snap 79 2026-02-21T22:19:21.410 INFO:tasks.rados.rados.0.trial178.stdout:726: done (0 left) 2026-02-21T22:19:21.411 INFO:tasks.rados.rados.0.trial178.stdout:727: read oid 23 snap -1 2026-02-21T22:19:21.411 INFO:tasks.rados.rados.0.trial178.stdout:727: expect (ObjNum 242 snap 75 seq_num 242) 2026-02-21T22:19:21.411 INFO:tasks.rados.rados.0.trial178.stdout:728: write (excl) oid 37 current snap is 81 2026-02-21T22:19:21.411 INFO:tasks.rados.rados.0.trial178.stdout:728: seq_num 262 ranges {403434=471240,1315341=400756} 2026-02-21T22:19:21.416 INFO:tasks.rados.rados.0.trial178.stdout:728: writing trial17818531-37 from 403434 to 874674 tid 1 2026-02-21T22:19:21.422 INFO:tasks.rados.rados.0.trial178.stdout:728: writing trial17818531-37 from 1315341 to 1716097 tid 2 2026-02-21T22:19:21.422 INFO:tasks.rados.rados.0.trial178.stdout:729: write oid 28 current snap is 81 2026-02-21T22:19:21.422 INFO:tasks.rados.rados.0.trial178.stdout:729: seq_num 263 ranges {420241=546489,1373823=544414} 2026-02-21T22:19:21.429 INFO:tasks.rados.rados.0.trial178.stdout:729: writing trial17818531-28 from 420241 to 966730 tid 1 2026-02-21T22:19:21.435 INFO:tasks.rados.rados.0.trial178.stdout:729: writing trial17818531-28 from 1373823 to 1918237 tid 2 2026-02-21T22:19:21.435 INFO:tasks.rados.rados.0.trial178.stdout:730: snap_remove snap 80 2026-02-21T22:19:22.399 INFO:tasks.rados.rados.0.trial178.stdout:730: done (3 left) 2026-02-21T22:19:22.399 INFO:tasks.rados.rados.0.trial178.stdout:731: copy_from oid 30 from oid 48 current snap is 81 2026-02-21T22:19:22.399 INFO:tasks.rados.rados.0.trial178.stdout:727: expect (ObjNum 242 snap 75 seq_num 242) 2026-02-21T22:19:22.480 INFO:tasks.rados.rados.0.trial178.stdout:728: finishing write tid 1 to trial17818531-37 2026-02-21T22:19:22.480 INFO:tasks.rados.rados.0.trial178.stdout:728: finishing write tid 2 to trial17818531-37 2026-02-21T22:19:22.481 INFO:tasks.rados.rados.0.trial178.stdout:727: done (3 left) 2026-02-21T22:19:22.481 INFO:tasks.rados.rados.0.trial178.stdout:732: write (excl) oid 21 current snap is 81 2026-02-21T22:19:22.481 INFO:tasks.rados.rados.0.trial178.stdout:732: seq_num 264 ranges {437048=621738,1832305=404425} 2026-02-21T22:19:22.487 INFO:tasks.rados.rados.0.trial178.stdout:732: writing trial17818531-21 from 437048 to 1058786 tid 1 2026-02-21T22:19:22.496 INFO:tasks.rados.rados.0.trial178.stdout:732: writing trial17818531-21 from 1832305 to 2236730 tid 2 2026-02-21T22:19:22.496 INFO:tasks.rados.rados.0.trial178.stdout:733: read oid 22 snap -1 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:733: expect (ObjNum 222 snap 69 seq_num 222) 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:734: read oid 26 snap 78 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:734: expect deleted 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:735: read oid 40 snap -1 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:735: expect (ObjNum 198 snap 62 seq_num 198) 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:736: snap_create 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:728: finishing write tid 3 to trial17818531-37 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:728: finishing write tid 4 to trial17818531-37 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:728: oid 37 updating version 0 to 360 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:728: oid 37 version 360 is already newer than 358 2026-02-21T22:19:22.497 INFO:tasks.rados.rados.0.trial178.stdout:728: oid 37 version 360 is already newer than 359 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 360 (ObjNum 262 snap 81 seq_num 262) dirty exists 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:728: left oid 37 (ObjNum 262 snap 81 seq_num 262) 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: finishing write tid 1 to trial17818531-28 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: finishing write tid 2 to trial17818531-28 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: finishing write tid 3 to trial17818531-28 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: finishing write tid 4 to trial17818531-28 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: oid 28 updating version 0 to 120 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: oid 28 version 120 is already newer than 119 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: oid 28 version 120 is already newer than 118 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 120 (ObjNum 263 snap 81 seq_num 263) dirty exists 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:729: left oid 28 (ObjNum 263 snap 81 seq_num 263) 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:731: finishing copy_from racing read to trial17818531-30 2026-02-21T22:19:22.498 INFO:tasks.rados.rados.0.trial178.stdout:731: finishing copy_from to trial17818531-30 2026-02-21T22:19:22.499 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:22.499 INFO:tasks.rados.rados.0.trial178.stdout:732: finishing write tid 1 to trial17818531-21 2026-02-21T22:19:22.499 INFO:tasks.rados.rados.0.trial178.stdout:732: finishing write tid 2 to trial17818531-21 2026-02-21T22:19:22.499 INFO:tasks.rados.rados.0.trial178.stdout:732: finishing write tid 3 to trial17818531-21 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:732: finishing write tid 4 to trial17818531-21 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:732: oid 21 updating version 0 to 375 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:732: oid 21 version 375 is already newer than 374 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:732: oid 21 version 375 is already newer than 373 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 375 (ObjNum 264 snap 81 seq_num 264) dirty exists 2026-02-21T22:19:22.500 INFO:tasks.rados.rados.0.trial178.stdout:732: left oid 21 (ObjNum 264 snap 81 seq_num 264) 2026-02-21T22:19:22.506 INFO:tasks.rados.rados.0.trial178.stdout:733: expect (ObjNum 222 snap 69 seq_num 222) 2026-02-21T22:19:22.561 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3455529 2026-02-21T22:19:22.566 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:22.616 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:22.666 INFO:tasks.rados.rados.0.trial178.stdout:735: expect (ObjNum 198 snap 62 seq_num 198) 2026-02-21T22:19:22.711 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:23.415 INFO:tasks.rados.rados.0.trial178.stdout:728: done (7 left) 2026-02-21T22:19:23.431 INFO:tasks.rados.rados.0.trial178.stdout:729: done (6 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:731: done (5 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:732: done (4 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:733: done (3 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:734: done (2 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:735: done (1 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:736: done (0 left) 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:737: write (excl) oid 49 current snap is 82 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:737: seq_num 265 ranges {453855=696987,1890787=548083} 2026-02-21T22:19:23.432 INFO:tasks.rados.rados.0.trial178.stdout:737: writing trial17818531-49 from 453855 to 1150842 tid 1 2026-02-21T22:19:23.434 INFO:tasks.rados.rados.0.trial178.stdout:737: writing trial17818531-49 from 1890787 to 2438870 tid 2 2026-02-21T22:19:23.434 INFO:tasks.rados.rados.0.trial178.stdout:738: delete oid 25 current snap is 82 2026-02-21T22:19:23.434 INFO:tasks.rados.rados.0.trial178.stdout:737: finishing write tid 1 to trial17818531-49 2026-02-21T22:19:23.435 INFO:tasks.rados.rados.0.trial178.stdout:737: finishing write tid 2 to trial17818531-49 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: finishing write tid 3 to trial17818531-49 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: finishing write tid 4 to trial17818531-49 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: oid 49 updating version 0 to 235 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: oid 49 version 235 is already newer than 234 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: oid 49 version 235 is already newer than 233 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 235 (ObjNum 265 snap 82 seq_num 265) dirty exists 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: left oid 49 (ObjNum 265 snap 82 seq_num 265) 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:737: done (1 left) 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:738: done (0 left) 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:739: copy_from oid 7 from oid 25 current snap is 82 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:740: write (excl) oid 20 current snap is 82 2026-02-21T22:19:23.437 INFO:tasks.rados.rados.0.trial178.stdout:740: seq_num 267 ranges {487469=563838,1607751=551752} 2026-02-21T22:19:23.444 INFO:tasks.rados.rados.0.trial178.stdout:740: writing trial17818531-20 from 487469 to 1051307 tid 1 2026-02-21T22:19:23.452 INFO:tasks.rados.rados.0.trial178.stdout:740: writing trial17818531-20 from 1607751 to 2159503 tid 2 2026-02-21T22:19:23.452 INFO:tasks.rados.rados.0.trial178.stdout:741: read oid 4 snap 81 2026-02-21T22:19:23.452 INFO:tasks.rados.rados.0.trial178.stdout:741: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:23.452 INFO:tasks.rados.rados.0.trial178.stdout:742: write (excl) oid 37 current snap is 82 2026-02-21T22:19:23.452 INFO:tasks.rados.rados.0.trial178.stdout:742: seq_num 268 ranges {504276=639087,1666233=695410} 2026-02-21T22:19:23.459 INFO:tasks.rados.rados.0.trial178.stdout:742: writing trial17818531-37 from 504276 to 1143363 tid 1 2026-02-21T22:19:23.468 INFO:tasks.rados.rados.0.trial178.stdout:742: writing trial17818531-37 from 1666233 to 2361643 tid 2 2026-02-21T22:19:23.468 INFO:tasks.rados.rados.0.trial178.stdout:743: snap_remove snap 76 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:19:23] 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:19:23, mode upmap, max misplaced 0.050000 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', 'unique_pool_0', '.mgr'] 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:19:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:19:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:19:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:19:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:19:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:19:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:19:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:19:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:19:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:19:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:19:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:19:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:19:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:19:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:19:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "184", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:19:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:19:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:19:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 184, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 84, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:19:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:19:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:19:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:19:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:19:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:19:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:19:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:19:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:19:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0002414703369140625 of space, bias 1.0, pg target 0.07244110107421875 quantized to 32 (current 16) 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:19:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:19:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:743: done (4 left) 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:744: snap_create 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:739: finishing copy_from to trial17818531-7 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:739: got expected ENOENT (src dne) 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:739: finishing copy_from racing read to trial17818531-7 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:740: finishing write tid 1 to trial17818531-20 2026-02-21T22:19:24.734 INFO:tasks.rados.rados.0.trial178.stdout:741: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:24.839 INFO:tasks.rados.rados.0.trial178.stdout:740: finishing write tid 2 to trial17818531-20 2026-02-21T22:19:24.839 INFO:tasks.rados.rados.0.trial178.stdout:740: finishing write tid 3 to trial17818531-20 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:740: finishing write tid 4 to trial17818531-20 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:740: oid 20 updating version 0 to 235 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:740: oid 20 updating version 235 to 236 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:740: oid 20 updating version 236 to 237 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 237 (ObjNum 267 snap 82 seq_num 267) dirty exists 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:740: left oid 20 (ObjNum 267 snap 82 seq_num 267) 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: finishing write tid 1 to trial17818531-37 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: finishing write tid 2 to trial17818531-37 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: finishing write tid 3 to trial17818531-37 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: finishing write tid 4 to trial17818531-37 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: oid 37 updating version 0 to 362 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: oid 37 version 362 is already newer than 361 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: oid 37 updating version 362 to 363 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 363 (ObjNum 268 snap 82 seq_num 268) dirty exists 2026-02-21T22:19:24.840 INFO:tasks.rados.rados.0.trial178.stdout:742: left oid 37 (ObjNum 268 snap 82 seq_num 268) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:739: done (4 left) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:740: done (3 left) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:741: done (2 left) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:742: done (1 left) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:744: done (0 left) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:745: copy_from oid 6 from oid 15 current snap is 83 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:746: read oid 28 snap 81 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:746: expect (ObjNum 263 snap 81 seq_num 263) 2026-02-21T22:19:25.420 INFO:tasks.rados.rados.0.trial178.stdout:747: delete oid 45 current snap is 83 2026-02-21T22:19:25.422 INFO:tasks.rados.rados.0.trial178.stdout:745: finishing copy_from to trial17818531-6 2026-02-21T22:19:25.422 INFO:tasks.rados.rados.0.trial178.stdout:745: got expected ENOENT (src dne) 2026-02-21T22:19:25.422 INFO:tasks.rados.rados.0.trial178.stdout:745: finishing copy_from racing read to trial17818531-6 2026-02-21T22:19:25.427 INFO:tasks.rados.rados.0.trial178.stdout:746: expect (ObjNum 263 snap 81 seq_num 263) 2026-02-21T22:19:25.446 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:25.475 INFO:tasks.rados.rados.0.trial178.stdout:745: done (2 left) 2026-02-21T22:19:25.475 INFO:tasks.rados.rados.0.trial178.stdout:746: done (1 left) 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:747: done (0 left) 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:748: read oid 36 snap -1 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:748: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:749: read oid 7 snap 77 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:749: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:750: rollback oid 13 current snap is 83 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 82 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:751: write (excl) oid 5 current snap is 83 2026-02-21T22:19:25.476 INFO:tasks.rados.rados.0.trial178.stdout:751: seq_num 270 ranges {537890=789585,1783197=699079} 2026-02-21T22:19:25.484 INFO:tasks.rados.rados.0.trial178.stdout:751: writing trial17818531-5 from 537890 to 1327475 tid 1 2026-02-21T22:19:25.492 INFO:tasks.rados.rados.0.trial178.stdout:751: writing trial17818531-5 from 1783197 to 2482276 tid 2 2026-02-21T22:19:25.492 INFO:tasks.rados.rados.0.trial178.stdout:752: snap_create 2026-02-21T22:19:25.492 INFO:tasks.rados.rados.0.trial178.stdout:750: finishing rollback tid 0 to trial17818531-13 2026-02-21T22:19:25.492 INFO:tasks.rados.rados.0.trial178.stdout:750: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:19:25.492 INFO:tasks.rados.rados.0.trial178.stdout:750: finishing rollback tid 2 to trial17818531-13 2026-02-21T22:19:25.493 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 394 (ObjNum 251 snap 75 seq_num 251) dirty exists 2026-02-21T22:19:25.493 INFO:tasks.rados.rados.0.trial178.stdout:748: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:25.605 INFO:tasks.rados.rados.0.trial178.stdout:749: expect (ObjNum 178 snap 51 seq_num 178) 2026-02-21T22:19:25.678 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 453167 2026-02-21T22:19:25.714 INFO:tasks.rados.rados.0.trial178.stdout:751: finishing write tid 1 to trial17818531-5 2026-02-21T22:19:25.714 INFO:tasks.rados.rados.0.trial178.stdout:751: finishing write tid 2 to trial17818531-5 2026-02-21T22:19:25.714 INFO:tasks.rados.rados.0.trial178.stdout:751: finishing write tid 3 to trial17818531-5 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:751: finishing write tid 4 to trial17818531-5 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:751: oid 5 updating version 0 to 222 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:751: oid 5 version 222 is already newer than 221 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:751: oid 5 version 222 is already newer than 220 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 222 (ObjNum 270 snap 83 seq_num 270) dirty exists 2026-02-21T22:19:25.715 INFO:tasks.rados.rados.0.trial178.stdout:751: left oid 5 (ObjNum 270 snap 83 seq_num 270) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:748: done (4 left) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:749: done (3 left) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:750: done (2 left) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:751: done (1 left) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:752: done (0 left) 2026-02-21T22:19:26.421 INFO:tasks.rados.rados.0.trial178.stdout:753: delete oid 20 current snap is 84 2026-02-21T22:19:26.424 INFO:tasks.rados.rados.0.trial178.stdout:753: done (0 left) 2026-02-21T22:19:26.424 INFO:tasks.rados.rados.0.trial178.stdout:754: read oid 13 snap 83 2026-02-21T22:19:26.424 INFO:tasks.rados.rados.0.trial178.stdout:754: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:26.424 INFO:tasks.rados.rados.0.trial178.stdout:755: write oid 50 current snap is 84 2026-02-21T22:19:26.424 INFO:tasks.rados.rados.0.trial178.stdout:755: seq_num 272 ranges {571504=540083,1783808=702748} 2026-02-21T22:19:26.431 INFO:tasks.rados.rados.0.trial178.stdout:755: writing trial17818531-50 from 571504 to 1111587 tid 1 2026-02-21T22:19:26.441 INFO:tasks.rados.rados.0.trial178.stdout:755: writing trial17818531-50 from 1783808 to 2486556 tid 2 2026-02-21T22:19:26.441 INFO:tasks.rados.rados.0.trial178.stdout:756: read oid 38 snap 77 2026-02-21T22:19:26.441 INFO:tasks.rados.rados.0.trial178.stdout:756: expect deleted 2026-02-21T22:19:26.441 INFO:tasks.rados.rados.0.trial178.stdout:757: copy_from oid 41 from oid 4 current snap is 84 2026-02-21T22:19:26.441 INFO:tasks.rados.rados.0.trial178.stdout:754: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:26.442 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 403891 2026-02-21T22:19:26.459 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 987173 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:755: finishing write tid 1 to trial17818531-50 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:755: finishing write tid 2 to trial17818531-50 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:754: done (3 left) 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:756: done (2 left) 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:758: write oid 18 current snap is 84 2026-02-21T22:19:26.486 INFO:tasks.rados.rados.0.trial178.stdout:758: seq_num 273 ranges {588311=615332,1842290=562759} 2026-02-21T22:19:26.494 INFO:tasks.rados.rados.0.trial178.stdout:758: writing trial17818531-18 from 588311 to 1203643 tid 1 2026-02-21T22:19:26.502 INFO:tasks.rados.rados.0.trial178.stdout:758: writing trial17818531-18 from 1842290 to 2405049 tid 2 2026-02-21T22:19:26.503 INFO:tasks.rados.rados.0.trial178.stdout:759: write oid 29 current snap is 84 2026-02-21T22:19:26.503 INFO:tasks.rados.rados.0.trial178.stdout:759: seq_num 274 ranges {605118=406934,1617125=706417} 2026-02-21T22:19:26.509 INFO:tasks.rados.rados.0.trial178.stdout:759: writing trial17818531-29 from 605118 to 1012052 tid 1 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:759: writing trial17818531-29 from 1617125 to 2323542 tid 2 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:760: read oid 27 snap -1 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:760: expect (ObjNum 257 snap 76 seq_num 257) 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:761: read oid 8 snap -1 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:761: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:762: snap_create 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:755: finishing write tid 3 to trial17818531-50 2026-02-21T22:19:26.518 INFO:tasks.rados.rados.0.trial178.stdout:755: finishing write tid 4 to trial17818531-50 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:755: oid 50 updating version 0 to 312 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:755: oid 50 version 312 is already newer than 311 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:755: oid 50 version 312 is already newer than 310 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 312 (ObjNum 272 snap 84 seq_num 272) dirty exists 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:755: left oid 50 (ObjNum 272 snap 84 seq_num 272) 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:757: finishing copy_from racing read to trial17818531-41 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:757: finishing copy_from to trial17818531-41 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 74 (ObjNum 188 snap 56 seq_num 188) dirty exists 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: finishing write tid 1 to trial17818531-18 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: finishing write tid 2 to trial17818531-18 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: finishing write tid 3 to trial17818531-18 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: finishing write tid 4 to trial17818531-18 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: oid 18 updating version 0 to 224 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: oid 18 version 224 is already newer than 223 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: oid 18 updating version 224 to 225 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 225 (ObjNum 273 snap 84 seq_num 273) dirty exists 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:758: left oid 18 (ObjNum 273 snap 84 seq_num 273) 2026-02-21T22:19:26.519 INFO:tasks.rados.rados.0.trial178.stdout:759: finishing write tid 1 to trial17818531-29 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: finishing write tid 2 to trial17818531-29 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: finishing write tid 3 to trial17818531-29 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: finishing write tid 4 to trial17818531-29 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: oid 29 updating version 0 to 105 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: oid 29 version 105 is already newer than 104 2026-02-21T22:19:26.525 INFO:tasks.rados.rados.0.trial178.stdout:759: oid 29 version 105 is already newer than 103 2026-02-21T22:19:26.526 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 105 (ObjNum 274 snap 84 seq_num 274) dirty exists 2026-02-21T22:19:26.526 INFO:tasks.rados.rados.0.trial178.stdout:759: left oid 29 (ObjNum 274 snap 84 seq_num 274) 2026-02-21T22:19:26.526 INFO:tasks.rados.rados.0.trial178.stdout:760: expect (ObjNum 257 snap 76 seq_num 257) 2026-02-21T22:19:26.542 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1196084 2026-02-21T22:19:26.615 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:26.652 INFO:tasks.rados.rados.0.trial178.stdout:761: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:19:26.677 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:26.714 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1921381 2026-02-21T22:19:27.401 INFO:tasks.rados.rados.0.trial178.stdout:755: done (6 left) 2026-02-21T22:19:27.401 INFO:tasks.rados.rados.0.trial178.stdout:757: done (5 left) 2026-02-21T22:19:27.401 INFO:tasks.rados.rados.0.trial178.stdout:758: done (4 left) 2026-02-21T22:19:27.402 INFO:tasks.rados.rados.0.trial178.stdout:759: done (3 left) 2026-02-21T22:19:27.402 INFO:tasks.rados.rados.0.trial178.stdout:760: done (2 left) 2026-02-21T22:19:27.402 INFO:tasks.rados.rados.0.trial178.stdout:761: done (1 left) 2026-02-21T22:19:27.402 INFO:tasks.rados.rados.0.trial178.stdout:762: done (0 left) 2026-02-21T22:19:27.402 INFO:tasks.rados.rados.0.trial178.stdout:763: delete oid 27 current snap is 85 2026-02-21T22:19:27.405 INFO:tasks.rados.rados.0.trial178.stdout:763: done (0 left) 2026-02-21T22:19:27.406 INFO:tasks.rados.rados.0.trial178.stdout:764: write oid 6 current snap is 85 2026-02-21T22:19:27.406 INFO:tasks.rados.rados.0.trial178.stdout:764: seq_num 276 ranges {638732=557432,1617736=710086} 2026-02-21T22:19:27.414 INFO:tasks.rados.rados.0.trial178.stdout:764: writing trial17818531-6 from 638732 to 1196164 tid 1 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:764: writing trial17818531-6 from 1617736 to 2327822 tid 2 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:765: read oid 28 snap -1 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:765: expect (ObjNum 263 snap 81 seq_num 263) 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:766: read oid 17 snap -1 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:766: expect deleted 2026-02-21T22:19:27.421 INFO:tasks.rados.rados.0.trial178.stdout:767: write oid 50 current snap is 85 2026-02-21T22:19:27.422 INFO:tasks.rados.rados.0.trial178.stdout:767: seq_num 277 ranges {655539=632681,2076218=570097} 2026-02-21T22:19:27.430 INFO:tasks.rados.rados.0.trial178.stdout:767: writing trial17818531-50 from 655539 to 1288220 tid 1 2026-02-21T22:19:27.439 INFO:tasks.rados.rados.0.trial178.stdout:767: writing trial17818531-50 from 2076218 to 2646315 tid 2 2026-02-21T22:19:27.439 INFO:tasks.rados.rados.0.trial178.stdout:768: read oid 25 snap -1 2026-02-21T22:19:27.439 INFO:tasks.rados.rados.0.trial178.stdout:768: expect deleted 2026-02-21T22:19:27.439 INFO:tasks.rados.rados.0.trial178.stdout:769: write oid 39 current snap is 85 2026-02-21T22:19:27.439 INFO:tasks.rados.rados.0.trial178.stdout:769: seq_num 278 ranges {672346=707930,2134700=537646} 2026-02-21T22:19:27.448 INFO:tasks.rados.rados.0.trial178.stdout:769: writing trial17818531-39 from 672346 to 1380276 tid 1 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:769: writing trial17818531-39 from 2134700 to 2672346 tid 2 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:770: read oid 29 snap 78 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:770: expect (ObjNum 259 snap 76 seq_num 259) 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:771: delete oid 33 current snap is 85 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:765: expect (ObjNum 263 snap 81 seq_num 263) 2026-02-21T22:19:27.454 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:764: finishing write tid 1 to trial17818531-6 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:764: finishing write tid 2 to trial17818531-6 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:764: finishing write tid 3 to trial17818531-6 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:765: done (7 left) 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:766: done (6 left) 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:771: done (5 left) 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:772: read oid 31 snap -1 2026-02-21T22:19:27.518 INFO:tasks.rados.rados.0.trial178.stdout:772: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:773: delete oid 28 current snap is 85 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:764: finishing write tid 4 to trial17818531-6 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:764: oid 6 updating version 0 to 307 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:764: oid 6 updating version 307 to 308 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:764: oid 6 updating version 308 to 309 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 309 (ObjNum 276 snap 85 seq_num 276) dirty exists 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:764: left oid 6 (ObjNum 276 snap 85 seq_num 276) 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: finishing write tid 1 to trial17818531-50 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: finishing write tid 2 to trial17818531-50 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: finishing write tid 3 to trial17818531-50 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: finishing write tid 4 to trial17818531-50 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: oid 50 updating version 0 to 315 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: oid 50 version 315 is already newer than 313 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: oid 50 version 315 is already newer than 314 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 315 (ObjNum 277 snap 85 seq_num 277) dirty exists 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:767: left oid 50 (ObjNum 277 snap 85 seq_num 277) 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: finishing write tid 1 to trial17818531-39 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: finishing write tid 2 to trial17818531-39 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: finishing write tid 3 to trial17818531-39 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: finishing write tid 4 to trial17818531-39 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: oid 39 updating version 0 to 317 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: oid 39 version 317 is already newer than 316 2026-02-21T22:19:27.519 INFO:tasks.rados.rados.0.trial178.stdout:769: oid 39 version 317 is already newer than 315 2026-02-21T22:19:27.520 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 317 (ObjNum 278 snap 85 seq_num 278) dirty exists 2026-02-21T22:19:27.520 INFO:tasks.rados.rados.0.trial178.stdout:769: left oid 39 (ObjNum 278 snap 85 seq_num 278) 2026-02-21T22:19:27.520 INFO:tasks.rados.rados.0.trial178.stdout:770: expect (ObjNum 259 snap 76 seq_num 259) 2026-02-21T22:19:27.540 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1946273 2026-02-21T22:19:27.565 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1714007 2026-02-21T22:19:27.578 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:27.615 INFO:tasks.rados.rados.0.trial178.stdout:772: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:19:27.687 INFO:tasks.rados.rados.0.trial178.stdout:764: done (6 left) 2026-02-21T22:19:27.687 INFO:tasks.rados.rados.0.trial178.stdout:767: done (5 left) 2026-02-21T22:19:27.687 INFO:tasks.rados.rados.0.trial178.stdout:768: done (4 left) 2026-02-21T22:19:27.687 INFO:tasks.rados.rados.0.trial178.stdout:769: done (3 left) 2026-02-21T22:19:27.687 INFO:tasks.rados.rados.0.trial178.stdout:770: done (2 left) 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:772: done (1 left) 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:773: done (0 left) 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:774: read oid 16 snap -1 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:774: expect (ObjNum 258 snap 76 seq_num 258) 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:775: read oid 29 snap -1 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:775: expect (ObjNum 274 snap 84 seq_num 274) 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:776: copy_from oid 44 from oid 9 current snap is 85 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:777: read oid 15 snap -1 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:777: expect deleted 2026-02-21T22:19:27.688 INFO:tasks.rados.rados.0.trial178.stdout:778: snap_create 2026-02-21T22:19:27.691 INFO:tasks.rados.rados.0.trial178.stdout:774: expect (ObjNum 258 snap 76 seq_num 258) 2026-02-21T22:19:27.733 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:27.758 INFO:tasks.rados.rados.0.trial178.stdout:775: expect (ObjNum 274 snap 84 seq_num 274) 2026-02-21T22:19:27.796 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:27.835 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 463408 2026-02-21T22:19:27.868 INFO:tasks.rados.rados.0.trial178.stdout:776: finishing copy_from racing read to trial17818531-44 2026-02-21T22:19:27.868 INFO:tasks.rados.rados.0.trial178.stdout:776: finishing copy_from to trial17818531-44 2026-02-21T22:19:27.868 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 309 (ObjNum 243 snap 75 seq_num 243) dirty exists 2026-02-21T22:19:28.428 INFO:tasks.rados.rados.0.trial178.stdout:774: done (4 left) 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:775: done (3 left) 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:776: done (2 left) 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:777: done (1 left) 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:778: done (0 left) 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:779: read oid 12 snap 82 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:779: expect deleted 2026-02-21T22:19:28.429 INFO:tasks.rados.rados.0.trial178.stdout:780: snap_create 2026-02-21T22:19:28.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:28.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:29.429 INFO:tasks.rados.rados.0.trial178.stdout:779: done (1 left) 2026-02-21T22:19:29.429 INFO:tasks.rados.rados.0.trial178.stdout:780: done (0 left) 2026-02-21T22:19:29.429 INFO:tasks.rados.rados.0.trial178.stdout:781: delete oid 11 current snap is 87 2026-02-21T22:19:29.433 INFO:tasks.rados.rados.0.trial178.stdout:781: done (0 left) 2026-02-21T22:19:29.433 INFO:tasks.rados.rados.0.trial178.stdout:782: write oid 14 current snap is 87 2026-02-21T22:19:29.433 INFO:tasks.rados.rados.0.trial178.stdout:782: seq_num 282 ranges {739574=725279,1968628=721093} 2026-02-21T22:19:29.443 INFO:tasks.rados.rados.0.trial178.stdout:782: writing trial17818531-14 from 739574 to 1464853 tid 1 2026-02-21T22:19:29.451 INFO:tasks.rados.rados.0.trial178.stdout:782: writing trial17818531-14 from 1968628 to 2689721 tid 2 2026-02-21T22:19:29.451 INFO:tasks.rados.rados.0.trial178.stdout:783: write (excl) oid 13 current snap is 87 2026-02-21T22:19:29.451 INFO:tasks.rados.rados.0.trial178.stdout:782: finishing write tid 1 to trial17818531-14 2026-02-21T22:19:29.452 INFO:tasks.rados.rados.0.trial178.stdout:783: seq_num 283 ranges {756381=400528,1910757=464751} 2026-02-21T22:19:29.459 INFO:tasks.rados.rados.0.trial178.stdout:783: writing trial17818531-13 from 756381 to 1156909 tid 1 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:783: writing trial17818531-13 from 1910757 to 2375508 tid 2 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:784: delete oid 49 current snap is 87 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: finishing write tid 2 to trial17818531-14 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: finishing write tid 3 to trial17818531-14 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: finishing write tid 4 to trial17818531-14 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: oid 14 updating version 0 to 318 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: oid 14 updating version 318 to 319 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: oid 14 updating version 319 to 320 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 320 (ObjNum 282 snap 87 seq_num 282) dirty exists 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:782: left oid 14 (ObjNum 282 snap 87 seq_num 282) 2026-02-21T22:19:29.468 INFO:tasks.rados.rados.0.trial178.stdout:783: finishing write tid 1 to trial17818531-13 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: finishing write tid 2 to trial17818531-13 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: finishing write tid 3 to trial17818531-13 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:782: done (2 left) 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:784: done (1 left) 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:785: delete oid 7 current snap is 87 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: finishing write tid 4 to trial17818531-13 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: oid 13 updating version 0 to 399 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: oid 13 version 399 is already newer than 398 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: oid 13 updating version 399 to 400 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 400 (ObjNum 283 snap 87 seq_num 283) dirty exists 2026-02-21T22:19:29.470 INFO:tasks.rados.rados.0.trial178.stdout:783: left oid 13 (ObjNum 283 snap 87 seq_num 283) 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:783: done (1 left) 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:785: done (0 left) 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:786: copy_from oid 7 from oid 49 current snap is 87 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:787: delete oid 37 current snap is 87 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:786: finishing copy_from to trial17818531-7 2026-02-21T22:19:29.472 INFO:tasks.rados.rados.0.trial178.stdout:786: got expected ENOENT (src dne) 2026-02-21T22:19:29.473 INFO:tasks.rados.rados.0.trial178.stdout:786: finishing copy_from racing read to trial17818531-7 2026-02-21T22:19:29.475 INFO:tasks.rados.rados.0.trial178.stdout:786: done (1 left) 2026-02-21T22:19:29.475 INFO:tasks.rados.rados.0.trial178.stdout:787: done (0 left) 2026-02-21T22:19:29.475 INFO:tasks.rados.rados.0.trial178.stdout:788: write oid 29 current snap is 87 2026-02-21T22:19:29.475 INFO:tasks.rados.rados.0.trial178.stdout:788: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2026-02-21T22:19:29.483 INFO:tasks.rados.rados.0.trial178.stdout:788: writing trial17818531-29 from 423609 to 1125133 tid 1 2026-02-21T22:19:29.489 INFO:tasks.rados.rados.0.trial178.stdout:788: writing trial17818531-29 from 1628332 to 2100421 tid 2 2026-02-21T22:19:29.494 INFO:tasks.rados.rados.0.trial178.stdout:788: writing trial17818531-29 from 2648987 to 2823609 tid 3 2026-02-21T22:19:29.494 INFO:tasks.rados.rados.0.trial178.stdout:789: write oid 10 current snap is 87 2026-02-21T22:19:29.494 INFO:tasks.rados.rados.0.trial178.stdout:788: finishing write tid 1 to trial17818531-29 2026-02-21T22:19:29.494 INFO:tasks.rados.rados.0.trial178.stdout:788: finishing write tid 2 to trial17818531-29 2026-02-21T22:19:29.494 INFO:tasks.rados.rados.0.trial178.stdout:789: seq_num 288 ranges {440416=776773,1686814=732100} 2026-02-21T22:19:29.503 INFO:tasks.rados.rados.0.trial178.stdout:789: writing trial17818531-10 from 440416 to 1217189 tid 1 2026-02-21T22:19:29.508 INFO:tasks.rados.rados.0.trial178.stdout:789: writing trial17818531-10 from 1686814 to 2418914 tid 2 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:790: delete oid 12 current snap is 87 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: finishing write tid 3 to trial17818531-29 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: finishing write tid 4 to trial17818531-29 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: finishing write tid 5 to trial17818531-29 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: oid 29 updating version 0 to 110 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: oid 29 version 110 is already newer than 109 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: oid 29 version 110 is already newer than 108 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: oid 29 version 110 is already newer than 107 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 110 (ObjNum 287 snap 87 seq_num 287) dirty exists 2026-02-21T22:19:29.509 INFO:tasks.rados.rados.0.trial178.stdout:788: left oid 29 (ObjNum 287 snap 87 seq_num 287) 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:789: finishing write tid 1 to trial17818531-10 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:788: done (2 left) 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:790: done (1 left) 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:791: copy_from oid 16 from oid 48 current snap is 87 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:792: copy_from oid 30 from oid 38 current snap is 87 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:793: read oid 13 snap 83 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:793: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:794: read oid 22 snap -1 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:794: expect (ObjNum 222 snap 69 seq_num 222) 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:795: copy_from oid 25 from oid 40 current snap is 87 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:796: copy_from oid 35 from oid 15 current snap is 87 2026-02-21T22:19:29.510 INFO:tasks.rados.rados.0.trial178.stdout:797: copy_from oid 11 from oid 42 current snap is 87 2026-02-21T22:19:29.511 INFO:tasks.rados.rados.0.trial178.stdout:798: write (excl) oid 37 current snap is 87 2026-02-21T22:19:29.511 INFO:tasks.rados.rados.0.trial178.stdout:798: seq_num 290 ranges {474030=643624,1520131=735769} 2026-02-21T22:19:29.512 INFO:tasks.rados.rados.0.trial178.stdout:798: writing trial17818531-37 from 474030 to 1117654 tid 1 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:798: writing trial17818531-37 from 1520131 to 2255900 tid 2 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:799: read oid 36 snap 81 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:799: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:800: copy_from oid 31 from oid 34 current snap is 87 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:789: finishing write tid 2 to trial17818531-10 2026-02-21T22:19:29.517 INFO:tasks.rados.rados.0.trial178.stdout:793: expect (ObjNum 251 snap 75 seq_num 251) 2026-02-21T22:19:29.571 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1177287 2026-02-21T22:19:29.579 INFO:tasks.rados.rados.0.trial178.stdout:792: finishing copy_from to trial17818531-30 2026-02-21T22:19:29.579 INFO:tasks.rados.rados.0.trial178.stdout:792: got expected ENOENT (src dne) 2026-02-21T22:19:29.579 INFO:tasks.rados.rados.0.trial178.stdout:792: finishing copy_from racing read to trial17818531-30 2026-02-21T22:19:29.580 INFO:tasks.rados.rados.0.trial178.stdout:798: finishing write tid 1 to trial17818531-37 2026-02-21T22:19:29.580 INFO:tasks.rados.rados.0.trial178.stdout:792: done (10 left) 2026-02-21T22:19:29.580 INFO:tasks.rados.rados.0.trial178.stdout:793: done (9 left) 2026-02-21T22:19:29.580 INFO:tasks.rados.rados.0.trial178.stdout:801: snap_remove snap 77 2026-02-21T22:19:30.623 INFO:tasks.rados.rados.0.trial178.stdout:801: done (9 left) 2026-02-21T22:19:30.623 INFO:tasks.rados.rados.0.trial178.stdout:802: delete oid 32 current snap is 87 2026-02-21T22:19:30.624 INFO:tasks.rados.rados.0.trial178.stdout:799: expect (ObjNum 238 snap 73 seq_num 238) 2026-02-21T22:19:30.639 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1445411 2026-02-21T22:19:30.702 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 978673 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: finishing write tid 2 to trial17818531-37 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: finishing write tid 3 to trial17818531-37 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: finishing write tid 4 to trial17818531-37 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: oid 37 updating version 0 to 368 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: oid 37 version 368 is already newer than 366 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: oid 37 version 368 is already newer than 367 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 368 (ObjNum 290 snap 87 seq_num 290) dirty exists 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:798: left oid 37 (ObjNum 290 snap 87 seq_num 290) 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:791: finishing copy_from racing read to trial17818531-16 2026-02-21T22:19:30.727 INFO:tasks.rados.rados.0.trial178.stdout:789: finishing write tid 3 to trial17818531-10 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:789: finishing write tid 4 to trial17818531-10 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:789: oid 10 updating version 0 to 311 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:789: oid 10 updating version 311 to 312 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:789: oid 10 version 312 is already newer than 310 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 312 (ObjNum 288 snap 87 seq_num 288) dirty exists 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:789: left oid 10 (ObjNum 288 snap 87 seq_num 288) 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:794: expect (ObjNum 222 snap 69 seq_num 222) 2026-02-21T22:19:30.728 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:30.813 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 424458 2026-02-21T22:19:30.859 INFO:tasks.rados.rados.0.trial178.stdout:791: finishing copy_from to trial17818531-16 2026-02-21T22:19:30.859 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:30.859 INFO:tasks.rados.rados.0.trial178.stdout:796: finishing copy_from to trial17818531-35 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:796: got expected ENOENT (src dne) 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:796: finishing copy_from racing read to trial17818531-35 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:797: finishing copy_from racing read to trial17818531-11 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:797: finishing copy_from to trial17818531-11 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 363 (ObjNum 260 snap 77 seq_num 260) dirty exists 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:795: finishing copy_from racing read to trial17818531-25 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:800: finishing copy_from racing read to trial17818531-31 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:800: finishing copy_from to trial17818531-31 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 93 (ObjNum 240 snap 75 seq_num 240) dirty exists 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:795: finishing copy_from to trial17818531-25 2026-02-21T22:19:30.860 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 309 (ObjNum 198 snap 62 seq_num 198) dirty exists 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:789: done (9 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:791: done (8 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:794: done (7 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:795: done (6 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:796: done (5 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:797: done (4 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:798: done (3 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:799: done (2 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:800: done (1 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:802: done (0 left) 2026-02-21T22:19:30.861 INFO:tasks.rados.rados.0.trial178.stdout:803: snap_remove snap 82 2026-02-21T22:19:31.626 INFO:tasks.rados.rados.0.trial178.stdout:803: done (0 left) 2026-02-21T22:19:31.626 INFO:tasks.rados.rados.0.trial178.stdout:804: copy_from oid 12 from oid 22 current snap is 87 2026-02-21T22:19:31.626 INFO:tasks.rados.rados.0.trial178.stdout:805: rollback oid 9 current snap is 87 2026-02-21T22:19:31.626 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 78 2026-02-21T22:19:31.626 INFO:tasks.rados.rados.0.trial178.stdout:806: delete oid 29 current snap is 87 2026-02-21T22:19:31.628 INFO:tasks.rados.rados.0.trial178.stdout:805: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:19:31.631 INFO:tasks.rados.rados.0.trial178.stdout:806: done (2 left) 2026-02-21T22:19:31.631 INFO:tasks.rados.rados.0.trial178.stdout:807: write oid 14 current snap is 87 2026-02-21T22:19:31.631 INFO:tasks.rados.rados.0.trial178.stdout:807: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2026-02-21T22:19:31.635 INFO:tasks.rados.rados.0.trial178.stdout:807: writing trial17818531-14 from 524451 to 993822 tid 1 2026-02-21T22:19:31.639 INFO:tasks.rados.rados.0.trial178.stdout:807: writing trial17818531-14 from 1579224 to 2062320 tid 2 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:807: writing trial17818531-14 from 2529878 to 2924451 tid 3 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:808: rollback oid 31 current snap is 87 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 86 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:805: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:805: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 323 (ObjNum 243 snap 75 seq_num 243) dirty exists 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:807: finishing write tid 1 to trial17818531-14 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:807: finishing write tid 2 to trial17818531-14 2026-02-21T22:19:31.642 INFO:tasks.rados.rados.0.trial178.stdout:804: finishing copy_from racing read to trial17818531-12 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:804: finishing copy_from to trial17818531-12 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 162 (ObjNum 222 snap 69 seq_num 222) dirty exists 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:804: done (3 left) 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:805: done (2 left) 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:809: rollback oid 10 current snap is 87 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 85 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:810: copy_from oid 50 from oid 9 current snap is 87 2026-02-21T22:19:31.643 INFO:tasks.rados.rados.0.trial178.stdout:811: snap_remove snap 78 2026-02-21T22:19:32.404 INFO:tasks.rados.rados.0.trial178.stdout:811: done (4 left) 2026-02-21T22:19:32.404 INFO:tasks.rados.rados.0.trial178.stdout:812: read oid 1 snap -1 2026-02-21T22:19:32.404 INFO:tasks.rados.rados.0.trial178.stdout:812: expect (ObjNum 225 snap 71 seq_num 225) 2026-02-21T22:19:32.404 INFO:tasks.rados.rados.0.trial178.stdout:813: rollback oid 4 current snap is 87 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 86 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:808: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:808: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:809: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:808: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 312 (ObjNum 239 snap 75 seq_num 239) dirty exists 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:809: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:809: finishing rollback tid 2 to trial17818531-10 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 315 (ObjNum 256 snap 75 seq_num 256) dirty exists 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:807: finishing write tid 3 to trial17818531-14 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:807: finishing write tid 4 to trial17818531-14 2026-02-21T22:19:32.405 INFO:tasks.rados.rados.0.trial178.stdout:807: finishing write tid 5 to trial17818531-14 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: oid 14 updating version 0 to 326 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: oid 14 version 326 is already newer than 325 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: oid 14 version 326 is already newer than 324 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: oid 14 updating version 326 to 327 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 327 (ObjNum 293 snap 87 seq_num 293) dirty exists 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: left oid 14 (ObjNum 293 snap 87 seq_num 293) 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:810: finishing copy_from racing read to trial17818531-50 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:810: finishing copy_from to trial17818531-50 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 323 (ObjNum 243 snap 75 seq_num 243) dirty exists 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:807: done (5 left) 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:808: done (4 left) 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:809: done (3 left) 2026-02-21T22:19:32.406 INFO:tasks.rados.rados.0.trial178.stdout:810: done (2 left) 2026-02-21T22:19:32.407 INFO:tasks.rados.rados.0.trial178.stdout:814: rollback oid 27 current snap is 87 2026-02-21T22:19:32.407 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 84 2026-02-21T22:19:32.407 INFO:tasks.rados.rados.0.trial178.stdout:815: write (excl) oid 13 current snap is 87 2026-02-21T22:19:32.407 INFO:tasks.rados.rados.0.trial178.stdout:815: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2026-02-21T22:19:32.409 INFO:tasks.rados.rados.0.trial178.stdout:815: writing trial17818531-13 from 541258 to 1085878 tid 1 2026-02-21T22:19:32.414 INFO:tasks.rados.rados.0.trial178.stdout:815: writing trial17818531-13 from 1637706 to 2264460 tid 2 2026-02-21T22:19:32.417 INFO:tasks.rados.rados.0.trial178.stdout:815: writing trial17818531-13 from 2840948 to 2941258 tid 3 2026-02-21T22:19:32.417 INFO:tasks.rados.rados.0.trial178.stdout:816: snap_remove snap 83 2026-02-21T22:19:33.402 INFO:tasks.rados.rados.0.trial178.stdout:816: done (4 left) 2026-02-21T22:19:33.402 INFO:tasks.rados.rados.0.trial178.stdout:817: read oid 41 snap -1 2026-02-21T22:19:33.402 INFO:tasks.rados.rados.0.trial178.stdout:817: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:818: snap_create 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:814: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:814: finishing rollback tid 2 to trial17818531-27 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 329 (ObjNum 257 snap 76 seq_num 257) dirty exists 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:815: finishing write tid 1 to trial17818531-13 2026-02-21T22:19:33.403 INFO:tasks.rados.rados.0.trial178.stdout:812: expect (ObjNum 225 snap 71 seq_num 225) 2026-02-21T22:19:33.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:33.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:33.573 INFO:tasks.rados.rados.0.trial178.stdout:813: finishing rollback tid 0 to trial17818531-4 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:813: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:813: finishing rollback tid 2 to trial17818531-4 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 312 (ObjNum 188 snap 56 seq_num 188) dirty exists 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: finishing write tid 2 to trial17818531-13 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: finishing write tid 3 to trial17818531-13 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: finishing write tid 4 to trial17818531-13 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: finishing write tid 5 to trial17818531-13 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: oid 13 updating version 0 to 420 2026-02-21T22:19:33.574 INFO:tasks.rados.rados.0.trial178.stdout:815: oid 13 updating version 420 to 422 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:815: oid 13 version 422 is already newer than 421 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:815: oid 13 updating version 422 to 423 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 423 (ObjNum 294 snap 87 seq_num 294) dirty exists 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:815: left oid 13 (ObjNum 294 snap 87 seq_num 294) 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:817: expect (ObjNum 188 snap 56 seq_num 188) 2026-02-21T22:19:33.575 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:33.620 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2081166 2026-02-21T22:19:33.631 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:34.407 INFO:tasks.rados.rados.0.trial178.stdout:812: done (5 left) 2026-02-21T22:19:34.407 INFO:tasks.rados.rados.0.trial178.stdout:813: done (4 left) 2026-02-21T22:19:34.407 INFO:tasks.rados.rados.0.trial178.stdout:814: done (3 left) 2026-02-21T22:19:34.407 INFO:tasks.rados.rados.0.trial178.stdout:815: done (2 left) 2026-02-21T22:19:34.407 INFO:tasks.rados.rados.0.trial178.stdout:817: done (1 left) 2026-02-21T22:19:34.408 INFO:tasks.rados.rados.0.trial178.stdout:818: done (0 left) 2026-02-21T22:19:34.408 INFO:tasks.rados.rados.0.trial178.stdout:819: write (excl) oid 27 current snap is 88 2026-02-21T22:19:34.408 INFO:tasks.rados.rados.0.trial178.stdout:819: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2026-02-21T22:19:34.414 INFO:tasks.rados.rados.0.trial178.stdout:819: writing trial17818531-27 from 558065 to 1177934 tid 1 2026-02-21T22:19:34.420 INFO:tasks.rados.rados.0.trial178.stdout:819: writing trial17818531-27 from 1579835 to 2066600 tid 2 2026-02-21T22:19:34.426 INFO:tasks.rados.rados.0.trial178.stdout:819: writing trial17818531-27 from 2468371 to 2958065 tid 3 2026-02-21T22:19:34.426 INFO:tasks.rados.rados.0.trial178.stdout:820: delete oid 30 current snap is 88 2026-02-21T22:19:34.426 INFO:tasks.rados.rados.0.trial178.stdout:819: finishing write tid 1 to trial17818531-27 2026-02-21T22:19:34.428 INFO:tasks.rados.rados.0.trial178.stdout:819: finishing write tid 2 to trial17818531-27 2026-02-21T22:19:34.428 INFO:tasks.rados.rados.0.trial178.stdout:819: finishing write tid 3 to trial17818531-27 2026-02-21T22:19:34.428 INFO:tasks.rados.rados.0.trial178.stdout:819: finishing write tid 4 to trial17818531-27 2026-02-21T22:19:34.428 INFO:tasks.rados.rados.0.trial178.stdout:819: finishing write tid 5 to trial17818531-27 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:819: oid 27 updating version 0 to 332 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:819: oid 27 version 332 is already newer than 330 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:819: oid 27 version 332 is already newer than 331 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:819: oid 27 updating version 332 to 333 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 333 (ObjNum 295 snap 88 seq_num 295) dirty exists 2026-02-21T22:19:34.429 INFO:tasks.rados.rados.0.trial178.stdout:819: left oid 27 (ObjNum 295 snap 88 seq_num 295) 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:819: done (1 left) 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:820: done (0 left) 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:821: rollback oid 21 current snap is 88 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 85 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:822: read oid 7 snap -1 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:822: expect deleted 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:823: write oid 43 current snap is 88 2026-02-21T22:19:34.430 INFO:tasks.rados.rados.0.trial178.stdout:823: seq_num 297 ranges {591679=486720,1813152=490434} 2026-02-21T22:19:34.437 INFO:tasks.rados.rados.0.trial178.stdout:823: writing trial17818531-43 from 591679 to 1078399 tid 1 2026-02-21T22:19:34.445 INFO:tasks.rados.rados.0.trial178.stdout:823: writing trial17818531-43 from 1813152 to 2303586 tid 2 2026-02-21T22:19:34.445 INFO:tasks.rados.rados.0.trial178.stdout:824: write (excl) oid 42 current snap is 88 2026-02-21T22:19:34.445 INFO:tasks.rados.rados.0.trial178.stdout:824: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2026-02-21T22:19:34.453 INFO:tasks.rados.rados.0.trial178.stdout:824: writing trial17818531-42 from 608486 to 1170455 tid 1 2026-02-21T22:19:34.461 INFO:tasks.rados.rados.0.trial178.stdout:824: writing trial17818531-42 from 1871634 to 2505726 tid 2 2026-02-21T22:19:34.463 INFO:tasks.rados.rados.0.trial178.stdout:824: writing trial17818531-42 from 2950640 to 3008486 tid 3 2026-02-21T22:19:34.463 INFO:tasks.rados.rados.0.trial178.stdout:825: copy_from oid 36 from oid 12 current snap is 88 2026-02-21T22:19:34.463 INFO:tasks.rados.rados.0.trial178.stdout:821: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:821: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:821: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 403 (ObjNum 264 snap 81 seq_num 264) dirty exists 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:823: finishing write tid 1 to trial17818531-43 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:821: done (4 left) 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:822: done (3 left) 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:826: read oid 37 snap -1 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:826: expect (ObjNum 290 snap 87 seq_num 290) 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:827: write oid 29 current snap is 88 2026-02-21T22:19:34.464 INFO:tasks.rados.rados.0.trial178.stdout:827: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2026-02-21T22:19:34.468 INFO:tasks.rados.rados.0.trial178.stdout:827: writing trial17818531-29 from 625293 to 1262511 tid 1 2026-02-21T22:19:34.472 INFO:tasks.rados.rados.0.trial178.stdout:827: writing trial17818531-29 from 1813763 to 2307866 tid 2 2026-02-21T22:19:34.475 INFO:tasks.rados.rados.0.trial178.stdout:827: writing trial17818531-29 from 2978063 to 3025293 tid 3 2026-02-21T22:19:34.475 INFO:tasks.rados.rados.0.trial178.stdout:828: write (excl) oid 34 current snap is 88 2026-02-21T22:19:34.475 INFO:tasks.rados.rados.0.trial178.stdout:828: seq_num 300 ranges {642100=712467,1872245=637761} 2026-02-21T22:19:34.480 INFO:tasks.rados.rados.0.trial178.stdout:828: writing trial17818531-34 from 642100 to 1354567 tid 1 2026-02-21T22:19:34.485 INFO:tasks.rados.rados.0.trial178.stdout:828: writing trial17818531-34 from 1872245 to 2510006 tid 2 2026-02-21T22:19:34.485 INFO:tasks.rados.rados.0.trial178.stdout:829: rollback oid 49 current snap is 88 2026-02-21T22:19:34.485 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 85 2026-02-21T22:19:34.485 INFO:tasks.rados.rados.0.trial178.stdout:823: finishing write tid 2 to trial17818531-43 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: finishing write tid 3 to trial17818531-43 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: finishing write tid 4 to trial17818531-43 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: oid 43 updating version 0 to 93 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: oid 43 version 93 is already newer than 91 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: oid 43 version 93 is already newer than 92 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 93 (ObjNum 297 snap 88 seq_num 297) dirty exists 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:823: left oid 43 (ObjNum 297 snap 88 seq_num 297) 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:824: finishing write tid 1 to trial17818531-42 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:824: finishing write tid 2 to trial17818531-42 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:824: finishing write tid 3 to trial17818531-42 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:824: finishing write tid 4 to trial17818531-42 2026-02-21T22:19:34.486 INFO:tasks.rados.rados.0.trial178.stdout:824: finishing write tid 5 to trial17818531-42 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:824: oid 42 updating version 0 to 408 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:824: oid 42 version 408 is already newer than 406 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:824: oid 42 version 408 is already newer than 407 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:824: oid 42 version 408 is already newer than 405 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 408 (ObjNum 298 snap 88 seq_num 298) dirty exists 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:824: left oid 42 (ObjNum 298 snap 88 seq_num 298) 2026-02-21T22:19:34.487 INFO:tasks.rados.rados.0.trial178.stdout:826: expect (ObjNum 290 snap 87 seq_num 290) 2026-02-21T22:19:34.488 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1411758 2026-02-21T22:19:34.555 INFO:tasks.rados.rados.0.trial178.stdout:827: finishing write tid 1 to trial17818531-29 2026-02-21T22:19:34.555 INFO:tasks.rados.rados.0.trial178.stdout:827: finishing write tid 2 to trial17818531-29 2026-02-21T22:19:34.555 INFO:tasks.rados.rados.0.trial178.stdout:827: finishing write tid 3 to trial17818531-29 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: finishing write tid 4 to trial17818531-29 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:823: done (6 left) 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:824: done (5 left) 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:826: done (4 left) 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:830: snap_create 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: finishing write tid 5 to trial17818531-29 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: oid 29 updating version 0 to 121 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: oid 29 version 121 is already newer than 120 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: oid 29 updating version 121 to 123 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: oid 29 version 123 is already newer than 122 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 123 (ObjNum 299 snap 88 seq_num 299) dirty exists 2026-02-21T22:19:34.556 INFO:tasks.rados.rados.0.trial178.stdout:827: left oid 29 (ObjNum 299 snap 88 seq_num 299) 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:825: finishing copy_from racing read to trial17818531-36 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:825: finishing copy_from to trial17818531-36 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 162 (ObjNum 222 snap 69 seq_num 222) dirty exists 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: finishing write tid 1 to trial17818531-34 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: finishing write tid 2 to trial17818531-34 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: finishing write tid 3 to trial17818531-34 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: finishing write tid 4 to trial17818531-34 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: oid 34 updating version 0 to 126 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: oid 34 version 126 is already newer than 125 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: oid 34 updating version 126 to 127 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 127 (ObjNum 300 snap 88 seq_num 300) dirty exists 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:828: left oid 34 (ObjNum 300 snap 88 seq_num 300) 2026-02-21T22:19:34.557 INFO:tasks.rados.rados.0.trial178.stdout:829: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:19:34.558 INFO:tasks.rados.rados.0.trial178.stdout:829: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:19:34.558 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 314 (ObjNum 265 snap 82 seq_num 265) dirty exists 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:825: done (4 left) 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:827: done (3 left) 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:828: done (2 left) 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:829: done (1 left) 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:830: done (0 left) 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:831: write oid 25 current snap is 89 2026-02-21T22:19:35.410 INFO:tasks.rados.rados.0.trial178.stdout:831: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2026-02-21T22:19:35.416 INFO:tasks.rados.rados.0.trial178.stdout:831: writing trial17818531-25 from 658907 to 1446623 tid 1 2026-02-21T22:19:35.419 INFO:tasks.rados.rados.0.trial178.stdout:831: writing trial17818531-25 from 1930727 to 2428499 tid 2 2026-02-21T22:19:35.422 INFO:tasks.rados.rados.0.trial178.stdout:831: writing trial17818531-25 from 3032909 to 3058907 tid 3 2026-02-21T22:19:35.422 INFO:tasks.rados.rados.0.trial178.stdout:832: read oid 23 snap 88 2026-02-21T22:19:35.422 INFO:tasks.rados.rados.0.trial178.stdout:832: expect (ObjNum 242 snap 75 seq_num 242) 2026-02-21T22:19:35.422 INFO:tasks.rados.rados.0.trial178.stdout:833: snap_remove snap 81 2026-02-21T22:19:36.414 INFO:tasks.rados.rados.0.trial178.stdout:833: done (2 left) 2026-02-21T22:19:36.414 INFO:tasks.rados.rados.0.trial178.stdout:834: read oid 7 snap -1 2026-02-21T22:19:36.414 INFO:tasks.rados.rados.0.trial178.stdout:834: expect deleted 2026-02-21T22:19:36.415 INFO:tasks.rados.rados.0.trial178.stdout:835: read oid 15 snap -1 2026-02-21T22:19:36.415 INFO:tasks.rados.rados.0.trial178.stdout:835: expect deleted 2026-02-21T22:19:36.415 INFO:tasks.rados.rados.0.trial178.stdout:836: snap_remove snap 85 2026-02-21T22:19:37.404 INFO:tasks.rados.rados.0.trial178.stdout:836: done (4 left) 2026-02-21T22:19:37.404 INFO:tasks.rados.rados.0.trial178.stdout:837: snap_create 2026-02-21T22:19:37.404 INFO:tasks.rados.rados.0.trial178.stdout:831: finishing write tid 1 to trial17818531-25 2026-02-21T22:19:37.404 INFO:tasks.rados.rados.0.trial178.stdout:831: finishing write tid 2 to trial17818531-25 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: finishing write tid 3 to trial17818531-25 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: finishing write tid 4 to trial17818531-25 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: finishing write tid 5 to trial17818531-25 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: oid 25 updating version 0 to 316 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: oid 25 updating version 316 to 317 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: oid 25 version 317 is already newer than 315 2026-02-21T22:19:37.405 INFO:tasks.rados.rados.0.trial178.stdout:831: oid 25 updating version 317 to 318 2026-02-21T22:19:37.406 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 318 (ObjNum 301 snap 89 seq_num 301) dirty exists 2026-02-21T22:19:37.406 INFO:tasks.rados.rados.0.trial178.stdout:831: left oid 25 (ObjNum 301 snap 89 seq_num 301) 2026-02-21T22:19:37.406 INFO:tasks.rados.rados.0.trial178.stdout:832: expect (ObjNum 242 snap 75 seq_num 242) 2026-02-21T22:19:37.463 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:37.517 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:19:37.518 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:19:37.523 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:19:37.524 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:19:37.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:19:37.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:19:37.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:831: done (4 left) 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:832: done (3 left) 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:834: done (2 left) 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:835: done (1 left) 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:837: done (0 left) 2026-02-21T22:19:38.407 INFO:tasks.rados.rados.0.trial178.stdout:838: snap_remove snap 89 2026-02-21T22:19:38.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:38.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:39.407 INFO:tasks.rados.rados.0.trial178.stdout:838: done (0 left) 2026-02-21T22:19:39.407 INFO:tasks.rados.rados.0.trial178.stdout:839: copy_from oid 40 from oid 46 current snap is 90 2026-02-21T22:19:39.407 INFO:tasks.rados.rados.0.trial178.stdout:840: snap_remove snap 88 2026-02-21T22:19:40.412 INFO:tasks.rados.rados.0.trial178.stdout:840: done (1 left) 2026-02-21T22:19:40.412 INFO:tasks.rados.rados.0.trial178.stdout:841: write (excl) oid 37 current snap is 90 2026-02-21T22:19:40.412 INFO:tasks.rados.rados.0.trial178.stdout:841: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2026-02-21T22:19:40.420 INFO:tasks.rados.rados.0.trial178.stdout:841: writing trial17818531-37 from 675714 to 1138679 tid 1 2026-02-21T22:19:40.427 INFO:tasks.rados.rados.0.trial178.stdout:841: writing trial17818531-37 from 1589209 to 2230639 tid 2 2026-02-21T22:19:40.433 INFO:tasks.rados.rados.0.trial178.stdout:841: writing trial17818531-37 from 2943979 to 3075714 tid 3 2026-02-21T22:19:40.433 INFO:tasks.rados.rados.0.trial178.stdout:842: snap_remove snap 87 2026-02-21T22:19:41.415 INFO:tasks.rados.rados.0.trial178.stdout:842: done (2 left) 2026-02-21T22:19:41.416 INFO:tasks.rados.rados.0.trial178.stdout:843: write oid 20 current snap is 90 2026-02-21T22:19:41.416 INFO:tasks.rados.rados.0.trial178.stdout:843: seq_num 303 ranges {692521=538214,1931338=785088} 2026-02-21T22:19:41.424 INFO:tasks.rados.rados.0.trial178.stdout:843: writing trial17818531-20 from 692521 to 1230735 tid 1 2026-02-21T22:19:41.435 INFO:tasks.rados.rados.0.trial178.stdout:843: writing trial17818531-20 from 1931338 to 2716426 tid 2 2026-02-21T22:19:41.435 INFO:tasks.rados.rados.0.trial178.stdout:844: read oid 19 snap -1 2026-02-21T22:19:41.435 INFO:tasks.rados.rados.0.trial178.stdout:844: expect (ObjNum 261 snap 80 seq_num 261) 2026-02-21T22:19:41.435 INFO:tasks.rados.rados.0.trial178.stdout:845: write oid 42 current snap is 90 2026-02-21T22:19:41.435 INFO:tasks.rados.rados.0.trial178.stdout:845: seq_num 304 ranges {709328=613463,1989820=645099} 2026-02-21T22:19:41.443 INFO:tasks.rados.rados.0.trial178.stdout:845: writing trial17818531-42 from 709328 to 1322791 tid 1 2026-02-21T22:19:41.453 INFO:tasks.rados.rados.0.trial178.stdout:845: writing trial17818531-42 from 1989820 to 2634919 tid 2 2026-02-21T22:19:41.453 INFO:tasks.rados.rados.0.trial178.stdout:846: read oid 32 snap -1 2026-02-21T22:19:41.453 INFO:tasks.rados.rados.0.trial178.stdout:846: expect deleted 2026-02-21T22:19:41.453 INFO:tasks.rados.rados.0.trial178.stdout:847: write oid 1 current snap is 90 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:839: finishing copy_from to trial17818531-40 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:839: got expected ENOENT (src dne) 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:839: finishing copy_from racing read to trial17818531-40 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: finishing write tid 1 to trial17818531-37 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: finishing write tid 2 to trial17818531-37 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: finishing write tid 3 to trial17818531-37 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: finishing write tid 4 to trial17818531-37 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: finishing write tid 5 to trial17818531-37 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: oid 37 updating version 0 to 372 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: oid 37 version 372 is already newer than 370 2026-02-21T22:19:41.454 INFO:tasks.rados.rados.0.trial178.stdout:841: oid 37 version 372 is already newer than 369 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:841: oid 37 version 372 is already newer than 371 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 372 (ObjNum 302 snap 90 seq_num 302) dirty exists 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:841: left oid 37 (ObjNum 302 snap 90 seq_num 302) 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:843: finishing write tid 1 to trial17818531-20 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:843: finishing write tid 2 to trial17818531-20 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:843: finishing write tid 3 to trial17818531-20 2026-02-21T22:19:41.455 INFO:tasks.rados.rados.0.trial178.stdout:847: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2026-02-21T22:19:41.460 INFO:tasks.rados.rados.0.trial178.stdout:847: writing trial17818531-1 from 726135 to 1131200 tid 1 2026-02-21T22:19:41.469 INFO:tasks.rados.rados.0.trial178.stdout:847: writing trial17818531-1 from 1764655 to 2553412 tid 2 2026-02-21T22:19:41.473 INFO:tasks.rados.rados.0.trial178.stdout:847: writing trial17818531-1 from 3026248 to 3126135 tid 3 2026-02-21T22:19:41.473 INFO:tasks.rados.rados.0.trial178.stdout:839: done (6 left) 2026-02-21T22:19:41.473 INFO:tasks.rados.rados.0.trial178.stdout:841: done (5 left) 2026-02-21T22:19:41.473 INFO:tasks.rados.rados.0.trial178.stdout:848: write oid 38 current snap is 90 2026-02-21T22:19:41.473 INFO:tasks.rados.rados.0.trial178.stdout:848: seq_num 306 ranges {742942=480314,1823137=648768} 2026-02-21T22:19:41.477 INFO:tasks.rados.rados.0.trial178.stdout:848: writing trial17818531-38 from 742942 to 1223256 tid 1 2026-02-21T22:19:41.482 INFO:tasks.rados.rados.0.trial178.stdout:848: writing trial17818531-38 from 1823137 to 2471905 tid 2 2026-02-21T22:19:41.482 INFO:tasks.rados.rados.0.trial178.stdout:849: snap_create 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:843: finishing write tid 4 to trial17818531-20 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:843: oid 20 updating version 0 to 268 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:843: oid 20 version 268 is already newer than 266 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:843: oid 20 version 268 is already newer than 267 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 268 (ObjNum 303 snap 90 seq_num 303) dirty exists 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:843: left oid 20 (ObjNum 303 snap 90 seq_num 303) 2026-02-21T22:19:41.483 INFO:tasks.rados.rados.0.trial178.stdout:844: expect (ObjNum 261 snap 80 seq_num 261) 2026-02-21T22:19:41.542 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 408965 2026-02-21T22:19:41.563 INFO:tasks.rados.rados.0.trial178.stdout:845: finishing write tid 1 to trial17818531-42 2026-02-21T22:19:41.563 INFO:tasks.rados.rados.0.trial178.stdout:845: finishing write tid 2 to trial17818531-42 2026-02-21T22:19:41.563 INFO:tasks.rados.rados.0.trial178.stdout:845: finishing write tid 3 to trial17818531-42 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:845: finishing write tid 4 to trial17818531-42 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:845: oid 42 updating version 0 to 425 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:845: oid 42 updating version 425 to 426 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:845: oid 42 updating version 426 to 427 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 427 (ObjNum 304 snap 90 seq_num 304) dirty exists 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:845: left oid 42 (ObjNum 304 snap 90 seq_num 304) 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:847: finishing write tid 1 to trial17818531-1 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:847: finishing write tid 2 to trial17818531-1 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:847: finishing write tid 3 to trial17818531-1 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:847: finishing write tid 4 to trial17818531-1 2026-02-21T22:19:41.564 INFO:tasks.rados.rados.0.trial178.stdout:847: finishing write tid 5 to trial17818531-1 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:847: oid 1 updating version 0 to 374 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:847: oid 1 updating version 374 to 376 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:847: oid 1 version 376 is already newer than 373 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:847: oid 1 version 376 is already newer than 375 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 376 (ObjNum 305 snap 90 seq_num 305) dirty exists 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:847: left oid 1 (ObjNum 305 snap 90 seq_num 305) 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: finishing write tid 1 to trial17818531-38 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: finishing write tid 2 to trial17818531-38 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: finishing write tid 3 to trial17818531-38 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: finishing write tid 4 to trial17818531-38 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: oid 38 updating version 0 to 122 2026-02-21T22:19:41.565 INFO:tasks.rados.rados.0.trial178.stdout:848: oid 38 updating version 122 to 123 2026-02-21T22:19:41.566 INFO:tasks.rados.rados.0.trial178.stdout:848: oid 38 updating version 123 to 124 2026-02-21T22:19:41.566 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 124 (ObjNum 306 snap 90 seq_num 306) dirty exists 2026-02-21T22:19:41.566 INFO:tasks.rados.rados.0.trial178.stdout:848: left oid 38 (ObjNum 306 snap 90 seq_num 306) 2026-02-21T22:19:42.404 INFO:tasks.rados.rados.0.trial178.stdout:843: done (6 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:844: done (5 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:845: done (4 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:846: done (3 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:847: done (2 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:848: done (1 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:849: done (0 left) 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:850: copy_from oid 32 from oid 16 current snap is 91 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:851: write (excl) oid 39 current snap is 91 2026-02-21T22:19:42.405 INFO:tasks.rados.rados.0.trial178.stdout:851: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2026-02-21T22:19:42.409 INFO:tasks.rados.rados.0.trial178.stdout:851: writing trial17818531-39 from 759749 to 1315312 tid 1 2026-02-21T22:19:42.414 INFO:tasks.rados.rados.0.trial178.stdout:851: writing trial17818531-39 from 1765266 to 2557692 tid 2 2026-02-21T22:19:42.417 INFO:tasks.rados.rados.0.trial178.stdout:851: writing trial17818531-39 from 2964741 to 3159749 tid 3 2026-02-21T22:19:42.417 INFO:tasks.rados.rados.0.trial178.stdout:852: snap_remove snap 90 2026-02-21T22:19:43.419 INFO:tasks.rados.rados.0.trial178.stdout:852: done (2 left) 2026-02-21T22:19:43.419 INFO:tasks.rados.rados.0.trial178.stdout:853: read oid 47 snap 84 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:853: expect (ObjNum 232 snap 72 seq_num 232) 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:854: write (excl) oid 30 current snap is 91 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:851: finishing write tid 1 to trial17818531-39 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:851: finishing write tid 2 to trial17818531-39 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:851: finishing write tid 3 to trial17818531-39 2026-02-21T22:19:43.420 INFO:tasks.rados.rados.0.trial178.stdout:854: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2026-02-21T22:19:43.426 INFO:tasks.rados.rados.0.trial178.stdout:854: writing trial17818531-30 from 776556 to 1407368 tid 1 2026-02-21T22:19:43.434 INFO:tasks.rados.rados.0.trial178.stdout:854: writing trial17818531-30 from 1823748 to 2476185 tid 2 2026-02-21T22:19:43.439 INFO:tasks.rados.rados.0.trial178.stdout:854: writing trial17818531-30 from 3108517 to 3176556 tid 3 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:855: delete oid 49 current snap is 91 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: finishing write tid 4 to trial17818531-39 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: finishing write tid 5 to trial17818531-39 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: oid 39 updating version 0 to 334 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: oid 39 updating version 334 to 335 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: oid 39 updating version 335 to 336 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:851: oid 39 updating version 336 to 337 2026-02-21T22:19:43.440 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 337 (ObjNum 307 snap 91 seq_num 307) dirty exists 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:851: left oid 39 (ObjNum 307 snap 91 seq_num 307) 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:850: finishing copy_from racing read to trial17818531-32 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:850: finishing copy_from to trial17818531-32 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:853: expect (ObjNum 232 snap 72 seq_num 232) 2026-02-21T22:19:43.441 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:43.488 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:43.529 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3819526 2026-02-21T22:19:43.532 INFO:tasks.rados.rados.0.trial178.stdout:854: finishing write tid 1 to trial17818531-30 2026-02-21T22:19:43.532 INFO:tasks.rados.rados.0.trial178.stdout:854: finishing write tid 2 to trial17818531-30 2026-02-21T22:19:43.532 INFO:tasks.rados.rados.0.trial178.stdout:854: finishing write tid 3 to trial17818531-30 2026-02-21T22:19:43.532 INFO:tasks.rados.rados.0.trial178.stdout:854: finishing write tid 4 to trial17818531-30 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:854: finishing write tid 5 to trial17818531-30 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:854: oid 30 updating version 0 to 242 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:854: oid 30 updating version 242 to 243 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:854: oid 30 updating version 243 to 245 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:854: oid 30 version 245 is already newer than 244 2026-02-21T22:19:43.533 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 245 (ObjNum 308 snap 91 seq_num 308) dirty exists 2026-02-21T22:19:43.534 INFO:tasks.rados.rados.0.trial178.stdout:854: left oid 30 (ObjNum 308 snap 91 seq_num 308) 2026-02-21T22:19:43.535 INFO:tasks.rados.rados.0.trial178.stdout:850: done (4 left) 2026-02-21T22:19:43.535 INFO:tasks.rados.rados.0.trial178.stdout:851: done (3 left) 2026-02-21T22:19:43.535 INFO:tasks.rados.rados.0.trial178.stdout:853: done (2 left) 2026-02-21T22:19:43.535 INFO:tasks.rados.rados.0.trial178.stdout:854: done (1 left) 2026-02-21T22:19:43.536 INFO:tasks.rados.rados.0.trial178.stdout:855: done (0 left) 2026-02-21T22:19:43.536 INFO:tasks.rados.rados.0.trial178.stdout:856: write (excl) oid 23 current snap is 91 2026-02-21T22:19:43.536 INFO:tasks.rados.rados.0.trial178.stdout:856: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2026-02-21T22:19:43.542 INFO:tasks.rados.rados.0.trial178.stdout:856: writing trial17818531-23 from 410170 to 1191480 tid 1 2026-02-21T22:19:43.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:43.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:43.551 INFO:tasks.rados.rados.0.trial178.stdout:856: writing trial17818531-23 from 1940712 to 2596818 tid 2 2026-02-21T22:19:43.556 INFO:tasks.rados.rados.0.trial178.stdout:856: writing trial17818531-23 from 3163363 to 3210170 tid 3 2026-02-21T22:19:43.556 INFO:tasks.rados.rados.0.trial178.stdout:857: snap_remove snap 86 2026-02-21T22:19:44.423 INFO:tasks.rados.rados.0.trial178.stdout:857: done (1 left) 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:858: read oid 25 snap -1 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:858: expect (ObjNum 301 snap 89 seq_num 301) 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:859: rollback oid 18 current snap is 91 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 84 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: finishing write tid 1 to trial17818531-23 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: finishing write tid 2 to trial17818531-23 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: finishing write tid 3 to trial17818531-23 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: finishing write tid 4 to trial17818531-23 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: finishing write tid 5 to trial17818531-23 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: oid 23 updating version 0 to 171 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: oid 23 version 171 is already newer than 169 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: oid 23 updating version 171 to 172 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: oid 23 version 172 is already newer than 170 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 172 (ObjNum 310 snap 91 seq_num 310) dirty exists 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: left oid 23 (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:856: done (2 left) 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:860: read oid 2 snap -1 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:860: expect deleted 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:861: rollback oid 31 current snap is 91 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 84 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:862: rollback oid 17 current snap is 91 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 84 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:863: write (excl) oid 14 current snap is 91 2026-02-21T22:19:44.424 INFO:tasks.rados.rados.0.trial178.stdout:863: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2026-02-21T22:19:44.432 INFO:tasks.rados.rados.0.trial178.stdout:863: writing trial17818531-14 from 426977 to 883536 tid 1 2026-02-21T22:19:44.442 INFO:tasks.rados.rados.0.trial178.stdout:863: writing trial17818531-14 from 1482841 to 2282605 tid 2 2026-02-21T22:19:44.449 INFO:tasks.rados.rados.0.trial178.stdout:863: writing trial17818531-14 from 2958080 to 3226977 tid 3 2026-02-21T22:19:44.449 INFO:tasks.rados.rados.0.trial178.stdout:864: rollback oid 47 current snap is 91 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 84 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:862: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 0 (ObjNum 129 snap 0 seq_num 101033400) dirty dne 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:859: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:859: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:859: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 249 (ObjNum 273 snap 84 seq_num 273) dirty exists 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:863: finishing write tid 1 to trial17818531-14 2026-02-21T22:19:44.450 INFO:tasks.rados.rados.0.trial178.stdout:858: expect (ObjNum 301 snap 89 seq_num 301) 2026-02-21T22:19:44.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 4857 2026-02-21T22:19:44.531 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:861: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:861: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:861: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 322 (ObjNum 239 snap 75 seq_num 239) dirty exists 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:863: finishing write tid 2 to trial17818531-14 2026-02-21T22:19:44.573 INFO:tasks.rados.rados.0.trial178.stdout:863: finishing write tid 3 to trial17818531-14 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: finishing write tid 4 to trial17818531-14 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: finishing write tid 5 to trial17818531-14 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: oid 14 updating version 0 to 338 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: oid 14 updating version 338 to 341 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: oid 14 version 341 is already newer than 340 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: oid 14 version 341 is already newer than 339 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 341 (ObjNum 311 snap 91 seq_num 311) dirty exists 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:863: left oid 14 (ObjNum 311 snap 91 seq_num 311) 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:864: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:864: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:864: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:19:44.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 437 (ObjNum 232 snap 72 seq_num 232) dirty exists 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:858: done (6 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:859: done (5 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:860: done (4 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:861: done (3 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:862: done (2 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:863: done (1 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:864: done (0 left) 2026-02-21T22:19:44.575 INFO:tasks.rados.rados.0.trial178.stdout:865: snap_remove snap 84 2026-02-21T22:19:45.425 INFO:tasks.rados.rados.0.trial178.stdout:865: done (0 left) 2026-02-21T22:19:45.425 INFO:tasks.rados.rados.0.trial178.stdout:866: rollback oid 26 current snap is 91 2026-02-21T22:19:45.425 INFO:tasks.rados.rados.0.trial178.stdout:866: done (0 left) 2026-02-21T22:19:45.425 INFO:tasks.rados.rados.0.trial178.stdout:867: copy_from oid 22 from oid 35 current snap is 91 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:868: read oid 14 snap -1 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:868: expect (ObjNum 311 snap 91 seq_num 311) 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:869: read oid 32 snap -1 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:869: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:870: read oid 48 snap -1 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:870: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:871: read oid 7 snap -1 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:871: expect deleted 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:872: write (excl) oid 9 current snap is 91 2026-02-21T22:19:45.426 INFO:tasks.rados.rados.0.trial178.stdout:872: seq_num 312 ranges {443784=648161,1657676=659775,2818209=425575} 2026-02-21T22:19:45.432 INFO:tasks.rados.rados.0.trial178.stdout:872: writing trial17818531-9 from 443784 to 1091945 tid 1 2026-02-21T22:19:45.440 INFO:tasks.rados.rados.0.trial178.stdout:872: writing trial17818531-9 from 1657676 to 2317451 tid 2 2026-02-21T22:19:45.447 INFO:tasks.rados.rados.0.trial178.stdout:872: writing trial17818531-9 from 2818209 to 3243784 tid 3 2026-02-21T22:19:45.447 INFO:tasks.rados.rados.0.trial178.stdout:873: copy_from oid 24 from oid 47 current snap is 91 2026-02-21T22:19:45.447 INFO:tasks.rados.rados.0.trial178.stdout:870: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:45.510 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2669624 2026-02-21T22:19:45.571 INFO:tasks.rados.rados.0.trial178.stdout:868: expect (ObjNum 311 snap 91 seq_num 311) 2026-02-21T22:19:45.571 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:45.625 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2038543 2026-02-21T22:19:45.678 INFO:tasks.rados.rados.0.trial178.stdout:869: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:19:45.699 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2360654 2026-02-21T22:19:45.718 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 166659 2026-02-21T22:19:45.804 INFO:tasks.rados.rados.0.trial178.stdout:872: finishing write tid 1 to trial17818531-9 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:872: finishing write tid 2 to trial17818531-9 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:868: done (6 left) 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:869: done (5 left) 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:870: done (4 left) 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:871: done (3 left) 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:874: snap_create 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:867: finishing copy_from racing read to trial17818531-22 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:872: finishing write tid 3 to trial17818531-9 2026-02-21T22:19:45.805 INFO:tasks.rados.rados.0.trial178.stdout:872: finishing write tid 4 to trial17818531-9 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: finishing write tid 5 to trial17818531-9 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: oid 9 updating version 0 to 343 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: oid 9 version 343 is already newer than 342 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: oid 9 updating version 343 to 345 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: oid 9 version 345 is already newer than 344 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 345 (ObjNum 312 snap 91 seq_num 312) dirty exists 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:872: left oid 9 (ObjNum 312 snap 91 seq_num 312) 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:867: finishing copy_from to trial17818531-22 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:873: finishing copy_from racing read to trial17818531-24 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:873: finishing copy_from to trial17818531-24 2026-02-21T22:19:45.806 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 437 (ObjNum 232 snap 72 seq_num 232) dirty exists 2026-02-21T22:19:46.432 INFO:tasks.rados.rados.0.trial178.stdout:867: done (3 left) 2026-02-21T22:19:46.432 INFO:tasks.rados.rados.0.trial178.stdout:872: done (2 left) 2026-02-21T22:19:46.432 INFO:tasks.rados.rados.0.trial178.stdout:873: done (1 left) 2026-02-21T22:19:46.433 INFO:tasks.rados.rados.0.trial178.stdout:874: done (0 left) 2026-02-21T22:19:46.433 INFO:tasks.rados.rados.0.trial178.stdout:875: write oid 13 current snap is 92 2026-02-21T22:19:46.433 INFO:tasks.rados.rados.0.trial178.stdout:875: seq_num 313 ranges {460591=723410,1716158=403433,2729279=440140} 2026-02-21T22:19:46.440 INFO:tasks.rados.rados.0.trial178.stdout:875: writing trial17818531-13 from 460591 to 1184001 tid 1 2026-02-21T22:19:46.447 INFO:tasks.rados.rados.0.trial178.stdout:875: writing trial17818531-13 from 1716158 to 2119591 tid 2 2026-02-21T22:19:46.454 INFO:tasks.rados.rados.0.trial178.stdout:875: writing trial17818531-13 from 2729279 to 3169419 tid 3 2026-02-21T22:19:46.455 INFO:tasks.rados.rados.0.trial178.stdout:876: read oid 33 snap -1 2026-02-21T22:19:46.455 INFO:tasks.rados.rados.0.trial178.stdout:876: expect deleted 2026-02-21T22:19:46.455 INFO:tasks.rados.rados.0.trial178.stdout:877: snap_remove snap 91 2026-02-21T22:19:46.455 INFO:tasks.rados.rados.0.trial178.stdout:875: finishing write tid 1 to trial17818531-13 2026-02-21T22:19:46.455 INFO:tasks.rados.rados.0.trial178.stdout:875: finishing write tid 2 to trial17818531-13 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:877: done (2 left) 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:878: delete oid 4 current snap is 92 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: finishing write tid 3 to trial17818531-13 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: finishing write tid 4 to trial17818531-13 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: finishing write tid 5 to trial17818531-13 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: oid 13 updating version 0 to 456 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: oid 13 updating version 456 to 459 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: oid 13 version 459 is already newer than 458 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: oid 13 version 459 is already newer than 457 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 459 (ObjNum 313 snap 92 seq_num 313) dirty exists 2026-02-21T22:19:47.405 INFO:tasks.rados.rados.0.trial178.stdout:875: left oid 13 (ObjNum 313 snap 92 seq_num 313) 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:875: done (2 left) 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:876: done (1 left) 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:878: done (0 left) 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:879: read oid 15 snap -1 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:879: expect deleted 2026-02-21T22:19:47.406 INFO:tasks.rados.rados.0.trial178.stdout:880: rollback oid 18 current snap is 92 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:880: done (1 left) 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:881: rollback oid 9 current snap is 92 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:881: done (1 left) 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:882: read oid 47 snap -1 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:882: expect (ObjNum 232 snap 72 seq_num 232) 2026-02-21T22:19:47.407 INFO:tasks.rados.rados.0.trial178.stdout:883: snap_create 2026-02-21T22:19:47.415 INFO:tasks.rados.rados.0.trial178.stdout:882: expect (ObjNum 232 snap 72 seq_num 232) 2026-02-21T22:19:47.415 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:19:47.497 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1105718 2026-02-21T22:19:48.104 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:19:48.104 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:19:48.104 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:19:48.104 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:19:48.107 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:19:48.107 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:19:48.107 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: get_monmap_and_config 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: monmap: 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:48.187914+0000 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:48.187914+0000 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:48.187 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 1 Processor -- start 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 1 -- start start 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: init 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-b con 0x7f82e408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-a con 0x7f82e408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-c con 0x7f82e408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e4086a10 con 0x7f82e408b7f0 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e408b130 con 0x7f82e408d8e0 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e408bd60 con 0x7f82e408ce00 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e8fce640 10 monclient(hunting): authenticate will time out at 1397.423798s 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e1d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 0x7f82e4181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58484/0 (socket says 10.20.193.102:58484) 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e1d74640 1 -- 10.20.193.102:0/778766543 learned_addr learned my addr 10.20.193.102:0/778766543 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.184+0000 7f82e2575640 1 --1- 10.20.193.102:0/778766543 >> v1:10.20.193.102:6790/0 conn(0x7f82e408ce00 0x7f82e408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49658/0 (socket says 10.20.193.102:49658) 2026-02-21T22:19:48.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.188+0000 7f82e1d74640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 legacy=0x7f82e4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.188+0000 7f82e2575640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.102:6790/0 conn(0x7f82e408ce00 legacy=0x7f82e408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.188+0000 7f82e2d76640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.178:6789/0 conn(0x7f82e408d8e0 legacy=0x7f82e41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859820628 0 0) 0x7f82e408bd60 con 0x7f82e408ce00 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient(hunting): my global_id is 4544 2026-02-21T22:19:48.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82d0003620 con 0x7f82e408ce00 2026-02-21T22:19:48.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3016801487 0 0) 0x7f82d0003620 con 0x7f82e408ce00 2026-02-21T22:19:48.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.178:6789/0 conn(0x7f82e408d8e0 legacy=0x7f82e41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 legacy=0x7f82e4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 monclient: found mon.noname-b 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e408b370 con 0x7f82e408ce00 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f82d8003110 con 0x7f82e408ce00 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: dump: 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.210 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 5 monclient: authenticate success, global_id 4544 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f82d80033e0 con 0x7f82e408ce00 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 1 -- 10.20.193.102:0/778766543 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f82d8004a90 con 0x7f82e408ce00 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e1573640 10 monclient: dump: 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.211 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 4 set_mon_vals no callback set 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: shutdown 2026-02-21T22:19:48.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- 10.20.193.102:0/778766543 >> v1:10.20.193.102:6790/0 conn(0x7f82e408ce00 legacy=0x7f82e408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- 10.20.193.102:0/778766543 shutdown_connections 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- 10.20.193.102:0/778766543 >> 10.20.193.102:0/778766543 conn(0x7f82e4091a50 msgr2=0x7f82e4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- 10.20.193.102:0/778766543 shutdown_connections 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- 10.20.193.102:0/778766543 wait complete. 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: monmap: 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:48.213 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 Processor -- start 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- start start 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: init 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:19:48.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-a con 0x7f82e408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-b con 0x7f82e408ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient(hunting): picked mon.noname-c con 0x7f82e408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e40820b0 con 0x7f82e408b7f0 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e40833f0 con 0x7f82e408ce00 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e4058a40 con 0x7f82e408d8e0 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e8fce640 10 monclient(hunting): authenticate will time out at 1397.443562s 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e2575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 0x7f82e41531d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58498/0 (socket says 10.20.193.102:58498) 2026-02-21T22:19:48.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.204+0000 7f82e2575640 1 -- 10.20.193.102:0/4082138847 learned_addr learned my addr 10.20.193.102:0/4082138847 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e2575640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 legacy=0x7f82e41531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e2d76640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.102:6790/0 conn(0x7f82e408d8e0 legacy=0x7f82e4195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e1d74640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.178:6789/0 conn(0x7f82e408ce00 legacy=0x7f82e4084920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392355516 0 0) 0x7f82e4058a40 con 0x7f82e408d8e0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): my global_id is 4547 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82b0003620 con 0x7f82e408d8e0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2155110786 0 0) 0x7f82b0003620 con 0x7f82e408d8e0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f82e4058a40 con 0x7f82e408d8e0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f82d4004890 con 0x7f82e408d8e0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:48.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): dump: 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:48.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 900339902 0 0) 0x7f82e4058a40 con 0x7f82e408d8e0 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.178:6789/0 conn(0x7f82e408ce00 legacy=0x7f82e4084920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.102:6789/0 conn(0x7f82e408b7f0 legacy=0x7f82e41531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 monclient: found mon.c 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e405e150 con 0x7f82e408d8e0 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 5 monclient: authenticate success, global_id 4547 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _renew_subs 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f82e4088a70 con 0x7f82e408d8e0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _renew_subs 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f82e4085030 con 0x7f82e408d8e0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f82d4004ed0 con 0x7f82e408d8e0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f82d40052a0 con 0x7f82e408d8e0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 10 monclient: dump: 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e0d72640 4 set_mon_vals no callback set 2026-02-21T22:19:48.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e40008e0 con 0x7f82e408d8e0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f82d4005570 con 0x7f82e408d8e0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(208..208 src has 1..208) ==== 4019+0+0 (unknown 4017025395 0 0) 0x7f82d404db00 con 0x7f82e408d8e0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.208+0000 7f82e2575640 10 monclient: get_auth_request con 0x7f82b003d450 auth_method 0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.212+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f82d4018aa0 con 0x7f82e408d8e0 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.212+0000 7f82caffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.212+0000 7f82caffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82e8fce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82e8fce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82e8fce640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f82e4082d50 con 0x7f82e408d8e0 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82caffd640 1 -- 10.20.193.102:0/4082138847 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3053867646) 0x7f82d4018380 con 0x7f82e408d8e0 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82caffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:48.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.328+0000 7f82caffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:48.332 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:19:48.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.178:6800/2549949439 conn(0x7f82b003d450 legacy=0x7f82b005d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 10 monclient: shutdown 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 >> v1:10.20.193.102:6790/0 conn(0x7f82e408d8e0 legacy=0x7f82e4195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 shutdown_connections 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 >> 10.20.193.102:0/4082138847 conn(0x7f82e4091a50 msgr2=0x7f82e40575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 shutdown_connections 2026-02-21T22:19:48.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.332+0000 7f82e8fce640 1 -- 10.20.193.102:0/4082138847 wait complete. 2026-02-21T22:19:48.342 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:19:48.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:19:48.342 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:19:48.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 10 monclient: get_monmap_and_config 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 10 monclient: monmap: 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:48.412222+0000 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:48.412222+0000 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 1 Processor -- start 2026-02-21T22:19:48.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 1 -- start start 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.408+0000 7ff7d68fd640 10 monclient: init 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-b con 0x7ff7d0091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-c con 0x7ff7d00909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-a con 0x7ff7d01854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d0057a20 con 0x7ff7d01854b0 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d0091d20 con 0x7ff7d00909a0 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d0057740 con 0x7ff7d0091240 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient(hunting): authenticate will time out at 1397.648136s 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7cffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 0x7ff7d0091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49674/0 (socket says 10.20.193.102:49674) 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7cffff640 1 -- 10.20.193.102:0/2120953038 learned_addr learned my addr 10.20.193.102:0/2120953038 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d4e73640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff7d01854b0 0x7ff7d01a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58514/0 (socket says 10.20.193.102:58514) 2026-02-21T22:19:48.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7cffff640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 legacy=0x7ff7d0091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d4e73640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.102:6789/0 conn(0x7ff7d01854b0 legacy=0x7ff7d01a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7cf7fe640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.178:6789/0 conn(0x7ff7d00909a0 legacy=0x7ff7d0183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491716900 0 0) 0x7ff7d0057740 con 0x7ff7d0091240 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): my global_id is 4550 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7b4003620 con 0x7ff7d0091240 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506969100 0 0) 0x7ff7d0057a20 con 0x7ff7d01854b0 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient(hunting): my global_id is 34200 2026-02-21T22:19:48.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d0057740 con 0x7ff7d01854b0 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3837401405 0 0) 0x7ff7b4003620 con 0x7ff7d0091240 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.178:6789/0 conn(0x7ff7d00909a0 legacy=0x7ff7d0183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.102:6789/0 conn(0x7ff7d01854b0 legacy=0x7ff7d01a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 monclient: found mon.noname-b 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0090dc0 con 0x7ff7d0091240 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7bc003110 con 0x7ff7d0091240 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 5 monclient: authenticate success, global_id 4550 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: dump: 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.421 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff7bc0033e0 con 0x7ff7d0091240 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 1 -- 10.20.193.102:0/2120953038 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7bc004b50 con 0x7ff7d0091240 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7ceffd640 10 monclient: dump: 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:48.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 4 set_mon_vals no callback set 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 10 monclient: shutdown 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- 10.20.193.102:0/2120953038 >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 legacy=0x7ff7d0091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- 10.20.193.102:0/2120953038 shutdown_connections 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- 10.20.193.102:0/2120953038 >> 10.20.193.102:0/2120953038 conn(0x7ff7d005e660 msgr2=0x7ff7d0095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- 10.20.193.102:0/2120953038 shutdown_connections 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.412+0000 7ff7d68fd640 1 -- 10.20.193.102:0/2120953038 wait complete. 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: monmap: 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:48.424 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 Processor -- start 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- start start 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: init 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-b con 0x7ff7d00909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-c con 0x7ff7d0091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient(hunting): picked mon.noname-a con 0x7ff7d01854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d01123e0 con 0x7ff7d01854b0 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d012c000 con 0x7ff7d00909a0 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d00592d0 con 0x7ff7d0091240 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient(hunting): authenticate will time out at 1397.652853s 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7cf7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 0x7ff7d0172cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49682/0 (socket says 10.20.193.102:49682) 2026-02-21T22:19:48.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7cf7fe640 1 -- 10.20.193.102:0/273404548 learned_addr learned my addr 10.20.193.102:0/273404548 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7cf7fe640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 legacy=0x7ff7d0172cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d4e73640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d01854b0 legacy=0x7ff7d01743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843500470 0 0) 0x7ff7d00592d0 con 0x7ff7d0091240 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): my global_id is 4553 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7a0003620 con 0x7ff7d0091240 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063038454 0 0) 0x7ff7d01123e0 con 0x7ff7d01854b0 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): my global_id is 34203 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d00592d0 con 0x7ff7d01854b0 2026-02-21T22:19:48.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7cffff640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.178:6789/0 conn(0x7ff7d00909a0 legacy=0x7ff7d012b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1158362069 0 0) 0x7ff7a0003620 con 0x7ff7d0091240 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7d01123e0 con 0x7ff7d0091240 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c0003420 con 0x7ff7d0091240 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): dump: 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.433 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1007871579 0 0) 0x7ff7d00592d0 con 0x7ff7d01854b0 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7a0003620 con 0x7ff7d01854b0 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c40048b0 con 0x7ff7d01854b0 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient(hunting): dump: 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.434 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 320992836 0 0) 0x7ff7d01123e0 con 0x7ff7d0091240 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.178:6789/0 conn(0x7ff7d00909a0 legacy=0x7ff7d012b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d01854b0 legacy=0x7ff7d01743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 monclient: found mon.c 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0110ef0 con 0x7ff7d0091240 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 5 monclient: authenticate success, global_id 4553 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _renew_subs 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d01132b0 con 0x7ff7d0091240 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _renew_subs 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff7d0111c20 con 0x7ff7d0091240 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff7c0003a30 con 0x7ff7d0091240 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c0003d90 con 0x7ff7d0091240 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ccff9640 10 monclient: dump: 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.416+0000 7ff7ce7fc640 4 set_mon_vals no callback set 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7d68fd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff7d012ec60 con 0x7ff7d012e890 2026-02-21T22:19:48.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff7c0004020 con 0x7ff7d0091240 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(208..208 src has 1..208) ==== 4019+0+0 (unknown 4017025395 0 0) 0x7ff7c004c810 con 0x7ff7d0091240 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7d4e73640 10 monclient: get_auth_request con 0x7ff7a003d6a0 auth_method 0 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7cffff640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d012e890 legacy=0x7ff7d008cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180489183 0 0) 0x7ff7d012ec60 con 0x7ff7d012e890 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient(hunting): my global_id is 34206 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d01123e0 con 0x7ff7d012e890 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 335707065 0 0) 0x7ff7d01123e0 con 0x7ff7d012e890 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7d012ec60 con 0x7ff7d012e890 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7bc002b30 con 0x7ff7d012e890 2026-02-21T22:19:48.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient: dump: 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2213423873 0 0) 0x7ff7d012ec60 con 0x7ff7d012e890 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7d0092340 con 0x7ff7d012e890 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ff7bc003450 con 0x7ff7d012e890 2026-02-21T22:19:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d012e890 legacy=0x7ff7d008cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.420+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d012e890 legacy=0x7ff7d008cbe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.424+0000 7ff7d68fd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.424+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff7d005eb70 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7cffff640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d008d2f0 legacy=0x7ff7d0176b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2811518655 0 0) 0x7ff7d005eb70 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient(hunting): my global_id is 34209 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d012ec60 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4244721487 0 0) 0x7ff7d012ec60 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7d005eb70 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7bc005000 con 0x7ff7d008d2f0 2026-02-21T22:19:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient: dump: 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 797281710 0 0) 0x7ff7d005eb70 con 0x7ff7d008d2f0 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff7d0095010 con 0x7ff7d008d2f0 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 3518167997) 0x7ff7bc005210 con 0x7ff7d008d2f0 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d008d2f0 legacy=0x7ff7d0176b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7ccff9640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6789/0 conn(0x7ff7d008d2f0 legacy=0x7ff7d0176b00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:19:48.442 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 137297, 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.443 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.444 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:48.445 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:48.446 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.447 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:48.448 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.449 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.450 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:48.451 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:879: done (2 left) 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:882: done (1 left) 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:883: done (0 left) 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:884: rollback oid 29 current snap is 93 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 92 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:885: copy_from oid 5 from oid 2 current snap is 93 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:886: snap_create 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:885: finishing copy_from to trial17818531-5 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:885: got expected ENOENT (src dne) 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:885: finishing copy_from racing read to trial17818531-5 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:884: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:884: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:884: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:19:48.452 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 149 (ObjNum 299 snap 88 seq_num 299) dirty exists 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff7a003d6a0 legacy=0x7ff7a005db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 10 monclient: shutdown 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 >> v1:10.20.193.102:6790/0 conn(0x7ff7d0091240 legacy=0x7ff7d0172cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d4e73640 1 -- 10.20.193.102:0/273404548 reap_dead start 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 shutdown_connections 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 >> 10.20.193.102:0/273404548 conn(0x7ff7d005e660 msgr2=0x7ff7d008bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 shutdown_connections 2026-02-21T22:19:48.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.428+0000 7ff7d68fd640 1 -- 10.20.193.102:0/273404548 wait complete. 2026-02-21T22:19:48.453 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:48.453 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:19:48.454 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:19:48.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: get_monmap_and_config 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: monmap: 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:48.523542+0000 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:48.523542+0000 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:48.524 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 1 Processor -- start 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 1 -- start start 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: init 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient(hunting): picked mon.noname-c con 0x7f415008c4e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient(hunting): picked mon.noname-a con 0x7f415008d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient(hunting): picked mon.noname-b con 0x7f415008d890 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415008bd50 con 0x7f415008d410 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415008c8b0 con 0x7f415008c4e0 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4150083560 con 0x7f415008d890 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f41581af640 10 monclient(hunting): authenticate will time out at 1397.759445s 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f4155723640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f415008d410 0x7f4150188a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58558/0 (socket says 10.20.193.102:58558) 2026-02-21T22:19:48.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.520+0000 7f4155723640 1 -- 10.20.193.102:0/4267203261 learned_addr learned my addr 10.20.193.102:0/4267203261 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4155723640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.102:6789/0 conn(0x7f415008d410 legacy=0x7f4150188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4156725640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.102:6790/0 conn(0x7f415008d890 legacy=0x7f41501aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4155f24640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.178:6789/0 conn(0x7f415008c4e0 legacy=0x7f415008cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656662351 0 0) 0x7f414c0041c0 con 0x7f415008d890 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient(hunting): my global_id is 4556 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4138003620 con 0x7f415008d890 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 798950775 0 0) 0x7f4138003620 con 0x7f415008d890 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.178:6789/0 conn(0x7f415008c4e0 legacy=0x7f415008cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.102:6789/0 conn(0x7f415008d410 legacy=0x7f4150188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 monclient: found mon.noname-b 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4150084bc0 con 0x7f415008d890 2026-02-21T22:19:48.529 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f414c004840 con 0x7f415008d890 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 5 monclient: authenticate success, global_id 4556 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: dump: 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.530 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f414c004c20 con 0x7f415008d890 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 1 -- 10.20.193.102:0/4267203261 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f414c004fb0 con 0x7f415008d890 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f4154f22640 10 monclient: dump: 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.531 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 4 set_mon_vals no callback set 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 10 monclient: shutdown 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- 10.20.193.102:0/4267203261 >> v1:10.20.193.102:6790/0 conn(0x7f415008d890 legacy=0x7f41501aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- 10.20.193.102:0/4267203261 shutdown_connections 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- 10.20.193.102:0/4267203261 >> 10.20.193.102:0/4267203261 conn(0x7f4150091a50 msgr2=0x7f4150091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- 10.20.193.102:0/4267203261 shutdown_connections 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- 10.20.193.102:0/4267203261 wait complete. 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 10 monclient: monmap: 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.533 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 Processor -- start 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 1 -- start start 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.524+0000 7f41581af640 10 monclient: init 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient(hunting): picked mon.noname-a con 0x7f415008c4e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient(hunting): picked mon.noname-c con 0x7f415008d410 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient(hunting): picked mon.noname-b con 0x7f415008d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4150088a70 con 0x7f415008c4e0 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415005ccb0 con 0x7f415008d890 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4150059cb0 con 0x7f415008d410 2026-02-21T22:19:48.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f41581af640 10 monclient(hunting): authenticate will time out at 1397.764241s 2026-02-21T22:19:48.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f4155f24640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f415008c4e0 0x7f4150058e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58560/0 (socket says 10.20.193.102:58560) 2026-02-21T22:19:48.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f4155f24640 1 -- 10.20.193.102:0/1239166355 learned_addr learned my addr 10.20.193.102:0/1239166355 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f4155f24640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6789/0 conn(0x7f415008c4e0 legacy=0x7f4150058e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f4155723640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f415008d410 legacy=0x7f41500595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756710980 0 0) 0x7f4150059cb0 con 0x7f415008d410 2026-02-21T22:19:48.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): my global_id is 4559 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4120003620 con 0x7f415008d410 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1416573006 0 0) 0x7f4120003620 con 0x7f415008d410 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4150059cb0 con 0x7f415008d410 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4144003330 con 0x7f415008d410 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): dump: 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.539 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.528+0000 7f4156725640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.178:6789/0 conn(0x7f415008d890 legacy=0x7f41502410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2713008714 0 0) 0x7f4150059cb0 con 0x7f415008d410 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.178:6789/0 conn(0x7f415008d890 legacy=0x7f41502410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6789/0 conn(0x7f415008c4e0 legacy=0x7f4150058e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 monclient: found mon.c 2026-02-21T22:19:48.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415005bdd0 con 0x7f415008d410 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 5 monclient: authenticate success, global_id 4559 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 10 monclient: _renew_subs 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f415005a950 con 0x7f415008d410 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 10 monclient: _renew_subs 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f415005b710 con 0x7f415008d410 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4144004b90 con 0x7f415008d410 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4144004f30 con 0x7f415008d410 2026-02-21T22:19:48.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413ffff640 4 set_mon_vals no callback set 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 10 monclient: dump: 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f41500008e0 con 0x7f4150089d70 2026-02-21T22:19:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f41440051d0 con 0x7f415008d410 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(209..209 src has 1..209) ==== 3931+0+0 (unknown 3603405359 0 0) 0x7f414404c710 con 0x7f415008d410 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f4156725640 10 monclient: get_auth_request con 0x7f412003d470 auth_method 0 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.532+0000 7f4155f24640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f4150089d70 legacy=0x7f415008a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168376528 0 0) 0x7f41500008e0 con 0x7f4150089d70 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient(hunting): my global_id is 4562 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4150059cb0 con 0x7f4150089d70 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1937707807 0 0) 0x7f4150059cb0 con 0x7f4150089d70 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f41500008e0 con 0x7f4150089d70 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f41400032b0 con 0x7f4150089d70 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient: dump: 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3010987464 0 0) 0x7f41500008e0 con 0x7f4150089d70 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f415005e4a0 con 0x7f4150089d70 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f41400058c0 con 0x7f4150089d70 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:48.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f4150089d70 legacy=0x7f415008a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f4150089d70 legacy=0x7f415008a140 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.536+0000 7f41581af640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f415005e650 con 0x7f415008a850 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f4155f24640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f415008a850 legacy=0x7f41502637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987242121 0 0) 0x7f415005e650 con 0x7f415008a850 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient(hunting): my global_id is 4565 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41500008e0 con 0x7f415008a850 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 210695703 0 0) 0x7f41500008e0 con 0x7f415008a850 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f415005e650 con 0x7f415008a850 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4140003af0 con 0x7f415008a850 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient: dump: 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.547 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1613403630 0 0) 0x7f415005e650 con 0x7f415008a850 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f415008ac50 con 0x7f415008a850 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3974039241) 0x7f4140005b20 con 0x7f415008a850 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f415008a850 legacy=0x7f41502637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f413e7fc640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f415008a850 legacy=0x7f41502637e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:48.548 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 271170, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:48.549 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.550 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.551 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:48.552 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:48.553 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.554 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.555 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:48.556 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.557 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:48.558 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:48.559 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.178:6800/2549949439 conn(0x7f412003d470 legacy=0x7f412005d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 10 monclient: shutdown 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 >> v1:10.20.193.102:6790/0 conn(0x7f415008d410 legacy=0x7f41500595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f4156725640 1 -- 10.20.193.102:0/1239166355 reap_dead start 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 shutdown_connections 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 >> 10.20.193.102:0/1239166355 conn(0x7f4150091a50 msgr2=0x7f415008ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 shutdown_connections 2026-02-21T22:19:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.540+0000 7f41581af640 1 -- 10.20.193.102:0/1239166355 wait complete. 2026-02-21T22:19:48.561 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:48.561 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:19:48.561 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:19:48.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:48.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: get_monmap_and_config 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: monmap: 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:48.631769+0000 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:48.631769+0000 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:48.631 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 1 Processor -- start 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 1 -- start start 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: init 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-a con 0x7f935c08e9d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-b con 0x7f935c08f4b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-c con 0x7f935c1854e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c091cd0 con 0x7f935c08e9d0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c05e2c0 con 0x7f935c1854e0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c08cba0 con 0x7f935c08f4b0 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9363a21640 10 monclient(hunting): authenticate will time out at 1397.867694s 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9360f95640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f935c08f4b0 0x7f935c183dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49718/0 (socket says 10.20.193.102:49718) 2026-02-21T22:19:48.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.628+0000 7f9360f95640 1 -- 10.20.193.102:0/3998902608 learned_addr learned my addr 10.20.193.102:0/3998902608 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9360f95640 1 -- 10.20.193.102:0/3998902608 >> v1:10.20.193.102:6790/0 conn(0x7f935c08f4b0 legacy=0x7f935c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9361796640 1 -- 10.20.193.102:0/3998902608 >> v1:10.20.193.102:6789/0 conn(0x7f935c08e9d0 legacy=0x7f935c08eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082160504 0 0) 0x7f935c08cba0 con 0x7f935c08f4b0 2026-02-21T22:19:48.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient(hunting): my global_id is 4568 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9344003620 con 0x7f935c08f4b0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389282757 0 0) 0x7f9344003620 con 0x7f935c08f4b0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 >> v1:10.20.193.178:6789/0 conn(0x7f935c1854e0 legacy=0x7f935c1a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 >> v1:10.20.193.102:6789/0 conn(0x7f935c08e9d0 legacy=0x7f935c08eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 monclient: found mon.noname-b 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c08f8d0 con 0x7f935c08f4b0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9350003110 con 0x7f935c08f4b0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 5 monclient: authenticate success, global_id 4568 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: dump: 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.635 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9350003450 con 0x7f935c08f4b0 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 1 -- 10.20.193.102:0/3998902608 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9350004b50 con 0x7f935c08f4b0 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f934bfff640 10 monclient: dump: 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 4 set_mon_vals no callback set 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 10 monclient: shutdown 2026-02-21T22:19:48.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- 10.20.193.102:0/3998902608 >> v1:10.20.193.102:6790/0 conn(0x7f935c08f4b0 legacy=0x7f935c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- 10.20.193.102:0/3998902608 shutdown_connections 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- 10.20.193.102:0/3998902608 >> 10.20.193.102:0/3998902608 conn(0x7f935c055730 msgr2=0x7f935c091010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- 10.20.193.102:0/3998902608 shutdown_connections 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- 10.20.193.102:0/3998902608 wait complete. 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 10 monclient: build_initial_monmap 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 10 monclient: monmap: 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:48.637 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 Processor -- start 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 1 -- start start 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.632+0000 7f9363a21640 10 monclient: init 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-c con 0x7f935c08e9d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-b con 0x7f935c08f4b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient(hunting): picked mon.noname-a con 0x7f935c1854e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c12e8e0 con 0x7f935c1854e0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c12c580 con 0x7f935c08f4b0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c12e040 con 0x7f935c08e9d0 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient(hunting): authenticate will time out at 1397.872216s 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9361796640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f935c08e9d0 0x7f935c18f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49720/0 (socket says 10.20.193.102:49720) 2026-02-21T22:19:48.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9361796640 1 -- 10.20.193.102:0/2599139889 learned_addr learned my addr 10.20.193.102:0/2599139889 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9361796640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.102:6790/0 conn(0x7f935c08e9d0 legacy=0x7f935c18f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9361f97640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.102:6789/0 conn(0x7f935c1854e0 legacy=0x7f935c131e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716790630 0 0) 0x7f935c12e040 con 0x7f935c08e9d0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): my global_id is 4571 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f932c003620 con 0x7f935c08e9d0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9360f95640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c08f4b0 legacy=0x7f935c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1047870969 0 0) 0x7f932c003620 con 0x7f935c08e9d0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f935c12e040 con 0x7f935c08e9d0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f934c003420 con 0x7f935c08e9d0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): dump: 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.640 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1527884103 0 0) 0x7f935c12e040 con 0x7f935c08e9d0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c08f4b0 legacy=0x7f935c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.102:6789/0 conn(0x7f935c1854e0 legacy=0x7f935c131e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 monclient: found mon.c 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c12e320 con 0x7f935c08e9d0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: _finish_auth 0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 5 monclient: authenticate success, global_id 4571 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _renew_subs 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f935c1906f0 con 0x7f935c08e9d0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _renew_subs 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f935c12d2b0 con 0x7f935c08e9d0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f934c0039c0 con 0x7f935c08e9d0 2026-02-21T22:19:48.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f934c003d20 con 0x7f935c08e9d0 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 10 monclient: dump: 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f934b7fe640 4 set_mon_vals no callback set 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f934c003ff0 con 0x7f935c08e9d0 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f935c1a6dc0 con 0x7f935c1a2070 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(209..209 src has 1..209) ==== 3931+0+0 (unknown 3603405359 0 0) 0x7f934c04c7b0 con 0x7f935c08e9d0 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.636+0000 7f9361f97640 10 monclient: get_auth_request con 0x7f932c03d450 auth_method 0 2026-02-21T22:19:48.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.640+0000 7f9360f95640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2070 legacy=0x7f935c1a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:48.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152497449 0 0) 0x7f935c1a6dc0 con 0x7f935c1a2070 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient(hunting): my global_id is 44104 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f935c12e040 con 0x7f935c1a2070 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 182888239 0 0) 0x7f935c12e040 con 0x7f935c1a2070 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f935c1a6dc0 con 0x7f935c1a2070 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f93500032f0 con 0x7f935c1a2070 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:48.736+0000 7f9349ffb640 10 monclient: dump: 2026-02-21T22:19:48.739 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:48.740 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1405379114 0 0) 0x7f935c1a6dc0 con 0x7f935c1a2070 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f935c0008e0 con 0x7f935c1a2070 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f93500034d0 con 0x7f935c1a2070 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2070 legacy=0x7f935c1a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.160+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2070 legacy=0x7f935c1a2440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:49.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.164+0000 7f9363a21640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:49.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.164+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f935c12e560 con 0x7f935c1a2b50 2026-02-21T22:19:49.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9360f95640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2b50 legacy=0x7f935c134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 105631682 0 0) 0x7f935c12e560 con 0x7f935c1a2b50 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient(hunting): my global_id is 44107 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f935c1a6dc0 con 0x7f935c1a2b50 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3475074793 0 0) 0x7f935c1a6dc0 con 0x7f935c1a2b50 2026-02-21T22:19:49.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f935c12e560 con 0x7f935c1a2b50 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9350005000 con 0x7f935c1a2b50 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient: dump: 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.172 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1084455843 0 0) 0x7f935c12e560 con 0x7f935c1a2b50 2026-02-21T22:19:49.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f935c1a2f50 con 0x7f935c1a2b50 2026-02-21T22:19:49.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 2396107019) 0x7f9350005220 con 0x7f935c1a2b50 2026-02-21T22:19:49.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:49.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:49.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2b50 legacy=0x7f935c134560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9349ffb640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6789/0 conn(0x7f935c1a2b50 legacy=0x7f935c134560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:49.175 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 1023, 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:19:49.176 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:49.177 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:49.178 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:49.179 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:49.180 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:49.181 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:49.182 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:49.183 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:49.184 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.185 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.186 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:49.188 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:49.189 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:49.190 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:49.191 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.178:6800/2549949439 conn(0x7f932c03d450 legacy=0x7f932c05d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 10 monclient: shutdown 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 >> v1:10.20.193.102:6790/0 conn(0x7f935c08e9d0 legacy=0x7f935c18f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9361f97640 1 -- 10.20.193.102:0/2599139889 reap_dead start 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9361f97640 1 -- 10.20.193.102:0/2599139889 reap_dead start 2026-02-21T22:19:49.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 shutdown_connections 2026-02-21T22:19:49.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 >> 10.20.193.102:0/2599139889 conn(0x7f935c055730 msgr2=0x7f935c1a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:49.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 shutdown_connections 2026-02-21T22:19:49.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.168+0000 7f9363a21640 1 -- 10.20.193.102:0/2599139889 wait complete. 2026-02-21T22:19:49.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:49.194 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:19:49.195 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: get_monmap_and_config 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: build_initial_monmap 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: monmap: 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.259 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:49.258857+0000 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:49.258857+0000 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 1 Processor -- start 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 1 -- start start 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: init 2026-02-21T22:19:49.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-b con 0x7f0e6413f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-c con 0x7f0e6413fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-a con 0x7f0e641401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e64092490 con 0x7f0e641401f0 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e640571f0 con 0x7f0e6413fd90 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e640574d0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e69294640 10 monclient(hunting): authenticate will time out at 1398.494775s 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e63fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0e6413f9c0 0x7f0e640906e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49726/0 (socket says 10.20.193.102:49726) 2026-02-21T22:19:49.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.256+0000 7f0e63fff640 1 -- 10.20.193.102:0/3658301056 learned_addr learned my addr 10.20.193.102:0/3658301056 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:49.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e63fff640 1 -- 10.20.193.102:0/3658301056 >> v1:10.20.193.102:6790/0 conn(0x7f0e6413f9c0 legacy=0x7f0e640906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e68a93640 1 -- 10.20.193.102:0/3658301056 >> v1:10.20.193.102:6789/0 conn(0x7f0e641401f0 legacy=0x7f0e641a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440212260 0 0) 0x7f0e640574d0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): my global_id is 4574 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e48003620 con 0x7f0e6413f9c0 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975405604 0 0) 0x7f0e64092490 con 0x7f0e641401f0 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient(hunting): my global_id is 34212 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e640574d0 con 0x7f0e641401f0 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3815406392 0 0) 0x7f0e48003620 con 0x7f0e6413f9c0 2026-02-21T22:19:49.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 >> v1:10.20.193.178:6789/0 conn(0x7f0e6413fd90 legacy=0x7f0e64183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 >> v1:10.20.193.102:6789/0 conn(0x7f0e641401f0 legacy=0x7f0e641a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 monclient: found mon.noname-b 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e640577b0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: _finish_auth 0 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e50003110 con 0x7f0e6413f9c0 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 5 monclient: authenticate success, global_id 4574 2026-02-21T22:19:49.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: dump: 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.268 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e500033e0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 1 -- 10.20.193.102:0/3658301056 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e50004b50 con 0x7f0e6413f9c0 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e62ffd640 10 monclient: dump: 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.269 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 4 set_mon_vals no callback set 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: shutdown 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- 10.20.193.102:0/3658301056 >> v1:10.20.193.102:6790/0 conn(0x7f0e6413f9c0 legacy=0x7f0e640906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- 10.20.193.102:0/3658301056 shutdown_connections 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- 10.20.193.102:0/3658301056 >> 10.20.193.102:0/3658301056 conn(0x7f0e6405e3f0 msgr2=0x7f0e64095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- 10.20.193.102:0/3658301056 shutdown_connections 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- 10.20.193.102:0/3658301056 wait complete. 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: build_initial_monmap 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: monmap: 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:49.271 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 Processor -- start 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- start start 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: init 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-b con 0x7f0e6413f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-c con 0x7f0e6413fd90 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient(hunting): picked mon.noname-a con 0x7f0e641401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e64133630 con 0x7f0e641401f0 2026-02-21T22:19:49.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6408a2f0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e640590d0 con 0x7f0e6413fd90 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e69294640 10 monclient(hunting): authenticate will time out at 1398.499336s 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e637fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0e6413fd90 0x7f0e64194010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49740/0 (socket says 10.20.193.102:49740) 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e68a93640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0e641401f0 0x7f0e64195730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58612/0 (socket says 10.20.193.102:58612) 2026-02-21T22:19:49.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.260+0000 7f0e637fe640 1 -- 10.20.193.102:0/3948136516 learned_addr learned my addr 10.20.193.102:0/3948136516 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:49.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e637fe640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.102:6790/0 conn(0x7f0e6413fd90 legacy=0x7f0e64194010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e68a93640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.102:6789/0 conn(0x7f0e641401f0 legacy=0x7f0e64195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e63fff640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.178:6789/0 conn(0x7f0e6413f9c0 legacy=0x7f0e6413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:49.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132830310 0 0) 0x7f0e64133630 con 0x7f0e641401f0 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): my global_id is 34215 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e34003620 con 0x7f0e641401f0 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496539076 0 0) 0x7f0e640590d0 con 0x7f0e6413fd90 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): my global_id is 4577 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e64133630 con 0x7f0e6413fd90 2026-02-21T22:19:49.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021161103 0 0) 0x7f0e6408a2f0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): my global_id is 44110 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e640590d0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 758142735 0 0) 0x7f0e34003620 con 0x7f0e641401f0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e6408a2f0 con 0x7f0e641401f0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e58004930 con 0x7f0e641401f0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): dump: 2026-02-21T22:19:49.279 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2670541855 0 0) 0x7f0e64133630 con 0x7f0e6413fd90 2026-02-21T22:19:49.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e34003620 con 0x7f0e6413fd90 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e540043f0 con 0x7f0e6413fd90 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient(hunting): dump: 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.281 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 299368162 0 0) 0x7f0e640590d0 con 0x7f0e6413f9c0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e64133630 con 0x7f0e6413f9c0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3815686195 0 0) 0x7f0e6408a2f0 con 0x7f0e641401f0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.102:6790/0 conn(0x7f0e6413fd90 legacy=0x7f0e64194010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.178:6789/0 conn(0x7f0e6413f9c0 legacy=0x7f0e6413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 monclient: found mon.a 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e641312d0 con 0x7f0e641401f0 2026-02-21T22:19:49.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: _finish_auth 0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 5 monclient: authenticate success, global_id 34215 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _renew_subs 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0e64133350 con 0x7f0e641401f0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _renew_subs 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0e64152cb0 con 0x7f0e641401f0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e58004ed0 con 0x7f0e641401f0 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:49.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e58005260 con 0x7f0e641401f0 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 10 monclient: dump: 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:49.284 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e627fc640 4 set_mon_vals no callback set 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0e58005400 con 0x7f0e641401f0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6408b570 con 0x7f0e641401f0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(209..209 src has 1..209) ==== 3931+0+0 (unknown 3603405359 0 0) 0x7f0e5804d9c0 con 0x7f0e641401f0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.264+0000 7f0e63fff640 10 monclient: get_auth_request con 0x7f0e3403d970 auth_method 0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.268+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0e580189e0 con 0x7f0e641401f0 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.268+0000 7f0e60ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:49.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.268+0000 7f0e60ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.384+0000 7f0e69294640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:19:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.384+0000 7f0e69294640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:19:49.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.384+0000 7f0e69294640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.384+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0e6405eb70 con 0x7f0e641401f0 2026-02-21T22:19:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e60ff9640 1 -- 10.20.193.102:0/3948136516 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f0e58020200 con 0x7f0e641401f0 2026-02-21T22:19:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e60ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:19:49.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e60ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:49.389 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.389 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=2,health=13,kv=12,logm=62} crc {auth=1927021529,config=1618278320,health=1796179531,kv=543724905,logm=2498796387}) 2026-02-21T22:19:49.389 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=2,health=11,kv=10,logm=66} crc {auth=1927021529,config=1618278320,health=588522927,kv=273370847,logm=1308178458}) 2026-02-21T22:19:49.390 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.390 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=11,config=2,health=13,kv=12,logm=62} crc {auth=1927021529,config=1618278320,health=1796179531,kv=543724905,logm=2498796387}) 2026-02-21T22:19:49.390 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=11,config=2,health=13,kv=11,logm=63} crc {auth=2290419421,config=1618278320,health=1796179531,kv=2624703368,logm=504288263}) 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0e3403d970 legacy=0x7f0e3405de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 10 monclient: shutdown 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 >> v1:10.20.193.102:6789/0 conn(0x7f0e641401f0 legacy=0x7f0e64195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 shutdown_connections 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 >> 10.20.193.102:0/3948136516 conn(0x7f0e6405e3f0 msgr2=0x7f0e6408a820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 shutdown_connections 2026-02-21T22:19:49.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:49.388+0000 7f0e69294640 1 -- 10.20.193.102:0/3948136516 wait complete. 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3156693324}) 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2224152661}) 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3156693324}) 2026-02-21T22:19:49.392 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.388+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1500166477}) 2026-02-21T22:19:49.399 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:19:49.736 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.736 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1725570673}) 2026-02-21T22:19:49.736 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3817411355}) 2026-02-21T22:19:49.736 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.737 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1725570673}) 2026-02-21T22:19:49.737 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.736+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1862388276}) 2026-02-21T22:19:49.741 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.741 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=10197226}) 2026-02-21T22:19:49.742 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2226060383}) 2026-02-21T22:19:49.742 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.742 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=10197226}) 2026-02-21T22:19:49.742 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.740+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2606491414}) 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1244812024}) 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3987052141}) 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1244812024}) 2026-02-21T22:19:49.744 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:49.744+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=159141355}) 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:884: done (2 left) 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:885: done (1 left) 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:886: done (0 left) 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:887: copy_from oid 9 from oid 28 current snap is 94 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:888: read oid 33 snap -1 2026-02-21T22:19:49.746 INFO:tasks.rados.rados.0.trial178.stdout:888: expect deleted 2026-02-21T22:19:49.747 INFO:tasks.rados.rados.0.trial178.stdout:889: snap_remove snap 92 2026-02-21T22:19:50.159 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.159 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1629078337}) 2026-02-21T22:19:50.159 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4193047801}) 2026-02-21T22:19:50.160 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.160 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1629078337}) 2026-02-21T22:19:50.160 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.156+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3188985772}) 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3620885656}) 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=977896459}) 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3620885656}) 2026-02-21T22:19:50.162 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.160+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=266118559}) 2026-02-21T22:19:50.314 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.314 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=628312741}) 2026-02-21T22:19:50.314 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1836681961}) 2026-02-21T22:19:50.314 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.315 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=628312741}) 2026-02-21T22:19:50.315 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2462508655}) 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410243207}) 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3712957878}) 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=410243207}) 2026-02-21T22:19:50.316 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.312+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2614253166}) 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=62,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3} crc {logm=2936779203,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1411138142,mon_config_key=2300943063,monmap=918322475}) 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=1} crc {logm=3853977621,mdsmap=3074481156,mgr=369547122,mgr_metadata=2589708306,mgrstat=1603739711,mon_config_key=2063301084,monmap=3809027407}) 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=62,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=10,monmap=3} crc {logm=2936779203,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1411138142,mon_config_key=2300943063,monmap=918322475}) 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=63,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=1} crc {logm=3648768453,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2449124131,mon_config_key=2063301084,monmap=3809027407}) 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.318 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=55} crc {nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=4014561337,osd_snap=222877518,osdmap=1279629344}) 2026-02-21T22:19:50.319 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=53} crc {nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=4014561337,osd_snap=3643188150,osdmap=224350063}) 2026-02-21T22:19:50.319 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.319 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=35,osdmap=55} crc {nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=4014561337,osd_snap=222877518,osdmap=1279629344}) 2026-02-21T22:19:50.319 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=37,osdmap=53} crc {nvmeofgw=3360707893,osd_metadata=3232105792,osd_pg_creating=4014561337,osd_snap=896136687,osdmap=2217894}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=122046585}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3909957458}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=122046585}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=407953220}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=815456763}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2115356986}) 2026-02-21T22:19:50.321 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=815456763}) 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.316+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3776140335}) 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=62} crc {osdmap=1557313036}) 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=1145893514}) 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=62} crc {osdmap=1557313036}) 2026-02-21T22:19:50.322 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:19:50.320+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=61} crc {osdmap=4268489320}) 2026-02-21T22:19:50.400 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:19:50.400 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: get_monmap_and_config 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: monmap: 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.458 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:50.459294+0000 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:50.459294+0000 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 1 Processor -- start 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 1 -- start start 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: init 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-b con 0x7f7ec408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-c con 0x7f7ec408b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-a con 0x7f7ec408d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec4086a10 con 0x7f7ec408d8e0 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec408b130 con 0x7f7ec408b7f0 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec408bd60 con 0x7f7ec408ce00 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec95b6640 10 monclient(hunting): authenticate will time out at 1399.695167s 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec2ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7ec408ce00 0x7f7ec408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49750/0 (socket says 10.20.193.102:49750) 2026-02-21T22:19:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.456+0000 7f7ec2ffd640 1 -- 10.20.193.102:0/719850562 learned_addr learned my addr 10.20.193.102:0/719850562 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec37fe640 1 -- 10.20.193.102:0/719850562 >> v1:10.20.193.102:6789/0 conn(0x7f7ec408d8e0 legacy=0x7f7ec41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec2ffd640 1 -- 10.20.193.102:0/719850562 >> v1:10.20.193.102:6790/0 conn(0x7f7ec408ce00 legacy=0x7f7ec408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777085389 0 0) 0x7f7ec4086a10 con 0x7f7ec408d8e0 2026-02-21T22:19:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): my global_id is 34218 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb0003620 con 0x7f7ec408d8e0 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333692418 0 0) 0x7f7ec408bd60 con 0x7f7ec408ce00 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient(hunting): my global_id is 4580 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ec4086a10 con 0x7f7ec408ce00 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3781684822 0 0) 0x7f7ec4086a10 con 0x7f7ec408ce00 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 >> v1:10.20.193.178:6789/0 conn(0x7f7ec408b7f0 legacy=0x7f7ec4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 >> v1:10.20.193.102:6789/0 conn(0x7f7ec408d8e0 legacy=0x7f7ec41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 monclient: found mon.noname-b 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec408b370 con 0x7f7ec408ce00 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 5 monclient: authenticate success, global_id 4580 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7eb8003090 con 0x7f7ec408ce00 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: dump: 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7eb8003360 con 0x7f7ec408ce00 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 1 -- 10.20.193.102:0/719850562 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7eb80049a0 con 0x7f7ec408ce00 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec1ffb640 10 monclient: dump: 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 4 set_mon_vals no callback set 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: shutdown 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- 10.20.193.102:0/719850562 >> v1:10.20.193.102:6790/0 conn(0x7f7ec408ce00 legacy=0x7f7ec408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- 10.20.193.102:0/719850562 shutdown_connections 2026-02-21T22:19:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- 10.20.193.102:0/719850562 >> 10.20.193.102:0/719850562 conn(0x7f7ec4091a50 msgr2=0x7f7ec4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- 10.20.193.102:0/719850562 shutdown_connections 2026-02-21T22:19:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- 10.20.193.102:0/719850562 wait complete. 2026-02-21T22:19:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: monmap: 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 Processor -- start 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- start start 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: init 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-c con 0x7f7ec408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-b con 0x7f7ec408ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient(hunting): picked mon.noname-a con 0x7f7ec408d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec4088a70 con 0x7f7ec408d8e0 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec40827e0 con 0x7f7ec408ce00 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec4082ac0 con 0x7f7ec408b7f0 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec95b6640 10 monclient(hunting): authenticate will time out at 1399.699852s 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec2ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7ec408b7f0 0x7f7ec4153190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49760/0 (socket says 10.20.193.102:49760) 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec2ffd640 1 -- 10.20.193.102:0/641354177 learned_addr learned my addr 10.20.193.102:0/641354177 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.460+0000 7f7ec37fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7ec408d8e0 0x7f7ec4195ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58622/0 (socket says 10.20.193.102:58622) 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec37fe640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.102:6789/0 conn(0x7f7ec408d8e0 legacy=0x7f7ec4195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec2ffd640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.102:6790/0 conn(0x7f7ec408b7f0 legacy=0x7f7ec4153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1685107773 0 0) 0x7f7ec4088a70 con 0x7f7ec408d8e0 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): my global_id is 34221 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e90003620 con 0x7f7ec408d8e0 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218053330 0 0) 0x7f7ec4082ac0 con 0x7f7ec408b7f0 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): my global_id is 4583 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ec4088a70 con 0x7f7ec408b7f0 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec27fc640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.178:6789/0 conn(0x7f7ec408ce00 legacy=0x7f7ec41943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3854118478 0 0) 0x7f7e90003620 con 0x7f7ec408d8e0 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ec4082ac0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7eb4004960 con 0x7f7ec408d8e0 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): dump: 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2167003304 0 0) 0x7f7ec4088a70 con 0x7f7ec408b7f0 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7e90003620 con 0x7f7ec408b7f0 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530857225 0 0) 0x7f7ec40827e0 con 0x7f7ec408ce00 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): my global_id is 44113 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ec4088a70 con 0x7f7ec408ce00 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7eb8002b30 con 0x7f7ec408b7f0 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient(hunting): dump: 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.472 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1929497860 0 0) 0x7f7ec4082ac0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.102:6790/0 conn(0x7f7ec408b7f0 legacy=0x7f7ec4153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.178:6789/0 conn(0x7f7ec408ce00 legacy=0x7f7ec41943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 monclient: found mon.a 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec40818a0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 5 monclient: authenticate success, global_id 34221 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _renew_subs 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec408a6d0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _renew_subs 2026-02-21T22:19:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7ec4089f60 con 0x7f7ec408d8e0 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7eb4004ed0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7eb40052a0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 10 monclient: dump: 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec17fa640 4 set_mon_vals no callback set 2026-02-21T22:19:50.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f7eb4005480 con 0x7f7ec408d8e0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec40008e0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(210..210 src has 1..210) ==== 3931+0+0 (unknown 4231557359 0 0) 0x7f7eb404db30 con 0x7f7ec408d8e0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.464+0000 7f7ec2ffd640 10 monclient: get_auth_request con 0x7f7e9003d950 auth_method 0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.468+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f7eb4018b30 con 0x7f7ec408d8e0 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.468+0000 7f7ea37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.468+0000 7f7ea37fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7ec4082ea0 con 0x7f7ec408d8e0 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ea37fe640 1 -- 10.20.193.102:0/641354177 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2571281086) 0x7f7eb4018410 con 0x7f7ec408d8e0 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ea37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:50.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ea37fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:50.588 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.178:6800/2549949439 conn(0x7f7e9003d950 legacy=0x7f7e9005de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 10 monclient: shutdown 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 >> v1:10.20.193.102:6789/0 conn(0x7f7ec408d8e0 legacy=0x7f7ec4195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 shutdown_connections 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 >> 10.20.193.102:0/641354177 conn(0x7f7ec4091a50 msgr2=0x7f7ec405de10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 shutdown_connections 2026-02-21T22:19:50.589 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.584+0000 7f7ec95b6640 1 -- 10.20.193.102:0/641354177 wait complete. 2026-02-21T22:19:50.597 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:19:50.597 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:19:50.597 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:19:50.598 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:19:50.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: get_monmap_and_config 2026-02-21T22:19:50.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: monmap: 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:50.667885+0000 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:50.667885+0000 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 1 Processor -- start 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 1 -- start start 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: init 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-a con 0x7f495408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-b con 0x7f495408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-c con 0x7f495408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4954086a10 con 0x7f495408ce00 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495408b130 con 0x7f495408d8e0 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495408bd60 con 0x7f495408b7f0 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4958b58640 10 monclient(hunting): authenticate will time out at 1399.903812s 2026-02-21T22:19:50.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4952575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 0x7f495408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58630/0 (socket says 10.20.193.102:58630) 2026-02-21T22:19:50.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4951d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f495408b7f0 0x7f4954181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49772/0 (socket says 10.20.193.102:49772) 2026-02-21T22:19:50.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.664+0000 7f4952575640 1 -- 10.20.193.102:0/2219296572 learned_addr learned my addr 10.20.193.102:0/2219296572 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4952575640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 legacy=0x7f495408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951d74640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.102:6790/0 conn(0x7f495408b7f0 legacy=0x7f4954181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4952d76640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.178:6789/0 conn(0x7f495408d8e0 legacy=0x7f49541a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647954443 0 0) 0x7f4954086a10 con 0x7f495408ce00 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): my global_id is 34224 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4940003620 con 0x7f495408ce00 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887693645 0 0) 0x7f495408bd60 con 0x7f495408b7f0 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): my global_id is 4586 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954086a10 con 0x7f495408b7f0 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155912557 0 0) 0x7f495408b130 con 0x7f495408d8e0 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient(hunting): my global_id is 44116 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495408bd60 con 0x7f495408d8e0 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220127801 0 0) 0x7f4940003620 con 0x7f495408ce00 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.102:6790/0 conn(0x7f495408b7f0 legacy=0x7f4954181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.178:6789/0 conn(0x7f495408d8e0 legacy=0x7f49541a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 monclient: found mon.noname-a 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495408b370 con 0x7f495408ce00 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4948003110 con 0x7f495408ce00 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:50.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: dump: 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 5 monclient: authenticate success, global_id 34224 2026-02-21T22:19:50.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f49480033e0 con 0x7f495408ce00 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 1 -- 10.20.193.102:0/2219296572 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4948004b50 con 0x7f495408ce00 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4951573640 10 monclient: dump: 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.682 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 4 set_mon_vals no callback set 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 10 monclient: shutdown 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 1 -- 10.20.193.102:0/2219296572 >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 legacy=0x7f495408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 1 -- 10.20.193.102:0/2219296572 shutdown_connections 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 1 -- 10.20.193.102:0/2219296572 >> 10.20.193.102:0/2219296572 conn(0x7f4954091a50 msgr2=0x7f4954091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 1 -- 10.20.193.102:0/2219296572 shutdown_connections 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 1 -- 10.20.193.102:0/2219296572 wait complete. 2026-02-21T22:19:50.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.668+0000 7f4958b58640 10 monclient: monmap: 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 Processor -- start 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- start start 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: init 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-b con 0x7f495408b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-a con 0x7f495408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient(hunting): picked mon.noname-c con 0x7f495408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4954088a70 con 0x7f495408ce00 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4954083030 con 0x7f495408b7f0 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4954083310 con 0x7f495408d8e0 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient(hunting): authenticate will time out at 1399.908569s 2026-02-21T22:19:50.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4951d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 0x7f4954084840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58636/0 (socket says 10.20.193.102:58636) 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4951d74640 1 -- 10.20.193.102:0/1001014891 learned_addr learned my addr 10.20.193.102:0/1001014891 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4951d74640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 legacy=0x7f4954084840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4952d76640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6790/0 conn(0x7f495408d8e0 legacy=0x7f4954195410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910647370 0 0) 0x7f4954086a10 con 0x7f495408ce00 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): my global_id is 34227 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4952575640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.178:6789/0 conn(0x7f495408b7f0 legacy=0x7f49541530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4920003620 con 0x7f495408ce00 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994630458 0 0) 0x7f4954083310 con 0x7f495408d8e0 2026-02-21T22:19:50.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): my global_id is 4589 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954086a10 con 0x7f495408d8e0 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45753291 0 0) 0x7f4954083030 con 0x7f495408b7f0 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): my global_id is 44119 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954083310 con 0x7f495408b7f0 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1989496295 0 0) 0x7f4920003620 con 0x7f495408ce00 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4954083030 con 0x7f495408ce00 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f493c003400 con 0x7f495408ce00 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): dump: 2026-02-21T22:19:50.688 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.689 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1922916830 0 0) 0x7f4954086a10 con 0x7f495408d8e0 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4920003620 con 0x7f495408d8e0 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f49440048d0 con 0x7f495408d8e0 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient(hunting): dump: 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.691 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3985790883 0 0) 0x7f4954083030 con 0x7f495408ce00 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6790/0 conn(0x7f495408d8e0 legacy=0x7f4954195410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.178:6789/0 conn(0x7f495408b7f0 legacy=0x7f49541530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 monclient: found mon.a 2026-02-21T22:19:50.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4954081b10 con 0x7f495408ce00 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 5 monclient: authenticate success, global_id 34227 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _renew_subs 2026-02-21T22:19:50.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4954084f50 con 0x7f495408ce00 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _renew_subs 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4954196d00 con 0x7f495408ce00 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f493c004b90 con 0x7f495408ce00 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f493c004f20 con 0x7f495408ce00 2026-02-21T22:19:50.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f493affd640 10 monclient: dump: 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.695 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4950d72640 4 set_mon_vals no callback set 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.672+0000 7f4958b58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f495405e3f0 con 0x7f495407a370 2026-02-21T22:19:50.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f493c0051b0 con 0x7f495408ce00 2026-02-21T22:19:50.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(210..210 src has 1..210) ==== 3931+0+0 (unknown 4231557359 0 0) 0x7f493c04c6e0 con 0x7f495408ce00 2026-02-21T22:19:50.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f4952d76640 10 monclient: get_auth_request con 0x7f492003d800 auth_method 0 2026-02-21T22:19:50.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f4952575640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407a370 legacy=0x7f495407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756739576 0 0) 0x7f495405e3f0 con 0x7f495407a370 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient(hunting): my global_id is 34230 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4954083030 con 0x7f495407a370 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 626730981 0 0) 0x7f4954083030 con 0x7f495407a370 2026-02-21T22:19:50.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f495405e3f0 con 0x7f495407a370 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4948002f60 con 0x7f495407a370 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient: dump: 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.703 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3559092400 0 0) 0x7f495405e3f0 con 0x7f495407a370 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4954080fb0 con 0x7f495407a370 2026-02-21T22:19:50.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f49480048e0 con 0x7f495407a370 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407a370 legacy=0x7f495407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.676+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407a370 legacy=0x7f495407a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.680+0000 7f4958b58640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.680+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4954062ce0 con 0x7f495407ae50 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4952575640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407ae50 legacy=0x7f4954198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660206251 0 0) 0x7f4954062ce0 con 0x7f495407ae50 2026-02-21T22:19:50.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient(hunting): my global_id is 34233 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f495405e3f0 con 0x7f495407ae50 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 234804716 0 0) 0x7f495405e3f0 con 0x7f495407ae50 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4954062ce0 con 0x7f495407ae50 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f49480051b0 con 0x7f495407ae50 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient: dump: 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.707 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 281865085 0 0) 0x7f4954062ce0 con 0x7f495407ae50 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f495407b250 con 0x7f495407ae50 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 3187317709) 0x7f49480073d0 con 0x7f495407ae50 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407ae50 legacy=0x7f4954198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f493affd640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495407ae50 legacy=0x7f4954198000 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 139553, 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:50.709 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.710 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.711 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.712 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:50.713 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:50.714 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:50.715 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.716 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.717 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:50.718 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:50.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.178:6800/2549949439 conn(0x7f492003d800 legacy=0x7f492005dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 10 monclient: shutdown 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 >> v1:10.20.193.102:6789/0 conn(0x7f495408ce00 legacy=0x7f4954084840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4952d76640 1 -- 10.20.193.102:0/1001014891 reap_dead start 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4952d76640 1 -- 10.20.193.102:0/1001014891 reap_dead start 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 shutdown_connections 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 >> 10.20.193.102:0/1001014891 conn(0x7f4954091a50 msgr2=0x7f495408eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 shutdown_connections 2026-02-21T22:19:50.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.684+0000 7f4958b58640 1 -- 10.20.193.102:0/1001014891 wait complete. 2026-02-21T22:19:50.720 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:50.720 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:19:50.720 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:19:50.746 INFO:tasks.rados.rados.0.trial178.stdout:889: done (2 left) 2026-02-21T22:19:50.746 INFO:tasks.rados.rados.0.trial178.stdout:890: snap_create 2026-02-21T22:19:50.746 INFO:tasks.rados.rados.0.trial178.stdout:887: finishing copy_from to trial17818531-9 2026-02-21T22:19:50.746 INFO:tasks.rados.rados.0.trial178.stdout:887: got expected ENOENT (src dne) 2026-02-21T22:19:50.746 INFO:tasks.rados.rados.0.trial178.stdout:887: finishing copy_from racing read to trial17818531-9 2026-02-21T22:19:50.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: get_monmap_and_config 2026-02-21T22:19:50.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: monmap: 2026-02-21T22:19:50.792 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:50.791863+0000 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:50.791863+0000 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 1 Processor -- start 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 1 -- start start 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: init 2026-02-21T22:19:50.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-a con 0x7f9050056270 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-c con 0x7f905008d7a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-b con 0x7f9050056d50 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905008e280 con 0x7f9050056270 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905008e560 con 0x7f905008d7a0 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905008cc10 con 0x7f9050056d50 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f9055b46640 10 monclient(hunting): authenticate will time out at 1400.027776s 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f904f7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9050056270 0x7f9050056640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58678/0 (socket says 10.20.193.102:58678) 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f904ffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 0x7f90501a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49796/0 (socket says 10.20.193.102:49796) 2026-02-21T22:19:50.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.788+0000 7f904f7fe640 1 -- 10.20.193.102:0/154865571 learned_addr learned my addr 10.20.193.102:0/154865571 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904ffff640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 legacy=0x7f90501a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904f7fe640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.102:6789/0 conn(0x7f9050056270 legacy=0x7f9050056640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884284158 0 0) 0x7f905008cc10 con 0x7f9050056d50 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): my global_id is 4592 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9034003620 con 0x7f9050056d50 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1071372678 0 0) 0x7f905008e280 con 0x7f9050056270 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient(hunting): my global_id is 34236 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f905008cc10 con 0x7f9050056270 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904effd640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.178:6789/0 conn(0x7f905008d7a0 legacy=0x7f9050181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 263175742 0 0) 0x7f9034003620 con 0x7f9050056d50 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.178:6789/0 conn(0x7f905008d7a0 legacy=0x7f9050181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.102:6789/0 conn(0x7f9050056270 legacy=0x7f9050056640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 monclient: found mon.noname-b 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905008c280 con 0x7f9050056d50 2026-02-21T22:19:50.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 5 monclient: authenticate success, global_id 4592 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9044004620 con 0x7f9050056d50 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: dump: 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f90440048f0 con 0x7f9050056d50 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 1 -- 10.20.193.102:0/154865571 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9044005fa0 con 0x7f9050056d50 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 4 set_mon_vals no callback set 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f904e7fc640 10 monclient: dump: 2026-02-21T22:19:50.801 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 10 monclient: shutdown 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- 10.20.193.102:0/154865571 >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 legacy=0x7f90501a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- 10.20.193.102:0/154865571 shutdown_connections 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- 10.20.193.102:0/154865571 >> 10.20.193.102:0/154865571 conn(0x7f9050054e40 msgr2=0x7f9050092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- 10.20.193.102:0/154865571 shutdown_connections 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- 10.20.193.102:0/154865571 wait complete. 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 10 monclient: monmap: 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 Processor -- start 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 1 -- start start 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.792+0000 7f9055b46640 10 monclient: init 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-b con 0x7f9050056270 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-c con 0x7f9050056d50 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient(hunting): picked mon.noname-a con 0x7f905008d7a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9050140b10 con 0x7f905008d7a0 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f905005ec60 con 0x7f9050056270 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90500594f0 con 0x7f9050056d50 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f9055b46640 10 monclient(hunting): authenticate will time out at 1400.032383s 2026-02-21T22:19:50.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f904effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 0x7f90501946c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49808/0 (socket says 10.20.193.102:49808) 2026-02-21T22:19:50.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f904effd640 1 -- 10.20.193.102:0/3796948585 learned_addr learned my addr 10.20.193.102:0/3796948585 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f904effd640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 legacy=0x7f90501946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f904ffff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6789/0 conn(0x7f905008d7a0 legacy=0x7f9050195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151283048 0 0) 0x7f90500594f0 con 0x7f9050056d50 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): my global_id is 4595 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9020003620 con 0x7f9050056d50 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489048353 0 0) 0x7f9050140b10 con 0x7f905008d7a0 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 10 monclient(hunting): my global_id is 34239 2026-02-21T22:19:50.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90500594f0 con 0x7f905008d7a0 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.796+0000 7f904f7fe640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.178:6789/0 conn(0x7f9050056270 legacy=0x7f9050150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 75298189 0 0) 0x7f9020003620 con 0x7f9050056d50 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9050140b10 con 0x7f9050056d50 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1710853587 0 0) 0x7f90500594f0 con 0x7f905008d7a0 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9020003620 con 0x7f905008d7a0 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 406359653 0 0) 0x7f905005ec60 con 0x7f9050056270 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): my global_id is 44125 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90500594f0 con 0x7f9050056270 2026-02-21T22:19:50.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9040003420 con 0x7f9050056d50 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): dump: 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9044004080 con 0x7f905008d7a0 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient(hunting): dump: 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.812 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3615437277 0 0) 0x7f9050140b10 con 0x7f9050056d50 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.178:6789/0 conn(0x7f9050056270 legacy=0x7f9050150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6789/0 conn(0x7f905008d7a0 legacy=0x7f9050195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 monclient: found mon.c 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9050140830 con 0x7f9050056d50 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 5 monclient: authenticate success, global_id 4595 2026-02-21T22:19:50.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 10 monclient: _renew_subs 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f905013f310 con 0x7f9050056d50 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 10 monclient: _renew_subs 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9050141460 con 0x7f9050056d50 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f90400039c0 con 0x7f9050056d50 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9040003d50 con 0x7f9050056d50 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f904dffb640 4 set_mon_vals no callback set 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 10 monclient: dump: 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.814 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f90500008e0 con 0x7f9050150a60 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9040003fe0 con 0x7f9050056d50 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7f904004c720 con 0x7f9050056d50 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f904ffff640 10 monclient: get_auth_request con 0x7f902003d8f0 auth_method 0 2026-02-21T22:19:50.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.800+0000 7f904f7fe640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f9050150a60 legacy=0x7f905005c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086267958 0 0) 0x7f90500008e0 con 0x7f9050150a60 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient(hunting): my global_id is 4598 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9050140b10 con 0x7f9050150a60 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 886449179 0 0) 0x7f9050140b10 con 0x7f9050150a60 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f90500008e0 con 0x7f9050150a60 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f903c003300 con 0x7f9050150a60 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient: dump: 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.816 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3261598689 0 0) 0x7f90500008e0 con 0x7f9050150a60 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9050091810 con 0x7f9050150a60 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f903c0048e0 con 0x7f9050150a60 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f9050150a60 legacy=0x7f905005c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.804+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f9050150a60 legacy=0x7f905005c0c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:50.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f90500919f0 con 0x7f905005c7d0 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f904f7fe640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f905005c7d0 legacy=0x7f9050198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215647393 0 0) 0x7f90500594f0 con 0x7f905005c7d0 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient(hunting): my global_id is 4601 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90500008e0 con 0x7f905005c7d0 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 569114669 0 0) 0x7f90500008e0 con 0x7f905005c7d0 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f90500594f0 con 0x7f905005c7d0 2026-02-21T22:19:50.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f903c004eb0 con 0x7f905005c7d0 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient: dump: 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3887324432 0 0) 0x7f90500594f0 con 0x7f905005c7d0 2026-02-21T22:19:50.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9050089a30 con 0x7f905005c7d0 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3552506779) 0x7f903c006550 con 0x7f905005c7d0 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f905005c7d0 legacy=0x7f9050198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f902bfff640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f905005c7d0 legacy=0x7f9050198510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:50.821 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 273437, 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.822 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.823 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.824 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.825 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:50.826 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.827 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:50.828 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.829 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:19:50.830 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.831 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.832 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.832 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:50.832 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.178:6800/2549949439 conn(0x7f902003d8f0 legacy=0x7f902005dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 10 monclient: shutdown 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 >> v1:10.20.193.102:6790/0 conn(0x7f9050056d50 legacy=0x7f90501946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f904ffff640 1 -- 10.20.193.102:0/3796948585 reap_dead start 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f904ffff640 1 -- 10.20.193.102:0/3796948585 reap_dead start 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 shutdown_connections 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 >> 10.20.193.102:0/3796948585 conn(0x7f9050054e40 msgr2=0x7f905008f330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 shutdown_connections 2026-02-21T22:19:50.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.808+0000 7f9055b46640 1 -- 10.20.193.102:0/3796948585 wait complete. 2026-02-21T22:19:50.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:50.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:19:50.834 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.900+0000 7f243f4ce640 10 monclient: get_monmap_and_config 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.900+0000 7f243f4ce640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.900+0000 7f243f4ce640 10 monclient: monmap: 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:50.904600+0000 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:50.904600+0000 2026-02-21T22:19:50.903 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 Processor -- start 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 -- start start 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient: init 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-a con 0x7f2438091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-b con 0x7f24380909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-c con 0x7f24381854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2438057a20 con 0x7f2438091240 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2438091d20 con 0x7f24381854b0 2026-02-21T22:19:50.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2438057740 con 0x7f24380909a0 2026-02-21T22:19:50.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient(hunting): authenticate will time out at 1400.140862s 2026-02-21T22:19:50.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243d243640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2438091240 0x7f2438091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58706/0 (socket says 10.20.193.102:58706) 2026-02-21T22:19:50.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243d243640 1 -- 10.20.193.102:0/1776965052 learned_addr learned my addr 10.20.193.102:0/1776965052 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243ca42640 1 --1- 10.20.193.102:0/1776965052 >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 0x7f2438183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49836/0 (socket says 10.20.193.102:49836) 2026-02-21T22:19:50.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243d243640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.102:6789/0 conn(0x7f2438091240 legacy=0x7f2438091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243ca42640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 legacy=0x7f2438183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243da44640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.178:6789/0 conn(0x7f24381854b0 legacy=0x7f24381a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563715906 0 0) 0x7f2438057740 con 0x7f24380909a0 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): my global_id is 4604 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2420003620 con 0x7f24380909a0 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516536580 0 0) 0x7f2438057a20 con 0x7f2438091240 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): my global_id is 34242 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2438057740 con 0x7f2438091240 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642840635 0 0) 0x7f2438091d20 con 0x7f24381854b0 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient(hunting): my global_id is 44128 2026-02-21T22:19:50.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2438057a20 con 0x7f24381854b0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3449778152 0 0) 0x7f2420003620 con 0x7f24380909a0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.178:6789/0 conn(0x7f24381854b0 legacy=0x7f24381a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.102:6789/0 conn(0x7f2438091240 legacy=0x7f2438091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 monclient: found mon.noname-b 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2438090dc0 con 0x7f24380909a0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f242c003110 con 0x7f24380909a0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 5 monclient: authenticate success, global_id 4604 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: dump: 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f242c0033e0 con 0x7f24380909a0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 1 -- 10.20.193.102:0/1776965052 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f242c004a90 con 0x7f24380909a0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f2427fff640 10 monclient: dump: 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.908 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 4 set_mon_vals no callback set 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 10 monclient: shutdown 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.904+0000 7f243f4ce640 1 -- 10.20.193.102:0/1776965052 >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 legacy=0x7f2438183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/1776965052 shutdown_connections 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/1776965052 >> 10.20.193.102:0/1776965052 conn(0x7f243805e660 msgr2=0x7f2438095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/1776965052 shutdown_connections 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/1776965052 wait complete. 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: build_initial_monmap 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: monmap: 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 Processor -- start 2026-02-21T22:19:50.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- start start 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: init 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-c con 0x7f24380909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-a con 0x7f2438091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient(hunting): picked mon.noname-b con 0x7f24381854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243812bea0 con 0x7f2438091240 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24381130f0 con 0x7f24381854b0 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2438112280 con 0x7f24380909a0 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient(hunting): authenticate will time out at 1400.144999s 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243d243640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 0x7f243812b790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49842/0 (socket says 10.20.193.102:49842) 2026-02-21T22:19:50.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243d243640 1 -- 10.20.193.102:0/2436456209 learned_addr learned my addr 10.20.193.102:0/2436456209 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243d243640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 legacy=0x7f243812b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243ca42640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.102:6789/0 conn(0x7f2438091240 legacy=0x7f2438172b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387963132 0 0) 0x7f2438112280 con 0x7f24380909a0 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): my global_id is 4607 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2408003620 con 0x7f24380909a0 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550819834 0 0) 0x7f243812bea0 con 0x7f2438091240 2026-02-21T22:19:50.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): my global_id is 34245 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2438112280 con 0x7f2438091240 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243da44640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f24381854b0 legacy=0x7f2438174270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3096216976 0 0) 0x7f2438112280 con 0x7f2438091240 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f243812bea0 con 0x7f2438091240 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f242c0048e0 con 0x7f2438091240 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): dump: 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.915 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1534368470 0 0) 0x7f24381130f0 con 0x7f24381854b0 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): my global_id is 44131 2026-02-21T22:19:50.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2438112280 con 0x7f24381854b0 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2131882721 0 0) 0x7f2408003620 con 0x7f24380909a0 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24381130f0 con 0x7f24380909a0 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2428003400 con 0x7f24380909a0 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient(hunting): dump: 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.917 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1314175831 0 0) 0x7f243812bea0 con 0x7f2438091240 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.102:6790/0 conn(0x7f24380909a0 legacy=0x7f243812b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f24381854b0 legacy=0x7f2438174270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 monclient: found mon.a 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2438110d90 con 0x7f2438091240 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient: _finish_auth 0 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:19:50.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 5 monclient: authenticate success, global_id 34245 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _renew_subs 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f24381128b0 con 0x7f2438091240 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _renew_subs 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2438111ac0 con 0x7f2438091240 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f242c0033c0 con 0x7f2438091240 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.908+0000 7f2425ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f242c005220 con 0x7f2438091240 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f24277fe640 4 set_mon_vals no callback set 2026-02-21T22:19:50.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: dump: 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f243f4ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f243812eb50 con 0x7f243812e780 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f242c0118b0 con 0x7f2438091240 2026-02-21T22:19:50.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7f242c04c650 con 0x7f2438091240 2026-02-21T22:19:50.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f243da44640 10 monclient: get_auth_request con 0x7f240803d830 auth_method 0 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f243d243640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243812e780 legacy=0x7f243808cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438553903 0 0) 0x7f243812eb50 con 0x7f243812e780 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient(hunting): my global_id is 44134 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243812bea0 con 0x7f243812e780 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1895006091 0 0) 0x7f243812bea0 con 0x7f243812e780 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f243812eb50 con 0x7f243812e780 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2428004c80 con 0x7f243812e780 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: dump: 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.923 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3608421214 0 0) 0x7f243812eb50 con 0x7f243812e780 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2438110710 con 0x7f243812e780 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2428006290 con 0x7f243812e780 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.912+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243812e780 legacy=0x7f243808cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.916+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243812e780 legacy=0x7f243808cc00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f243f4ce640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:19:50.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2438112ca0 con 0x7f243808d310 2026-02-21T22:19:50.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f243d243640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243808d310 legacy=0x7f24381769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695478129 0 0) 0x7f24381130f0 con 0x7f243808d310 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 10 monclient(hunting): my global_id is 44137 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.920+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f243812eb50 con 0x7f243808d310 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1412652852 0 0) 0x7f243812eb50 con 0x7f243808d310 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24381130f0 con 0x7f243808d310 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2428006820 con 0x7f243808d310 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 10 monclient: dump: 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:50.926 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3383092545 0 0) 0x7f24381130f0 con 0x7f243808d310 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f243805eb70 con 0x7f243808d310 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 4180907107) 0x7f2428005d50 con 0x7f243808d310 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243808d310 legacy=0x7f24381769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f2425ffb640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6789/0 conn(0x7f243808d310 legacy=0x7f24381769a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 20, 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 2779, 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.928 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.929 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.930 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:19:50.931 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:19:50.932 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:19:50.933 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:19:50.934 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.935 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:19:50.936 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:19:50.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.178:6800/2549949439 conn(0x7f240803d830 legacy=0x7f240805dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 10 monclient: shutdown 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 >> v1:10.20.193.102:6789/0 conn(0x7f2438091240 legacy=0x7f2438172b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243da44640 1 -- 10.20.193.102:0/2436456209 reap_dead start 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 shutdown_connections 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 >> 10.20.193.102:0/2436456209 conn(0x7f243805e660 msgr2=0x7f243808bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 shutdown_connections 2026-02-21T22:19:50.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:50.924+0000 7f243f4ce640 1 -- 10.20.193.102:0/2436456209 wait complete. 2026-02-21T22:19:50.938 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:19:50.938 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-21T22:19:50.938 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:19:50.938 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-21T22:19:50.938 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-21T22:19:50.939 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:19:50.939 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:19:50.949 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:19:50.949 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:19:50.949 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: get_monmap_and_config 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: build_initial_monmap 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: monmap: 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:51.010004+0000 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:51.010004+0000 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:51.009 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 1 Processor -- start 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 1 -- start start 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: init 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient(hunting): picked mon.noname-a con 0x7f7818085970 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient(hunting): picked mon.noname-b con 0x7f7818085d40 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient(hunting): picked mon.noname-c con 0x7f78181854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781808df30 con 0x7f7818085970 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781808d900 con 0x7f78181854c0 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7818056dd0 con 0x7f7818085d40 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f781d197640 10 monclient(hunting): authenticate will time out at 1400.246227s 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f7816575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7818085d40 0x7f7818183da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49848/0 (socket says 10.20.193.102:49848) 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f7817577640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f78181854c0 legacy=0x7f78181a58a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f7816575640 1 -- 10.20.193.102:0/2840713730 learned_addr learned my addr 10.20.193.102:0/2840713730 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:51.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.008+0000 7f7816d76640 1 --1- 10.20.193.102:0/2840713730 >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 0x7f781808e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58730/0 (socket says 10.20.193.102:58730) 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7816d76640 1 -- 10.20.193.102:0/2840713730 >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 legacy=0x7f781808e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7816575640 1 -- 10.20.193.102:0/2840713730 >> v1:10.20.193.102:6790/0 conn(0x7f7818085d40 legacy=0x7f7818183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788696491 0 0) 0x7f781808df30 con 0x7f7818085970 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _negotiate 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:51.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): my global_id is 34248 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7804003620 con 0x7f7818085970 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434596680 0 0) 0x7f7818056dd0 con 0x7f7818085d40 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _negotiate 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient(hunting): my global_id is 4610 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f781808df30 con 0x7f7818085d40 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320324209 0 0) 0x7f7804003620 con 0x7f7818085970 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 >> v1:10.20.193.102:6790/0 conn(0x7f7818085d40 legacy=0x7f7818183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 >> v1:10.20.193.178:6789/0 conn(0x7f78181854c0 legacy=0x7f78181a58a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: _finish_hunting 0 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 monclient: found mon.noname-a 2026-02-21T22:19:51.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781808d620 con 0x7f7818085970 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: _finish_auth 0 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: _check_auth_tickets 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f780c003190 con 0x7f7818085970 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 5 monclient: authenticate success, global_id 34248 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: dump: 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.016 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f780c003420 con 0x7f7818085970 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 1 -- 10.20.193.102:0/2840713730 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f780c004a90 con 0x7f7818085970 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 4 set_mon_vals no callback set 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7815d74640 10 monclient: dump: 2026-02-21T22:19:51.017 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: shutdown 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- 10.20.193.102:0/2840713730 >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 legacy=0x7f781808e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- 10.20.193.102:0/2840713730 shutdown_connections 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- 10.20.193.102:0/2840713730 >> 10.20.193.102:0/2840713730 conn(0x7f7818054e40 msgr2=0x7f7818092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:51.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- 10.20.193.102:0/2840713730 shutdown_connections 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- 10.20.193.102:0/2840713730 wait complete. 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: build_initial_monmap 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: monmap: 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 Processor -- start 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- start start 2026-02-21T22:19:51.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: init 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient(hunting): picked mon.noname-a con 0x7f7818085970 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient(hunting): picked mon.noname-b con 0x7f7818085d40 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient(hunting): picked mon.noname-c con 0x7f78181854c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7818084250 con 0x7f7818085970 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7818089a80 con 0x7f7818085d40 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78181852b0 con 0x7f78181854c0 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f781d197640 10 monclient(hunting): authenticate will time out at 1400.250697s 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7816575640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f7818085d40 legacy=0x7f7818183b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7816d76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 0x7f7818131e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:58746/0 (socket says 10.20.193.102:58746) 2026-02-21T22:19:51.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.012+0000 7f7816d76640 1 -- 10.20.193.102:0/4184975076 learned_addr learned my addr 10.20.193.102:0/4184975076 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f7816d76640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 legacy=0x7f7818131e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f7817577640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.102:6790/0 conn(0x7f78181854c0 legacy=0x7f7818132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181373511 0 0) 0x7f7818084250 con 0x7f7818085970 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): my global_id is 34251 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77e4003620 con 0x7f7818085970 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2670765637 0 0) 0x7f78181852b0 con 0x7f78181854c0 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): my global_id is 4613 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7818084250 con 0x7f78181854c0 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 269576377 0 0) 0x7f77e4003620 con 0x7f7818085970 2026-02-21T22:19:51.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78181852b0 con 0x7f7818085970 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f780c003190 con 0x7f7818085970 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): dump: 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1494819077 0 0) 0x7f78181852b0 con 0x7f7818085970 2026-02-21T22:19:51.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.102:6790/0 conn(0x7f78181854c0 legacy=0x7f7818132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.178:6789/0 conn(0x7f7818085d40 legacy=0x7f7818183b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 monclient: found mon.a 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781807d9e0 con 0x7f7818085970 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: _finish_auth 0 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 5 monclient: authenticate success, global_id 34251 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _renew_subs 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f781807f4a0 con 0x7f7818085970 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _renew_subs 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7818080740 con 0x7f7818085970 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f780c0033b0 con 0x7f7818085970 2026-02-21T22:19:51.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f780c0051b0 con 0x7f7818085970 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f7815573640 4 set_mon_vals no callback set 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 10 monclient: dump: 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:51.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781808f100 con 0x7f7818085970 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f780c005450 con 0x7f7818085970 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7f780c04c800 con 0x7f7818085970 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.016+0000 7f7816575640 10 monclient: get_auth_request con 0x7f77e403d420 auth_method 0 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.020+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f780c017780 con 0x7f7818085970 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.020+0000 7f77f77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:51.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.020+0000 7f77f77fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7818062ce0 con 0x7f7818085970 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f77f77fe640 1 -- 10.20.193.102:0/4184975076 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2571281086) 0x7f780c01ca40 con 0x7f7818085970 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f77f77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f77f77fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:51.138 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.178:6800/2549949439 conn(0x7f77e403d420 legacy=0x7f77e405d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 10 monclient: shutdown 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 >> v1:10.20.193.102:6789/0 conn(0x7f7818085970 legacy=0x7f7818131e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 shutdown_connections 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 >> 10.20.193.102:0/4184975076 conn(0x7f7818054e40 msgr2=0x7f7818087d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 shutdown_connections 2026-02-21T22:19:51.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:51.136+0000 7f781d197640 1 -- 10.20.193.102:0/4184975076 wait complete. 2026-02-21T22:19:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:19:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:19:53.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:54.150 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:19:54.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: get_monmap_and_config 2026-02-21T22:19:54.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: build_initial_monmap 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: monmap: 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:54.211540+0000 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:54.211540+0000 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 1 Processor -- start 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 1 -- start start 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: init 2026-02-21T22:19:54.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-b con 0x7f32d008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-a con 0x7f32d008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-c con 0x7f32d008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d0086a10 con 0x7f32d008b7f0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d008b130 con 0x7f32d008d8e0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d008bd60 con 0x7f32d008ce00 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32d5b07640 10 monclient(hunting): authenticate will time out at 1403.447412s 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32ceffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f32d008b7f0 0x7f32d0181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38870/0 (socket says 10.20.193.102:38870) 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32cffff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f32d008d8e0 legacy=0x7f32d01a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:54.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.207+0000 7f32ceffd640 1 -- 10.20.193.102:0/48608916 learned_addr learned my addr 10.20.193.102:0/48608916 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ceffd640 1 -- 10.20.193.102:0/48608916 >> v1:10.20.193.102:6789/0 conn(0x7f32d008b7f0 legacy=0x7f32d0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32cf7fe640 1 -- 10.20.193.102:0/48608916 >> v1:10.20.193.102:6790/0 conn(0x7f32d008ce00 legacy=0x7f32d008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574047340 0 0) 0x7f32d008bd60 con 0x7f32d008ce00 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:54.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient(hunting): my global_id is 4616 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32b4003620 con 0x7f32d008ce00 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1045002359 0 0) 0x7f32b4003620 con 0x7f32d008ce00 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 >> v1:10.20.193.178:6789/0 conn(0x7f32d008d8e0 legacy=0x7f32d01a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 >> v1:10.20.193.102:6789/0 conn(0x7f32d008b7f0 legacy=0x7f32d0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 monclient: found mon.noname-b 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d008b370 con 0x7f32d008ce00 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: _finish_auth 0 2026-02-21T22:19:54.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 5 monclient: authenticate success, global_id 4616 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f32bc0031e0 con 0x7f32d008ce00 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: dump: 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f32bc003470 con 0x7f32d008ce00 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 1 -- 10.20.193.102:0/48608916 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f32bc0038c0 con 0x7f32d008ce00 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32ce7fc640 10 monclient: dump: 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:54.215 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: get_monmap_and_config success 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 4 set_mon_vals no callback set 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: shutdown 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- 10.20.193.102:0/48608916 >> v1:10.20.193.102:6790/0 conn(0x7f32d008ce00 legacy=0x7f32d008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- 10.20.193.102:0/48608916 shutdown_connections 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- 10.20.193.102:0/48608916 >> 10.20.193.102:0/48608916 conn(0x7f32d0091a50 msgr2=0x7f32d0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- 10.20.193.102:0/48608916 shutdown_connections 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- 10.20.193.102:0/48608916 wait complete. 2026-02-21T22:19:54.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: build_initial_monmap 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: monmap: 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 Processor -- start 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- start start 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: init 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-c con 0x7f32d008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-a con 0x7f32d008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 10 monclient(hunting): picked mon.noname-b con 0x7f32d008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d0082c90 con 0x7f32d008ce00 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.211+0000 7f32d5b07640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d0082f70 con 0x7f32d008d8e0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32d5b07640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32d00817a0 con 0x7f32d008b7f0 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32d5b07640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32d5b07640 10 monclient(hunting): authenticate will time out at 1403.452040s 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32ceffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f32d008ce00 0x7f32d0084680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38882/0 (socket says 10.20.193.102:38882) 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32cf7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f32d008b7f0 0x7f32d0153010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:52900/0 (socket says 10.20.193.102:52900) 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32ceffd640 1 -- 10.20.193.102:0/2366740395 learned_addr learned my addr 10.20.193.102:0/2366740395 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:54.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32cffff640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.178:6789/0 conn(0x7f32d008d8e0 legacy=0x7f32d0195380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:54.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32ceffd640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.102:6789/0 conn(0x7f32d008ce00 legacy=0x7f32d0084680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:54.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32cf7fe640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.102:6790/0 conn(0x7f32d008b7f0 legacy=0x7f32d0153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:54.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.215+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446383386 0 0) 0x7f32d00817a0 con 0x7f32d008b7f0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): my global_id is 4619 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32a0003620 con 0x7f32d008b7f0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4219528526 0 0) 0x7f32a0003620 con 0x7f32d008b7f0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32d00817a0 con 0x7f32d008b7f0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f32bc0038c0 con 0x7f32d008b7f0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:19:54.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): dump: 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3006681010 0 0) 0x7f32d00817a0 con 0x7f32d008b7f0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.178:6789/0 conn(0x7f32d008d8e0 legacy=0x7f32d0195380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.102:6789/0 conn(0x7f32d008ce00 legacy=0x7f32d0084680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: _finish_hunting 0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 monclient: found mon.c 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d0084d90 con 0x7f32d008b7f0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: _finish_auth 0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: _check_auth_tickets 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 5 monclient: authenticate success, global_id 4619 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _renew_subs 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f32d0085020 con 0x7f32d008b7f0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _renew_subs 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f32d0196c70 con 0x7f32d008b7f0 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f32bc002e00 con 0x7f32d008b7f0 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f32bc003ee0 con 0x7f32d008b7f0 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 10 monclient: dump: 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32cdffb640 4 set_mon_vals no callback set 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:54.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d00008e0 con 0x7f32d008b7f0 2026-02-21T22:19:54.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f32bc0041b0 con 0x7f32d008b7f0 2026-02-21T22:19:54.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7f32bc04c780 con 0x7f32d008b7f0 2026-02-21T22:19:54.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.219+0000 7f32ceffd640 10 monclient: get_auth_request con 0x7f32a003d3c0 auth_method 0 2026-02-21T22:19:54.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.223+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f32bc0177c0 con 0x7f32d008b7f0 2026-02-21T22:19:54.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.223+0000 7f32abfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:19:54.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.223+0000 7f32abfff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f32d00831b0 con 0x7f32d008b7f0 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32abfff640 1 -- 10.20.193.102:0/2366740395 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3351987006) 0x7f32bc0170a0 con 0x7f32d008b7f0 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32abfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32abfff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:19:54.342 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.178:6800/2549949439 conn(0x7f32a003d3c0 legacy=0x7f32a005d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 10 monclient: shutdown 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 >> v1:10.20.193.102:6790/0 conn(0x7f32d008b7f0 legacy=0x7f32d0153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 shutdown_connections 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 >> 10.20.193.102:0/2366740395 conn(0x7f32d0091a50 msgr2=0x7f32d005a140 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 shutdown_connections 2026-02-21T22:19:54.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:54.339+0000 7f32d5b07640 1 -- 10.20.193.102:0/2366740395 wait complete. 2026-02-21T22:19:57.353 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:19:57.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 10 monclient: get_monmap_and_config 2026-02-21T22:19:57.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 10 monclient: build_initial_monmap 2026-02-21T22:19:57.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 10 monclient: monmap: 2026-02-21T22:19:57.415 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:19:57.415 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:19:57.415989+0000 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:19:57.415989+0000 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 1 Processor -- start 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 1 -- start start 2026-02-21T22:19:57.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.411+0000 7f6771d55640 10 monclient: init 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient: _reopen_session rank -1 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-c con 0x7f676c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-a con 0x7f676c086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-b con 0x7f676c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c086a70 con 0x7f676c086e90 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c08b130 con 0x7f676c084860 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c085340 con 0x7f676c183350 2026-02-21T22:19:57.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient(hunting): _renew_subs 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f6771d55640 10 monclient(hunting): authenticate will time out at 1406.652127s 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f676b7fe640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c084c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f676affd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f676c086e90 0x7f676c181c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38890/0 (socket says 10.20.193.102:38890) 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 learned_addr learned my addr 10.20.193.102:0/1376377193 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f676c086e90 legacy=0x7f676c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:57.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.415+0000 7f676bfff640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f676c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:19:57.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:57.619+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c084c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:58.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:58.023+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c084c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:19:58.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:19:58.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:19:58.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:19:58.827+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c084c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:00.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): tick 2026-02-21T22:20:00.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 monclient(hunting): continuing hunt 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f676c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c084c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f676c086e90 legacy=0x7f676c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-a con 0x7f67480038c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-c con 0x7f676c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-b con 0x7f676c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c086790 con 0x7f67480038c0 2026-02-21T22:20:00.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c08b130 con 0x7f676c084860 2026-02-21T22:20:00.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674806a5e0 con 0x7f676c183350 2026-02-21T22:20:00.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f67697fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:00.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:00.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f676bfff640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f6748066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:00.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.415+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f67480038c0 legacy=0x7f6748023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:00.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:00.619+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:01.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:01.023+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:01.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:01.827+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:03.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): tick 2026-02-21T22:20:03.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:20:03.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 monclient(hunting): continuing hunt 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f6748066d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f67480038c0 legacy=0x7f6748023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-c con 0x7f676c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-b con 0x7f676c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): picked mon.noname-a con 0x7f67480038c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674806b940 con 0x7f67480038c0 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c08b130 con 0x7f676c084860 2026-02-21T22:20:03.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67480697c0 con 0x7f676c183350 2026-02-21T22:20:03.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f67697fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:03.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:03.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f676affd640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f6748066d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:03.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.415+0000 7f676bfff640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f67480038c0 legacy=0x7f6748023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:03.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:03.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:03.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:03.619+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:04.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:04.023+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:04.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:04.827+0000 7f676b7fe640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046585435 0 0) 0x7f674806b940 con 0x7f67480038c0 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient(hunting): my global_id is 34254 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6750003620 con 0x7f67480038c0 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2727501037 0 0) 0x7f6750003620 con 0x7f67480038c0 2026-02-21T22:20:05.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6790/0 conn(0x7f676c183350 legacy=0x7f6748066d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f6748045590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 monclient: found mon.noname-a 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6748069a50 con 0x7f67480038c0 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67600045d0 con 0x7f67480038c0 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 5 monclient: authenticate success, global_id 34254 2026-02-21T22:20:05.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: dump: 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:05.960 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67600047b0 con 0x7f67480038c0 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 1 -- 10.20.193.102:0/1376377193 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6760004c20 con 0x7f67480038c0 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f676a7fc640 10 monclient: dump: 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.961 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 10 monclient: get_monmap_and_config success 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 4 set_mon_vals no callback set 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 10 monclient: shutdown 2026-02-21T22:20:05.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 1 -- 10.20.193.102:0/1376377193 >> v1:10.20.193.102:6789/0 conn(0x7f67480038c0 legacy=0x7f6748023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 1 -- 10.20.193.102:0/1376377193 shutdown_connections 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 1 -- 10.20.193.102:0/1376377193 >> 10.20.193.102:0/1376377193 conn(0x7f676c091a50 msgr2=0x7f676c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 1 -- 10.20.193.102:0/1376377193 shutdown_connections 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.951+0000 7f6771d55640 1 -- 10.20.193.102:0/1376377193 wait complete. 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: build_initial_monmap 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: monmap: 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:20:05.963 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 Processor -- start 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 -- start start 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: init 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: _reopen_session rank -1 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-c con 0x7f67480038c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:05.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-b con 0x7f676c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient(hunting): picked mon.noname-a con 0x7f676c183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c082480 con 0x7f676c183350 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c082760 con 0x7f676c084860 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676c05ea90 con 0x7f67480038c0 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient(hunting): authenticate will time out at 1415.193067s 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f676affd640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c07f0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f676bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f676c183350 0x7f676c175b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40172/0 (socket says 10.20.193.102:40172) 2026-02-21T22:20:05.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f676bfff640 1 -- 10.20.193.102:0/1718386181 learned_addr learned my addr 10.20.193.102:0/1718386181 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:20:05.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f676bfff640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.102:6789/0 conn(0x7f676c183350 legacy=0x7f676c175b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f676b7fe640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.102:6790/0 conn(0x7f67480038c0 legacy=0x7f676c07e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510599443 0 0) 0x7f676c082480 con 0x7f676c183350 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): _negotiate 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): my global_id is 34257 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f673c003620 con 0x7f676c183350 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1698766973 0 0) 0x7f673c003620 con 0x7f676c183350 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f676c082480 con 0x7f676c183350 2026-02-21T22:20:05.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67600051a0 con 0x7f676c183350 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): dump: 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:05.970 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 636794056 0 0) 0x7f676c082480 con 0x7f676c183350 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.102:6790/0 conn(0x7f67480038c0 legacy=0x7f676c07e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.178:6789/0 conn(0x7f676c084860 legacy=0x7f676c07f0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient: _finish_hunting 0 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 monclient: found mon.a 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c081940 con 0x7f676c183350 2026-02-21T22:20:05.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient: _finish_auth 0 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 10 monclient: _check_auth_tickets 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6747fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 5 monclient: authenticate success, global_id 34257 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: _renew_subs 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.955+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f676c081ec0 con 0x7f676c183350 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 10 monclient: _renew_subs 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f676c081190 con 0x7f676c183350 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6760004a40 con 0x7f676c183350 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:05.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67600047b0 con 0x7f676c183350 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 10 monclient: dump: 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:05.973 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6769ffb640 4 set_mon_vals no callback set 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676c05e420 con 0x7f676c183350 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f67600053c0 con 0x7f676c183350 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7f676004d9e0 con 0x7f676c183350 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.959+0000 7f676b7fe640 10 monclient: get_auth_request con 0x7f673c03d390 auth_method 0 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.963+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f6760018a20 con 0x7f676c183350 2026-02-21T22:20:05.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.963+0000 7f6747fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:05.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:05.963+0000 7f6747fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:20:06.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.075+0000 7f6771d55640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:20:06.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.075+0000 7f6771d55640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:20:06.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.075+0000 7f6771d55640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:06.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.075+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f676c05d290 con 0x7f676c183350 2026-02-21T22:20:06.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6747fff640 1 -- 10.20.193.102:0/1718386181 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 2891288349) 0x7f6760018300 con 0x7f676c183350 2026-02-21T22:20:06.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6747fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:20:06.081 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6747fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:20:06.081 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":24,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.178:6800/2549949439 conn(0x7f673c03d390 legacy=0x7f673c05d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 10 monclient: shutdown 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 >> v1:10.20.193.102:6789/0 conn(0x7f676c183350 legacy=0x7f676c175b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 shutdown_connections 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 >> 10.20.193.102:0/1718386181 conn(0x7f676c091a50 msgr2=0x7f676c087410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 shutdown_connections 2026-02-21T22:20:06.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.079+0000 7f6771d55640 1 -- 10.20.193.102:0/1718386181 wait complete. 2026-02-21T22:20:06.091 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:20:06.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:20:06.091 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:20:06.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: get_monmap_and_config 2026-02-21T22:20:06.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: build_initial_monmap 2026-02-21T22:20:06.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: monmap: 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:20:06.154689+0000 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:20:06.154689+0000 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 1 Processor -- start 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 1 -- start start 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: init 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: _reopen_session rank -1 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-b con 0x7ff0b4084860 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-a con 0x7ff0b4086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-c con 0x7ff0b4183350 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b4086a70 con 0x7ff0b4086e90 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b408b130 con 0x7ff0b4183350 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b4085340 con 0x7ff0b4084860 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:06.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bc739640 10 monclient(hunting): authenticate will time out at 1415.390835s 2026-02-21T22:20:06.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0bacaf640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7ff0b4183350 legacy=0x7ff0b41a3730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:06.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0ba4ae640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff0b4084860 0x7ff0b4084c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37310/0 (socket says 10.20.193.102:37310) 2026-02-21T22:20:06.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.151+0000 7ff0ba4ae640 1 -- 10.20.193.102:0/812645815 learned_addr learned my addr 10.20.193.102:0/812645815 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0ba4ae640 1 -- 10.20.193.102:0/812645815 >> v1:10.20.193.102:6790/0 conn(0x7ff0b4084860 legacy=0x7ff0b4084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b9cad640 1 -- 10.20.193.102:0/812645815 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4086e90 legacy=0x7ff0b4181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451704880 0 0) 0x7ff0b4085340 con 0x7ff0b4084860 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): my global_id is 4622 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff09c003620 con 0x7ff0b4084860 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835853647 0 0) 0x7ff0b4086a70 con 0x7ff0b4086e90 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient(hunting): my global_id is 34260 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b4085340 con 0x7ff0b4086e90 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 715154905 0 0) 0x7ff09c003620 con 0x7ff0b4084860 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 >> v1:10.20.193.178:6789/0 conn(0x7ff0b4183350 legacy=0x7ff0b41a3730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4086e90 legacy=0x7ff0b4181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: _finish_hunting 0 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 monclient: found mon.noname-b 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4086790 con 0x7ff0b4084860 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: _finish_auth 0 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: _check_auth_tickets 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a4003100 con 0x7ff0b4084860 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: dump: 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.166 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 5 monclient: authenticate success, global_id 4622 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff0a40033d0 con 0x7ff0b4084860 2026-02-21T22:20:06.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 1 -- 10.20.193.102:0/812645815 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a4004b50 con 0x7ff0b4084860 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0b94ac640 10 monclient: dump: 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.168 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: get_monmap_and_config success 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 4 set_mon_vals no callback set 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: shutdown 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- 10.20.193.102:0/812645815 >> v1:10.20.193.102:6790/0 conn(0x7ff0b4084860 legacy=0x7ff0b4084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- 10.20.193.102:0/812645815 shutdown_connections 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- 10.20.193.102:0/812645815 >> 10.20.193.102:0/812645815 conn(0x7ff0b4091a50 msgr2=0x7ff0b4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- 10.20.193.102:0/812645815 shutdown_connections 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- 10.20.193.102:0/812645815 wait complete. 2026-02-21T22:20:06.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: build_initial_monmap 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: monmap: 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 Processor -- start 2026-02-21T22:20:06.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- start start 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: init 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: _reopen_session rank -1 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-b con 0x7ff0b4084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-c con 0x7ff0b4086e90 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient(hunting): picked mon.noname-a con 0x7ff0b4183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b405e220 con 0x7ff0b4183350 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b4080bc0 con 0x7ff0b4084860 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b4087810 con 0x7ff0b4086e90 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bc739640 10 monclient(hunting): authenticate will time out at 1415.395379s 2026-02-21T22:20:06.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0ba4ae640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7ff0b4084860 legacy=0x7ff0b4131d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bacaf640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff0b4183350 0x7ff0b4174680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40190/0 (socket says 10.20.193.102:40190) 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.155+0000 7ff0bacaf640 1 -- 10.20.193.102:0/729904858 learned_addr learned my addr 10.20.193.102:0/729904858 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bacaf640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4183350 legacy=0x7ff0b4174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0b9cad640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6790/0 conn(0x7ff0b4086e90 legacy=0x7ff0b4172f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3594142275 0 0) 0x7ff0b4087810 con 0x7ff0b4086e90 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): my global_id is 4625 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff084003620 con 0x7ff0b4086e90 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507136348 0 0) 0x7ff0b0005340 con 0x7ff0b4183350 2026-02-21T22:20:06.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): my global_id is 34263 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b4087810 con 0x7ff0b4183350 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 94105295 0 0) 0x7ff084003620 con 0x7ff0b4086e90 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0b0005340 con 0x7ff0b4086e90 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a80033f0 con 0x7ff0b4086e90 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): dump: 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.173 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 893423675 0 0) 0x7ff0b4087810 con 0x7ff0b4183350 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff084003620 con 0x7ff0b4183350 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0b0005b40 con 0x7ff0b4183350 2026-02-21T22:20:06.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient(hunting): dump: 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.175 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3137941674 0 0) 0x7ff0b0005340 con 0x7ff0b4086e90 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.178:6789/0 conn(0x7ff0b4084860 legacy=0x7ff0b4131d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4183350 legacy=0x7ff0b4174680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: _finish_hunting 0 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 monclient: found mon.c 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b40808e0 con 0x7ff0b4086e90 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: _finish_auth 0 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: _check_auth_tickets 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 5 monclient: authenticate success, global_id 4625 2026-02-21T22:20:06.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 10 monclient: _renew_subs 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b407f6a0 con 0x7ff0b4086e90 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 10 monclient: _renew_subs 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff0b407e390 con 0x7ff0b4086e90 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff0a80039c0 con 0x7ff0b4086e90 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a8003da0 con 0x7ff0b4086e90 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0b8cab640 4 set_mon_vals no callback set 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 10 monclient: dump: 2026-02-21T22:20:06.177 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff0b407eb00 con 0x7ff0b4132460 2026-02-21T22:20:06.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff0a8003fc0 con 0x7ff0b4086e90 2026-02-21T22:20:06.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7ff0a804c7e0 con 0x7ff0b4086e90 2026-02-21T22:20:06.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.159+0000 7ff0bacaf640 10 monclient: get_auth_request con 0x7ff08403d5c0 auth_method 0 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff0ba4ae640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4132460 legacy=0x7ff0b4089e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050144567 0 0) 0x7ff0b407eb00 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient(hunting): my global_id is 34266 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b0005340 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 363463451 0 0) 0x7ff0b0005340 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0b407eb00 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a4002d60 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient: dump: 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1490761179 0 0) 0x7ff0b407eb00 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0b40008e0 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ff0a4002fb0 con 0x7ff0b4132460 2026-02-21T22:20:06.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4132460 legacy=0x7ff0b4089e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.163+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b4132460 legacy=0x7ff0b4089e00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff0bc739640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff0b4132830 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff0ba4ae640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b404d280 legacy=0x7ff0b4176db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258307631 0 0) 0x7ff0b4132830 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient(hunting): my global_id is 34269 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b407eb00 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1095330119 0 0) 0x7ff0b407eb00 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0b4132830 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff0a4005000 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient: dump: 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4249159611 0 0) 0x7ff0b4132830 con 0x7ff0b404d280 2026-02-21T22:20:06.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff0b408c6b0 con 0x7ff0b404d280 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4503 (unknown 0 0 2514664703) 0x7ff0a4007220 con 0x7ff0b404d280 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b404d280 legacy=0x7ff0b4176db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.167+0000 7ff09affd640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6789/0 conn(0x7ff0b404d280 legacy=0x7ff0b4176db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 24, 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 155036, 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.184 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:20:06.185 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.186 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.187 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.188 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.189 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:20:06.190 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff08403d5c0 legacy=0x7ff08405da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 10 monclient: shutdown 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 >> v1:10.20.193.102:6790/0 conn(0x7ff0b4086e90 legacy=0x7ff0b4172f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bacaf640 1 -- 10.20.193.102:0/729904858 reap_dead start 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bacaf640 1 -- 10.20.193.102:0/729904858 reap_dead start 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 shutdown_connections 2026-02-21T22:20:06.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 >> 10.20.193.102:0/729904858 conn(0x7ff0b4091a50 msgr2=0x7ff0b41a3730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:06.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 shutdown_connections 2026-02-21T22:20:06.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.171+0000 7ff0bc739640 1 -- 10.20.193.102:0/729904858 wait complete. 2026-02-21T22:20:06.192 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:20:06.192 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:20:06.192 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:20:06.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 10 monclient: get_monmap_and_config 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 10 monclient: build_initial_monmap 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 10 monclient: monmap: 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:20:06.264052+0000 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:20:06.264052+0000 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:20:06.265 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 1 Processor -- start 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 1 -- start start 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.259+0000 7ffb35281640 10 monclient: init 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient: _reopen_session rank -1 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:20:06.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-b con 0x7ffb3008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-a con 0x7ffb3008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-c con 0x7ffb3008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb30086a10 con 0x7ffb3008b7f0 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3008b130 con 0x7ffb3008d8e0 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3008bd60 con 0x7ffb3008ce00 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient(hunting): authenticate will time out at 1415.500218s 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2ed76640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 0x7ffb3008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37318/0 (socket says 10.20.193.102:37318) 2026-02-21T22:20:06.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2ed76640 1 -- 10.20.193.102:0/3975814811 learned_addr learned my addr 10.20.193.102:0/3975814811 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:20:06.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2f577640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.178:6789/0 conn(0x7ffb3008d8e0 legacy=0x7ffb301a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:06.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2ed76640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 legacy=0x7ffb3008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2e575640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.102:6789/0 conn(0x7ffb3008b7f0 legacy=0x7ffb30181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804361493 0 0) 0x7ffb3008bd60 con 0x7ffb3008ce00 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): my global_id is 4628 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb1c003620 con 0x7ffb3008ce00 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869242058 0 0) 0x7ffb30086a10 con 0x7ffb3008b7f0 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient(hunting): my global_id is 34272 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb3008bd60 con 0x7ffb3008b7f0 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2387907640 0 0) 0x7ffb1c003620 con 0x7ffb3008ce00 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.178:6789/0 conn(0x7ffb3008d8e0 legacy=0x7ffb301a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.102:6789/0 conn(0x7ffb3008b7f0 legacy=0x7ffb30181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: _finish_hunting 0 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 monclient: found mon.noname-b 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:06.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3008b370 con 0x7ffb3008ce00 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: _finish_auth 0 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: _check_auth_tickets 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb24003110 con 0x7ffb3008ce00 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: dump: 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.281 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 5 monclient: authenticate success, global_id 4628 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb240033e0 con 0x7ffb3008ce00 2026-02-21T22:20:06.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 1 -- 10.20.193.102:0/3975814811 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb24004b50 con 0x7ffb3008ce00 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb2dd74640 10 monclient: dump: 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.283 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient: get_monmap_and_config success 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 4 set_mon_vals no callback set 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient: shutdown 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- 10.20.193.102:0/3975814811 >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 legacy=0x7ffb3008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- 10.20.193.102:0/3975814811 shutdown_connections 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- 10.20.193.102:0/3975814811 >> 10.20.193.102:0/3975814811 conn(0x7ffb30091a50 msgr2=0x7ffb30091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- 10.20.193.102:0/3975814811 shutdown_connections 2026-02-21T22:20:06.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 1 -- 10.20.193.102:0/3975814811 wait complete. 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.263+0000 7ffb35281640 10 monclient: build_initial_monmap 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: monmap: 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 Processor -- start 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- start start 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: init 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _reopen_session rank -1 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-b con 0x7ffb3008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-c con 0x7ffb3008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient(hunting): picked mon.noname-a con 0x7ffb3008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb30080eb0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb30088a70 con 0x7ffb3008b7f0 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb30083210 con 0x7ffb3008ce00 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient(hunting): _renew_subs 2026-02-21T22:20:06.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient(hunting): authenticate will time out at 1415.504786s 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2ed76640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7ffb3008b7f0 legacy=0x7ffb30153030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2e575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 0x7ffb30084740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37334/0 (socket says 10.20.193.102:37334) 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2e575640 1 -- 10.20.193.102:0/2643592853 learned_addr learned my addr 10.20.193.102:0/2643592853 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2e575640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 legacy=0x7ffb30084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2f577640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6789/0 conn(0x7ffb3008d8e0 legacy=0x7ffb301953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142229711 0 0) 0x7ffb30083210 con 0x7ffb3008ce00 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): my global_id is 4631 2026-02-21T22:20:06.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffafc003620 con 0x7ffb3008ce00 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586727631 0 0) 0x7ffb30080eb0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): my global_id is 34275 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb30083210 con 0x7ffb3008d8e0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1592588696 0 0) 0x7ffb30083210 con 0x7ffb3008d8e0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb30080eb0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb200058f0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): dump: 2026-02-21T22:20:06.288 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3547891969 0 0) 0x7ffafc003620 con 0x7ffb3008ce00 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb30083210 con 0x7ffb3008ce00 2026-02-21T22:20:06.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb18002f00 con 0x7ffb3008ce00 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient(hunting): dump: 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.290 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2872163454 0 0) 0x7ffb30080eb0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb3008ce00 legacy=0x7ffb30084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.178:6789/0 conn(0x7ffb3008b7f0 legacy=0x7ffb30153030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 monclient: found mon.a 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb30082970 con 0x7ffb3008d8e0 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 5 monclient: authenticate success, global_id 34275 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _renew_subs 2026-02-21T22:20:06.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffb30081a10 con 0x7ffb3008d8e0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _renew_subs 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ffb30085020 con 0x7ffb3008d8e0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb20005bc0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb20005f20 con 0x7ffb3008d8e0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb2d573640 4 set_mon_vals no callback set 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb0f7fe640 10 monclient: dump: 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.292 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.267+0000 7ffb35281640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffb300008e0 con 0x7ffb30059f60 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ffb200061f0 con 0x7ffb3008d8e0 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(211..211 src has 1..211) ==== 3959+0+0 (unknown 2544432259 0 0) 0x7ffb2004da70 con 0x7ffb3008d8e0 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb2e575640 10 monclient: get_auth_request con 0x7ffafc03d5c0 auth_method 0 2026-02-21T22:20:06.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb2ed76640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb30059f60 legacy=0x7ffb3005a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116570736 0 0) 0x7ffb300008e0 con 0x7ffb30059f60 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient(hunting): my global_id is 4634 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb30080eb0 con 0x7ffb30059f60 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 642088659 0 0) 0x7ffb30080eb0 con 0x7ffb30059f60 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb300008e0 con 0x7ffb30059f60 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb24003360 con 0x7ffb30059f60 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient: dump: 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.294 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4191931700 0 0) 0x7ffb300008e0 con 0x7ffb30059f60 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb30153740 con 0x7ffb30059f60 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ffb240049a0 con 0x7ffb30059f60 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb30059f60 legacy=0x7ffb3005a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.271+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb30059f60 legacy=0x7ffb3005a330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.275+0000 7ffb35281640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:20:06.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.275+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffb3005ae10 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb2ed76640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb3005aa40 legacy=0x7ffb30197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1958952862 0 0) 0x7ffb3005ae10 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient(hunting): my global_id is 4637 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb300008e0 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1733437524 0 0) 0x7ffb300008e0 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb3005ae10 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffb240051b0 con 0x7ffb3005aa40 2026-02-21T22:20:06.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient: dump: 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1910252720 0 0) 0x7ffb3005ae10 con 0x7ffb3005aa40 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffb30090810 con 0x7ffb3005aa40 2026-02-21T22:20:06.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 457590329) 0x7ffb240073f0 con 0x7ffb3005aa40 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb3005aa40 legacy=0x7ffb30197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb0f7fe640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6790/0 conn(0x7ffb3005aa40 legacy=0x7ffb30197ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 24, 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 288909, 2026-02-21T22:20:06.299 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.300 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.301 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:20:06.302 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:20:06.303 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.304 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.305 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:20:06.306 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.307 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:20:06.308 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:20:06.309 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.178:6800/2549949439 conn(0x7ffafc03d5c0 legacy=0x7ffafc05da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 10 monclient: shutdown 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 >> v1:10.20.193.102:6789/0 conn(0x7ffb3008d8e0 legacy=0x7ffb301953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb2f577640 1 -- 10.20.193.102:0/2643592853 reap_dead start 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb2f577640 1 -- 10.20.193.102:0/2643592853 reap_dead start 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 shutdown_connections 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 >> 10.20.193.102:0/2643592853 conn(0x7ffb30091a50 msgr2=0x7ffb30088d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 shutdown_connections 2026-02-21T22:20:06.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:20:06.279+0000 7ffb35281640 1 -- 10.20.193.102:0/2643592853 wait complete. 2026-02-21T22:20:06.311 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:20:06.311 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:20:06.956 INFO:tasks.rados.rados.0.trial178.stdout:887: done (2 left) 2026-02-21T22:20:06.956 INFO:tasks.rados.rados.0.trial178.stdout:888: done (1 left) 2026-02-21T22:20:06.956 INFO:tasks.rados.rados.0.trial178.stdout:890: done (0 left) 2026-02-21T22:20:06.956 INFO:tasks.rados.rados.0.trial178.stdout:891: delete oid 34 current snap is 95 2026-02-21T22:20:06.958 INFO:tasks.rados.rados.0.trial178.stdout:891: done (0 left) 2026-02-21T22:20:06.958 INFO:tasks.rados.rados.0.trial178.stdout:892: write oid 20 current snap is 95 2026-02-21T22:20:06.958 INFO:tasks.rados.rados.0.trial178.stdout:892: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2026-02-21T22:20:06.965 INFO:tasks.rados.rados.0.trial178.stdout:892: writing trial17818531-20 from 511012 to 1060169 tid 1 2026-02-21T22:20:06.973 INFO:tasks.rados.rados.0.trial178.stdout:892: writing trial17818531-20 from 1775251 to 2326011 tid 2 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:892: writing trial17818531-20 from 3095195 to 3311012 tid 3 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:893: rollback oid 17 current snap is 95 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 94 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:892: finishing write tid 1 to trial17818531-20 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:892: finishing write tid 2 to trial17818531-20 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:894: read oid 30 snap -1 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:894: expect (ObjNum 308 snap 91 seq_num 308) 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:895: read oid 32 snap -1 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:895: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:06.981 INFO:tasks.rados.rados.0.trial178.stdout:896: write oid 34 current snap is 95 2026-02-21T22:20:06.982 INFO:tasks.rados.rados.0.trial178.stdout:896: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2026-02-21T22:20:06.988 INFO:tasks.rados.rados.0.trial178.stdout:896: writing trial17818531-34 from 527819 to 1152225 tid 1 2026-02-21T22:20:06.996 INFO:tasks.rados.rados.0.trial178.stdout:896: writing trial17818531-34 from 1833733 to 2244504 tid 2 2026-02-21T22:20:07.004 INFO:tasks.rados.rados.0.trial178.stdout:896: writing trial17818531-34 from 2722618 to 3324199 tid 3 2026-02-21T22:20:07.004 INFO:tasks.rados.rados.0.trial178.stdout:897: read oid 8 snap 94 2026-02-21T22:20:07.004 INFO:tasks.rados.rados.0.trial178.stdout:897: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:20:07.004 INFO:tasks.rados.rados.0.trial178.stdout:898: copy_from oid 31 from oid 45 current snap is 95 2026-02-21T22:20:07.004 INFO:tasks.rados.rados.0.trial178.stdout:894: expect (ObjNum 308 snap 91 seq_num 308) 2026-02-21T22:20:07.050 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1215903 2026-02-21T22:20:07.080 INFO:tasks.rados.rados.0.trial178.stdout:892: finishing write tid 3 to trial17818531-20 2026-02-21T22:20:07.080 INFO:tasks.rados.rados.0.trial178.stdout:892: finishing write tid 4 to trial17818531-20 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:893: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 0 (ObjNum 33 snap 0 seq_num 5) dirty dne 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: finishing write tid 5 to trial17818531-20 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: oid 20 updating version 0 to 272 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: oid 20 updating version 272 to 273 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: oid 20 updating version 273 to 274 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: oid 20 version 274 is already newer than 271 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 274 (ObjNum 316 snap 95 seq_num 316) dirty exists 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:892: left oid 20 (ObjNum 316 snap 95 seq_num 316) 2026-02-21T22:20:07.081 INFO:tasks.rados.rados.0.trial178.stdout:895: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:07.082 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:07.173 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:896: finishing write tid 1 to trial17818531-34 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:896: finishing write tid 2 to trial17818531-34 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:898: finishing copy_from to trial17818531-31 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:898: got expected ENOENT (src dne) 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:898: finishing copy_from racing read to trial17818531-31 2026-02-21T22:20:07.221 INFO:tasks.rados.rados.0.trial178.stdout:896: finishing write tid 3 to trial17818531-34 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: finishing write tid 4 to trial17818531-34 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: finishing write tid 5 to trial17818531-34 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: oid 34 updating version 0 to 155 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: oid 34 version 155 is already newer than 154 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: oid 34 updating version 155 to 157 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: oid 34 version 157 is already newer than 156 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 157 (ObjNum 317 snap 95 seq_num 317) dirty exists 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:896: left oid 34 (ObjNum 317 snap 95 seq_num 317) 2026-02-21T22:20:07.222 INFO:tasks.rados.rados.0.trial178.stdout:897: expect (ObjNum 137 snap 33 seq_num 137) 2026-02-21T22:20:07.273 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1524123 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:892: done (6 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:893: done (5 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:894: done (4 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:895: done (3 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:896: done (2 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:897: done (1 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:898: done (0 left) 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:899: write (excl) oid 26 current snap is 95 2026-02-21T22:20:07.285 INFO:tasks.rados.rados.0.trial178.stdout:899: seq_num 318 ranges {544626=699655,1892215=554429,3150041=194585} 2026-02-21T22:20:07.294 INFO:tasks.rados.rados.0.trial178.stdout:899: writing trial17818531-26 from 544626 to 1244281 tid 1 2026-02-21T22:20:07.302 INFO:tasks.rados.rados.0.trial178.stdout:899: writing trial17818531-26 from 1892215 to 2446644 tid 2 2026-02-21T22:20:07.309 INFO:tasks.rados.rados.0.trial178.stdout:899: writing trial17818531-26 from 3150041 to 3344626 tid 3 2026-02-21T22:20:07.309 INFO:tasks.rados.rados.0.trial178.stdout:900: write (excl) oid 27 current snap is 95 2026-02-21T22:20:07.309 INFO:tasks.rados.rados.0.trial178.stdout:900: seq_num 319 ranges {561433=774904,1950697=414440,2777464=583969} 2026-02-21T22:20:07.318 INFO:tasks.rados.rados.0.trial178.stdout:900: writing trial17818531-27 from 561433 to 1336337 tid 1 2026-02-21T22:20:07.325 INFO:tasks.rados.rados.0.trial178.stdout:900: writing trial17818531-27 from 1950697 to 2365137 tid 2 2026-02-21T22:20:07.332 INFO:tasks.rados.rados.0.trial178.stdout:900: writing trial17818531-27 from 2777464 to 3361433 tid 3 2026-02-21T22:20:07.332 INFO:tasks.rados.rados.0.trial178.stdout:901: write (excl) oid 46 current snap is 95 2026-02-21T22:20:07.332 INFO:tasks.rados.rados.0.trial178.stdout:901: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2026-02-21T22:20:07.336 INFO:tasks.rados.rados.0.trial178.stdout:901: writing trial17818531-46 from 578240 to 1144746 tid 1 2026-02-21T22:20:07.340 INFO:tasks.rados.rados.0.trial178.stdout:901: writing trial17818531-46 from 1609179 to 2167277 tid 2 2026-02-21T22:20:07.344 INFO:tasks.rados.rados.0.trial178.stdout:901: writing trial17818531-46 from 2804887 to 3356637 tid 3 2026-02-21T22:20:07.345 INFO:tasks.rados.rados.0.trial178.stdout:902: write oid 24 current snap is 95 2026-02-21T22:20:07.345 INFO:tasks.rados.rados.0.trial178.stdout:902: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2026-02-21T22:20:07.349 INFO:tasks.rados.rados.0.trial178.stdout:902: writing trial17818531-24 from 595047 to 1236802 tid 1 2026-02-21T22:20:07.353 INFO:tasks.rados.rados.0.trial178.stdout:902: writing trial17818531-24 from 1667661 to 2085770 tid 2 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:902: writing trial17818531-24 from 2548663 to 3311685 tid 3 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:903: delete oid 19 current snap is 95 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: finishing write tid 1 to trial17818531-26 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: finishing write tid 2 to trial17818531-26 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: finishing write tid 3 to trial17818531-26 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: finishing write tid 4 to trial17818531-26 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: finishing write tid 5 to trial17818531-26 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: oid 26 updating version 0 to 318 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: oid 26 version 318 is already newer than 316 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: oid 26 version 318 is already newer than 317 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: oid 26 updating version 318 to 319 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 319 (ObjNum 318 snap 95 seq_num 318) dirty exists 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:899: left oid 26 (ObjNum 318 snap 95 seq_num 318) 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: finishing write tid 1 to trial17818531-27 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: finishing write tid 2 to trial17818531-27 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: finishing write tid 3 to trial17818531-27 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: finishing write tid 4 to trial17818531-27 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: finishing write tid 1 to trial17818531-46 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: finishing write tid 5 to trial17818531-27 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: oid 27 updating version 0 to 348 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: oid 27 updating version 348 to 349 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: oid 27 version 349 is already newer than 347 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: oid 27 version 349 is already newer than 346 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 349 (ObjNum 319 snap 95 seq_num 319) dirty exists 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:900: left oid 27 (ObjNum 319 snap 95 seq_num 319) 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: finishing write tid 2 to trial17818531-46 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: finishing write tid 3 to trial17818531-46 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: finishing write tid 4 to trial17818531-46 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: finishing write tid 5 to trial17818531-46 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: oid 46 updating version 0 to 175 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: oid 46 updating version 175 to 176 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: oid 46 version 176 is already newer than 173 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: oid 46 version 176 is already newer than 174 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 176 (ObjNum 320 snap 95 seq_num 320) dirty exists 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:901: left oid 46 (ObjNum 320 snap 95 seq_num 320) 2026-02-21T22:20:07.358 INFO:tasks.rados.rados.0.trial178.stdout:902: finishing write tid 1 to trial17818531-24 2026-02-21T22:20:07.359 INFO:tasks.rados.rados.0.trial178.stdout:902: finishing write tid 2 to trial17818531-24 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: finishing write tid 3 to trial17818531-24 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: finishing write tid 4 to trial17818531-24 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: finishing write tid 5 to trial17818531-24 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: oid 24 updating version 0 to 448 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: oid 24 updating version 448 to 449 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: oid 24 updating version 449 to 451 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: oid 24 version 451 is already newer than 450 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 451 (ObjNum 321 snap 95 seq_num 321) dirty exists 2026-02-21T22:20:07.370 INFO:tasks.rados.rados.0.trial178.stdout:902: left oid 24 (ObjNum 321 snap 95 seq_num 321) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:899: done (4 left) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:900: done (3 left) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:901: done (2 left) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:902: done (1 left) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:903: done (0 left) 2026-02-21T22:20:07.371 INFO:tasks.rados.rados.0.trial178.stdout:904: delete oid 26 current snap is 95 2026-02-21T22:20:07.372 INFO:tasks.rados.rados.0.trial178.stdout:904: done (0 left) 2026-02-21T22:20:07.372 INFO:tasks.rados.rados.0.trial178.stdout:905: snap_remove snap 94 2026-02-21T22:20:07.406 INFO:tasks.rados.rados.0.trial178.stdout:905: done (0 left) 2026-02-21T22:20:07.406 INFO:tasks.rados.rados.0.trial178.stdout:906: read oid 50 snap -1 2026-02-21T22:20:07.406 INFO:tasks.rados.rados.0.trial178.stdout:906: expect (ObjNum 243 snap 75 seq_num 243) 2026-02-21T22:20:07.406 INFO:tasks.rados.rados.0.trial178.stdout:907: write oid 40 current snap is 95 2026-02-21T22:20:07.406 INFO:tasks.rados.rados.0.trial178.stdout:907: seq_num 324 ranges {645468=467502,1726754=565436,2798226=596838} 2026-02-21T22:20:07.410 INFO:tasks.rados.rados.0.trial178.stdout:907: writing trial17818531-40 from 645468 to 1112970 tid 1 2026-02-21T22:20:07.415 INFO:tasks.rados.rados.0.trial178.stdout:907: writing trial17818531-40 from 1726754 to 2292190 tid 2 2026-02-21T22:20:07.420 INFO:tasks.rados.rados.0.trial178.stdout:907: writing trial17818531-40 from 2798226 to 3395064 tid 3 2026-02-21T22:20:07.420 INFO:tasks.rados.rados.0.trial178.stdout:908: write (excl) oid 14 current snap is 95 2026-02-21T22:20:07.420 INFO:tasks.rados.rados.0.trial178.stdout:908: seq_num 325 ranges {662275=542751,1785236=425447,2942002=520273} 2026-02-21T22:20:07.424 INFO:tasks.rados.rados.0.trial178.stdout:908: writing trial17818531-14 from 662275 to 1205026 tid 1 2026-02-21T22:20:07.427 INFO:tasks.rados.rados.0.trial178.stdout:908: writing trial17818531-14 from 1785236 to 2210683 tid 2 2026-02-21T22:20:07.432 INFO:tasks.rados.rados.0.trial178.stdout:908: writing trial17818531-14 from 2942002 to 3462275 tid 3 2026-02-21T22:20:07.432 INFO:tasks.rados.rados.0.trial178.stdout:909: read oid 23 snap 93 2026-02-21T22:20:07.433 INFO:tasks.rados.rados.0.trial178.stdout:909: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:07.433 INFO:tasks.rados.rados.0.trial178.stdout:910: write (excl) oid 20 current snap is 95 2026-02-21T22:20:07.433 INFO:tasks.rados.rados.0.trial178.stdout:910: seq_num 326 ranges {679082=618000,1843718=569105,2853072=626010} 2026-02-21T22:20:07.437 INFO:tasks.rados.rados.0.trial178.stdout:910: writing trial17818531-20 from 679082 to 1297082 tid 1 2026-02-21T22:20:07.442 INFO:tasks.rados.rados.0.trial178.stdout:910: writing trial17818531-20 from 1843718 to 2412823 tid 2 2026-02-21T22:20:07.446 INFO:tasks.rados.rados.0.trial178.stdout:910: writing trial17818531-20 from 2853072 to 3479082 tid 3 2026-02-21T22:20:07.446 INFO:tasks.rados.rados.0.trial178.stdout:911: write (excl) oid 38 current snap is 95 2026-02-21T22:20:07.446 INFO:tasks.rados.rados.0.trial178.stdout:911: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2026-02-21T22:20:07.450 INFO:tasks.rados.rados.0.trial178.stdout:911: writing trial17818531-38 from 695889 to 1105491 tid 1 2026-02-21T22:20:07.454 INFO:tasks.rados.rados.0.trial178.stdout:911: writing trial17818531-38 from 1618553 to 2331316 tid 2 2026-02-21T22:20:07.459 INFO:tasks.rados.rados.0.trial178.stdout:911: writing trial17818531-38 from 2996848 to 3495889 tid 3 2026-02-21T22:20:07.459 INFO:tasks.rados.rados.0.trial178.stdout:912: read oid 15 snap -1 2026-02-21T22:20:07.459 INFO:tasks.rados.rados.0.trial178.stdout:912: expect deleted 2026-02-21T22:20:07.459 INFO:tasks.rados.rados.0.trial178.stdout:913: write oid 27 current snap is 95 2026-02-21T22:20:07.459 INFO:tasks.rados.rados.0.trial178.stdout:913: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2026-02-21T22:20:07.463 INFO:tasks.rados.rados.0.trial178.stdout:913: writing trial17818531-27 from 712696 to 1197547 tid 1 2026-02-21T22:20:07.468 INFO:tasks.rados.rados.0.trial178.stdout:913: writing trial17818531-27 from 1960682 to 2533456 tid 2 2026-02-21T22:20:07.472 INFO:tasks.rados.rados.0.trial178.stdout:913: writing trial17818531-27 from 3307918 to 3512696 tid 3 2026-02-21T22:20:07.473 INFO:tasks.rados.rados.0.trial178.stdout:914: delete oid 45 current snap is 95 2026-02-21T22:20:07.473 INFO:tasks.rados.rados.0.trial178.stdout:906: expect (ObjNum 243 snap 75 seq_num 243) 2026-02-21T22:20:07.503 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:07.527 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: finishing write tid 1 to trial17818531-40 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: finishing write tid 2 to trial17818531-40 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: finishing write tid 3 to trial17818531-40 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: finishing write tid 4 to trial17818531-40 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: finishing write tid 5 to trial17818531-40 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: oid 40 updating version 0 to 325 2026-02-21T22:20:07.546 INFO:tasks.rados.rados.0.trial178.stdout:907: oid 40 updating version 325 to 326 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:907: oid 40 updating version 326 to 327 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:907: oid 40 version 327 is already newer than 324 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 327 (ObjNum 324 snap 95 seq_num 324) dirty exists 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:907: left oid 40 (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:908: finishing write tid 1 to trial17818531-14 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:908: finishing write tid 2 to trial17818531-14 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:908: finishing write tid 3 to trial17818531-14 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:908: finishing write tid 4 to trial17818531-14 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:906: done (8 left) 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:907: done (7 left) 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:914: done (6 left) 2026-02-21T22:20:07.547 INFO:tasks.rados.rados.0.trial178.stdout:915: write oid 17 current snap is 95 2026-02-21T22:20:07.548 INFO:tasks.rados.rados.0.trial178.stdout:915: seq_num 330 ranges {746310=635349,2077646=576443,3362764=183546} 2026-02-21T22:20:07.551 INFO:tasks.rados.rados.0.trial178.stdout:915: writing trial17818531-17 from 746310 to 1381659 tid 1 2026-02-21T22:20:07.556 INFO:tasks.rados.rados.0.trial178.stdout:915: writing trial17818531-17 from 2077646 to 2654089 tid 2 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:915: writing trial17818531-17 from 3362764 to 3546310 tid 3 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:916: copy_from oid 26 from oid 21 current snap is 95 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:908: finishing write tid 5 to trial17818531-14 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:908: oid 14 updating version 0 to 350 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:908: oid 14 updating version 350 to 353 2026-02-21T22:20:07.560 INFO:tasks.rados.rados.0.trial178.stdout:908: oid 14 version 353 is already newer than 351 2026-02-21T22:20:07.561 INFO:tasks.rados.rados.0.trial178.stdout:908: oid 14 version 353 is already newer than 352 2026-02-21T22:20:07.561 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 353 (ObjNum 325 snap 95 seq_num 325) dirty exists 2026-02-21T22:20:07.561 INFO:tasks.rados.rados.0.trial178.stdout:908: left oid 14 (ObjNum 325 snap 95 seq_num 325) 2026-02-21T22:20:07.561 INFO:tasks.rados.rados.0.trial178.stdout:909: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: finishing write tid 1 to trial17818531-20 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: finishing write tid 2 to trial17818531-20 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: finishing write tid 3 to trial17818531-20 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: finishing write tid 4 to trial17818531-20 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: finishing write tid 5 to trial17818531-20 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: oid 20 updating version 0 to 280 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: oid 20 version 280 is already newer than 278 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: oid 20 updating version 280 to 281 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: oid 20 version 281 is already newer than 279 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 281 (ObjNum 326 snap 95 seq_num 326) dirty exists 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:910: left oid 20 (ObjNum 326 snap 95 seq_num 326) 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: finishing write tid 1 to trial17818531-38 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: finishing write tid 2 to trial17818531-38 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: finishing write tid 3 to trial17818531-38 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: finishing write tid 4 to trial17818531-38 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: finishing write tid 5 to trial17818531-38 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: oid 38 updating version 0 to 126 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: oid 38 updating version 126 to 128 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: oid 38 version 128 is already newer than 127 2026-02-21T22:20:07.670 INFO:tasks.rados.rados.0.trial178.stdout:911: oid 38 version 128 is already newer than 125 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 128 (ObjNum 327 snap 95 seq_num 327) dirty exists 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:911: left oid 38 (ObjNum 327 snap 95 seq_num 327) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: finishing write tid 1 to trial17818531-27 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: finishing write tid 2 to trial17818531-27 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: finishing write tid 3 to trial17818531-27 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: finishing write tid 4 to trial17818531-27 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: finishing write tid 5 to trial17818531-27 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: oid 27 updating version 0 to 354 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: oid 27 updating version 354 to 355 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: oid 27 updating version 355 to 356 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: oid 27 updating version 356 to 357 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 357 (ObjNum 328 snap 95 seq_num 328) dirty exists 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: left oid 27 (ObjNum 328 snap 95 seq_num 328) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: finishing write tid 1 to trial17818531-17 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: finishing write tid 2 to trial17818531-17 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: finishing write tid 3 to trial17818531-17 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: finishing write tid 4 to trial17818531-17 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: finishing write tid 5 to trial17818531-17 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: oid 17 updating version 0 to 283 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: oid 17 updating version 283 to 284 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: oid 17 version 284 is already newer than 282 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: oid 17 updating version 284 to 285 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 285 (ObjNum 330 snap 95 seq_num 330) dirty exists 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: left oid 17 (ObjNum 330 snap 95 seq_num 330) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:916: finishing copy_from racing read to trial17818531-26 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:916: finishing copy_from to trial17818531-26 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 403 (ObjNum 264 snap 81 seq_num 264) dirty exists 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:908: done (7 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:909: done (6 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:910: done (5 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:911: done (4 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:912: done (3 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:913: done (2 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:915: done (1 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:916: done (0 left) 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:917: write oid 1 current snap is 95 2026-02-21T22:20:07.671 INFO:tasks.rados.rados.0.trial178.stdout:917: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2026-02-21T22:20:07.677 INFO:tasks.rados.rados.0.trial178.stdout:917: writing trial17818531-1 from 763117 to 1473715 tid 1 2026-02-21T22:20:07.686 INFO:tasks.rados.rados.0.trial178.stdout:917: writing trial17818531-1 from 2136128 to 2856229 tid 2 2026-02-21T22:20:07.691 INFO:tasks.rados.rados.0.trial178.stdout:917: writing trial17818531-1 from 3390187 to 3563117 tid 3 2026-02-21T22:20:07.691 INFO:tasks.rados.rados.0.trial178.stdout:918: write oid 37 current snap is 95 2026-02-21T22:20:07.691 INFO:tasks.rados.rados.0.trial178.stdout:918: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2026-02-21T22:20:07.703 INFO:tasks.rados.rados.0.trial178.stdout:918: writing trial17818531-37 from 779924 to 1565771 tid 1 2026-02-21T22:20:07.710 INFO:tasks.rados.rados.0.trial178.stdout:918: writing trial17818531-37 from 2078257 to 2658369 tid 2 2026-02-21T22:20:07.714 INFO:tasks.rados.rados.0.trial178.stdout:918: writing trial17818531-37 from 3301257 to 3579924 tid 3 2026-02-21T22:20:07.714 INFO:tasks.rados.rados.0.trial178.stdout:919: write oid 14 current snap is 95 2026-02-21T22:20:07.714 INFO:tasks.rados.rados.0.trial178.stdout:919: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2026-02-21T22:20:07.719 INFO:tasks.rados.rados.0.trial178.stdout:919: writing trial17818531-14 from 796731 to 1257827 tid 1 2026-02-21T22:20:07.724 INFO:tasks.rados.rados.0.trial178.stdout:919: writing trial17818531-14 from 1736739 to 2460509 tid 2 2026-02-21T22:20:07.728 INFO:tasks.rados.rados.0.trial178.stdout:919: writing trial17818531-14 from 2928680 to 3596731 tid 3 2026-02-21T22:20:07.728 INFO:tasks.rados.rados.0.trial178.stdout:920: write (excl) oid 7 current snap is 95 2026-02-21T22:20:07.729 INFO:tasks.rados.rados.0.trial178.stdout:920: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2026-02-21T22:20:07.732 INFO:tasks.rados.rados.0.trial178.stdout:920: writing trial17818531-7 from 413538 to 949883 tid 1 2026-02-21T22:20:07.735 INFO:tasks.rados.rados.0.trial178.stdout:920: writing trial17818531-7 from 1395221 to 1979002 tid 2 2026-02-21T22:20:07.740 INFO:tasks.rados.rados.0.trial178.stdout:920: writing trial17818531-7 from 2556103 to 3214720 tid 3 2026-02-21T22:20:07.740 INFO:tasks.rados.rados.0.trial178.stdout:921: write (excl) oid 43 current snap is 95 2026-02-21T22:20:07.741 INFO:tasks.rados.rados.0.trial178.stdout:921: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2026-02-21T22:20:07.745 INFO:tasks.rados.rados.0.trial178.stdout:921: writing trial17818531-43 from 430345 to 1158292 tid 1 2026-02-21T22:20:07.749 INFO:tasks.rados.rados.0.trial178.stdout:921: writing trial17818531-43 from 1570056 to 2297495 tid 2 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:921: writing trial17818531-43 from 2699879 to 3169768 tid 3 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:922: copy_from oid 21 from oid 4 current snap is 95 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:917: finishing write tid 1 to trial17818531-1 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:917: finishing write tid 2 to trial17818531-1 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:917: finishing write tid 3 to trial17818531-1 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:917: finishing write tid 4 to trial17818531-1 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:923: rollback oid 47 current snap is 95 2026-02-21T22:20:07.754 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 93 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: finishing write tid 5 to trial17818531-1 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: oid 1 updating version 0 to 377 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: oid 1 updating version 377 to 380 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: oid 1 version 380 is already newer than 378 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: oid 1 version 380 is already newer than 379 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 380 (ObjNum 331 snap 95 seq_num 331) dirty exists 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: left oid 1 (ObjNum 331 snap 95 seq_num 331) 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:918: finishing write tid 1 to trial17818531-37 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:917: done (6 left) 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:924: delete oid 27 current snap is 95 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:918: finishing write tid 2 to trial17818531-37 2026-02-21T22:20:07.755 INFO:tasks.rados.rados.0.trial178.stdout:918: finishing write tid 3 to trial17818531-37 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: finishing write tid 4 to trial17818531-37 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: finishing write tid 5 to trial17818531-37 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: oid 37 updating version 0 to 382 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: oid 37 updating version 382 to 383 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: oid 37 version 383 is already newer than 381 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: oid 37 updating version 383 to 384 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 384 (ObjNum 332 snap 95 seq_num 332) dirty exists 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:918: left oid 37 (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:919: finishing write tid 1 to trial17818531-14 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:919: finishing write tid 2 to trial17818531-14 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:919: finishing write tid 3 to trial17818531-14 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:919: finishing write tid 4 to trial17818531-14 2026-02-21T22:20:07.756 INFO:tasks.rados.rados.0.trial178.stdout:919: finishing write tid 5 to trial17818531-14 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:919: oid 14 updating version 0 to 358 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:919: oid 14 updating version 358 to 359 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:919: oid 14 updating version 359 to 360 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:919: oid 14 updating version 360 to 361 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 361 (ObjNum 333 snap 95 seq_num 333) dirty exists 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:919: left oid 14 (ObjNum 333 snap 95 seq_num 333) 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: finishing write tid 1 to trial17818531-7 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: finishing write tid 2 to trial17818531-7 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: finishing write tid 3 to trial17818531-7 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: finishing write tid 4 to trial17818531-7 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: finishing write tid 5 to trial17818531-7 2026-02-21T22:20:07.757 INFO:tasks.rados.rados.0.trial178.stdout:920: oid 7 updating version 0 to 458 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:920: oid 7 updating version 458 to 461 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:920: oid 7 version 461 is already newer than 460 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:920: oid 7 version 461 is already newer than 459 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 461 (ObjNum 334 snap 95 seq_num 334) dirty exists 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:920: left oid 7 (ObjNum 334 snap 95 seq_num 334) 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:921: finishing write tid 1 to trial17818531-43 2026-02-21T22:20:07.758 INFO:tasks.rados.rados.0.trial178.stdout:921: finishing write tid 2 to trial17818531-43 2026-02-21T22:20:07.759 INFO:tasks.rados.rados.0.trial178.stdout:922: finishing copy_from to trial17818531-21 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:922: got expected ENOENT (src dne) 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:922: finishing copy_from racing read to trial17818531-21 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:923: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:923: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:923: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 465 (ObjNum 232 snap 72 seq_num 232) dirty exists 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: finishing write tid 3 to trial17818531-43 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: finishing write tid 4 to trial17818531-43 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: finishing write tid 5 to trial17818531-43 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: oid 43 updating version 0 to 105 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: oid 43 updating version 105 to 107 2026-02-21T22:20:07.760 INFO:tasks.rados.rados.0.trial178.stdout:921: oid 43 updating version 107 to 108 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:921: oid 43 version 108 is already newer than 106 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 108 (ObjNum 335 snap 95 seq_num 335) dirty exists 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:921: left oid 43 (ObjNum 335 snap 95 seq_num 335) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:918: done (6 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:919: done (5 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:920: done (4 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:921: done (3 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:922: done (2 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:923: done (1 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:924: done (0 left) 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:925: write oid 43 current snap is 95 2026-02-21T22:20:07.761 INFO:tasks.rados.rados.0.trial178.stdout:925: seq_num 337 ranges {463959=478445,1570667=731108,3038372=492433} 2026-02-21T22:20:07.762 INFO:tasks.rados.rados.0.trial178.stdout:925: writing trial17818531-43 from 463959 to 942404 tid 1 2026-02-21T22:20:07.766 INFO:tasks.rados.rados.0.trial178.stdout:925: writing trial17818531-43 from 1570667 to 2301775 tid 2 2026-02-21T22:20:07.770 INFO:tasks.rados.rados.0.trial178.stdout:925: writing trial17818531-43 from 3038372 to 3530805 tid 3 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:926: rollback oid 31 current snap is 95 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 93 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:925: finishing write tid 1 to trial17818531-43 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:925: finishing write tid 2 to trial17818531-43 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:927: read oid 33 snap -1 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:927: expect deleted 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:928: write (excl) oid 3 current snap is 95 2026-02-21T22:20:07.771 INFO:tasks.rados.rados.0.trial178.stdout:928: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2026-02-21T22:20:07.774 INFO:tasks.rados.rados.0.trial178.stdout:928: writing trial17818531-3 from 480766 to 1034460 tid 1 2026-02-21T22:20:07.779 INFO:tasks.rados.rados.0.trial178.stdout:928: writing trial17818531-3 from 1629149 to 2220268 tid 2 2026-02-21T22:20:07.783 INFO:tasks.rados.rados.0.trial178.stdout:928: writing trial17818531-3 from 2782148 to 3202206 tid 3 2026-02-21T22:20:07.783 INFO:tasks.rados.rados.0.trial178.stdout:929: write oid 12 current snap is 95 2026-02-21T22:20:07.783 INFO:tasks.rados.rados.0.trial178.stdout:929: seq_num 339 ranges {497573=628943,1687631=734777,3093218=604355} 2026-02-21T22:20:07.787 INFO:tasks.rados.rados.0.trial178.stdout:929: writing trial17818531-12 from 497573 to 1126516 tid 1 2026-02-21T22:20:07.792 INFO:tasks.rados.rados.0.trial178.stdout:929: writing trial17818531-12 from 1687631 to 2422408 tid 2 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:929: writing trial17818531-12 from 3093218 to 3697573 tid 3 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:930: read oid 37 snap -1 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:930: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:931: read oid 48 snap -1 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:931: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:932: snap_create 2026-02-21T22:20:07.798 INFO:tasks.rados.rados.0.trial178.stdout:926: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:926: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: finishing write tid 3 to trial17818531-43 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:926: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 325 (ObjNum 239 snap 75 seq_num 239) dirty exists 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: finishing write tid 4 to trial17818531-43 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: finishing write tid 5 to trial17818531-43 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: oid 43 updating version 0 to 110 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: oid 43 version 110 is already newer than 109 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: oid 43 updating version 110 to 112 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: oid 43 version 112 is already newer than 111 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 112 (ObjNum 337 snap 95 seq_num 337) dirty exists 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:925: left oid 43 (ObjNum 337 snap 95 seq_num 337) 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:928: finishing write tid 1 to trial17818531-3 2026-02-21T22:20:07.799 INFO:tasks.rados.rados.0.trial178.stdout:928: finishing write tid 2 to trial17818531-3 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: finishing write tid 3 to trial17818531-3 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: finishing write tid 4 to trial17818531-3 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: finishing write tid 5 to trial17818531-3 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: oid 3 updating version 0 to 466 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: oid 3 version 466 is already newer than 465 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: oid 3 version 466 is already newer than 464 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: oid 3 version 466 is already newer than 463 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 466 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:928: left oid 3 (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:929: finishing write tid 1 to trial17818531-12 2026-02-21T22:20:07.800 INFO:tasks.rados.rados.0.trial178.stdout:929: finishing write tid 2 to trial17818531-12 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: finishing write tid 3 to trial17818531-12 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: finishing write tid 4 to trial17818531-12 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: finishing write tid 5 to trial17818531-12 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: oid 12 updating version 0 to 316 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: oid 12 version 316 is already newer than 314 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: oid 12 updating version 316 to 317 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: oid 12 version 317 is already newer than 315 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 317 (ObjNum 339 snap 95 seq_num 339) dirty exists 2026-02-21T22:20:07.803 INFO:tasks.rados.rados.0.trial178.stdout:929: left oid 12 (ObjNum 339 snap 95 seq_num 339) 2026-02-21T22:20:07.805 INFO:tasks.rados.rados.0.trial178.stdout:930: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:07.899 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3016699 2026-02-21T22:20:07.906 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:07.950 INFO:tasks.rados.rados.0.trial178.stdout:931: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:07.951 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:925: done (7 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:926: done (6 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:927: done (5 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:928: done (4 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:929: done (3 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:930: done (2 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:931: done (1 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:932: done (0 left) 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:933: rollback oid 38 current snap is 96 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 95 2026-02-21T22:20:08.411 INFO:tasks.rados.rados.0.trial178.stdout:934: snap_remove snap 93 2026-02-21T22:20:08.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:08.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:934: done (1 left) 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:935: delete oid 26 current snap is 96 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:933: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:933: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:933: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:20:09.412 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 131 (ObjNum 327 snap 95 seq_num 327) dirty exists 2026-02-21T22:20:09.415 INFO:tasks.rados.rados.0.trial178.stdout:933: done (1 left) 2026-02-21T22:20:09.415 INFO:tasks.rados.rados.0.trial178.stdout:935: done (0 left) 2026-02-21T22:20:09.415 INFO:tasks.rados.rados.0.trial178.stdout:936: snap_remove snap 95 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:936: done (0 left) 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:937: read oid 19 snap -1 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:937: expect deleted 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:938: read oid 48 snap -1 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:938: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:939: write (excl) oid 30 current snap is 96 2026-02-21T22:20:10.417 INFO:tasks.rados.rados.0.trial178.stdout:939: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2026-02-21T22:20:10.426 INFO:tasks.rados.rados.0.trial178.stdout:939: writing trial17818531-30 from 531187 to 1310628 tid 1 2026-02-21T22:20:10.436 INFO:tasks.rados.rados.0.trial178.stdout:939: writing trial17818531-30 from 2088242 to 2826688 tid 2 2026-02-21T22:20:10.443 INFO:tasks.rados.rados.0.trial178.stdout:939: writing trial17818531-30 from 3431711 to 3731187 tid 3 2026-02-21T22:20:10.443 INFO:tasks.rados.rados.0.trial178.stdout:940: delete oid 6 current snap is 96 2026-02-21T22:20:10.443 INFO:tasks.rados.rados.0.trial178.stdout:938: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:10.443 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:10.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:10.537 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: finishing write tid 1 to trial17818531-30 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: finishing write tid 2 to trial17818531-30 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: finishing write tid 3 to trial17818531-30 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: finishing write tid 4 to trial17818531-30 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: finishing write tid 5 to trial17818531-30 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: oid 30 updating version 0 to 257 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: oid 30 updating version 257 to 259 2026-02-21T22:20:10.585 INFO:tasks.rados.rados.0.trial178.stdout:939: oid 30 version 259 is already newer than 258 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:939: oid 30 version 259 is already newer than 256 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 259 (ObjNum 341 snap 96 seq_num 341) dirty exists 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:939: left oid 30 (ObjNum 341 snap 96 seq_num 341) 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:937: done (3 left) 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:938: done (2 left) 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:939: done (1 left) 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:940: done (0 left) 2026-02-21T22:20:10.586 INFO:tasks.rados.rados.0.trial178.stdout:941: snap_create 2026-02-21T22:20:11.417 INFO:tasks.rados.rados.0.trial178.stdout:941: done (0 left) 2026-02-21T22:20:11.418 INFO:tasks.rados.rados.0.trial178.stdout:942: snap_remove snap 96 2026-02-21T22:20:12.419 INFO:tasks.rados.rados.0.trial178.stdout:942: done (0 left) 2026-02-21T22:20:12.419 INFO:tasks.rados.rados.0.trial178.stdout:943: write oid 39 current snap is 97 2026-02-21T22:20:12.419 INFO:tasks.rados.rados.0.trial178.stdout:943: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2026-02-21T22:20:12.427 INFO:tasks.rados.rados.0.trial178.stdout:943: writing trial17818531-39 from 564801 to 1211093 tid 1 2026-02-21T22:20:12.436 INFO:tasks.rados.rados.0.trial178.stdout:943: writing trial17818531-39 from 1921559 to 2663674 tid 2 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:943: writing trial17818531-39 from 3202910 to 3764801 tid 3 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:944: read oid 19 snap -1 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:944: expect deleted 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:945: write (excl) oid 48 current snap is 97 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:943: finishing write tid 1 to trial17818531-39 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:943: finishing write tid 2 to trial17818531-39 2026-02-21T22:20:12.444 INFO:tasks.rados.rados.0.trial178.stdout:945: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2026-02-21T22:20:12.453 INFO:tasks.rados.rados.0.trial178.stdout:945: writing trial17818531-48 from 581608 to 1303149 tid 1 2026-02-21T22:20:12.460 INFO:tasks.rados.rados.0.trial178.stdout:945: writing trial17818531-48 from 1863688 to 2349461 tid 2 2026-02-21T22:20:12.469 INFO:tasks.rados.rados.0.trial178.stdout:945: writing trial17818531-48 from 3113980 to 3718023 tid 3 2026-02-21T22:20:12.469 INFO:tasks.rados.rados.0.trial178.stdout:946: read oid 32 snap -1 2026-02-21T22:20:12.469 INFO:tasks.rados.rados.0.trial178.stdout:946: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:947: snap_create 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: finishing write tid 3 to trial17818531-39 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: finishing write tid 4 to trial17818531-39 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: finishing write tid 5 to trial17818531-39 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: oid 39 updating version 0 to 364 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: oid 39 version 364 is already newer than 363 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: oid 39 updating version 364 to 366 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: oid 39 version 366 is already newer than 365 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 366 (ObjNum 343 snap 97 seq_num 343) dirty exists 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:943: left oid 39 (ObjNum 343 snap 97 seq_num 343) 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:945: finishing write tid 1 to trial17818531-48 2026-02-21T22:20:12.470 INFO:tasks.rados.rados.0.trial178.stdout:945: finishing write tid 2 to trial17818531-48 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: finishing write tid 3 to trial17818531-48 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: finishing write tid 4 to trial17818531-48 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: finishing write tid 5 to trial17818531-48 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: oid 48 updating version 0 to 387 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: oid 48 version 387 is already newer than 385 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: oid 48 version 387 is already newer than 386 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: oid 48 updating version 387 to 388 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 388 (ObjNum 344 snap 97 seq_num 344) dirty exists 2026-02-21T22:20:12.472 INFO:tasks.rados.rados.0.trial178.stdout:945: left oid 48 (ObjNum 344 snap 97 seq_num 344) 2026-02-21T22:20:12.479 INFO:tasks.rados.rados.0.trial178.stdout:946: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:12.479 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:943: done (4 left) 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:944: done (3 left) 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:945: done (2 left) 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:946: done (1 left) 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:947: done (0 left) 2026-02-21T22:20:13.428 INFO:tasks.rados.rados.0.trial178.stdout:948: read oid 22 snap -1 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:948: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:949: rollback oid 19 current snap is 98 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 97 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:950: read oid 18 snap -1 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:950: expect (ObjNum 273 snap 84 seq_num 273) 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:951: write (excl) oid 26 current snap is 98 2026-02-21T22:20:13.429 INFO:tasks.rados.rados.0.trial178.stdout:951: seq_num 345 ranges {598415=796790,1922170=745784,3141403=415315} 2026-02-21T22:20:13.438 INFO:tasks.rados.rados.0.trial178.stdout:951: writing trial17818531-26 from 598415 to 1395205 tid 1 2026-02-21T22:20:13.447 INFO:tasks.rados.rados.0.trial178.stdout:951: writing trial17818531-26 from 1922170 to 2667954 tid 2 2026-02-21T22:20:13.453 INFO:tasks.rados.rados.0.trial178.stdout:951: writing trial17818531-26 from 3141403 to 3556718 tid 3 2026-02-21T22:20:13.453 INFO:tasks.rados.rados.0.trial178.stdout:952: delete oid 20 current snap is 98 2026-02-21T22:20:13.453 INFO:tasks.rados.rados.0.trial178.stdout:949: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:20:13.453 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:20:13.453 INFO:tasks.rados.rados.0.trial178.stdout:950: expect (ObjNum 273 snap 84 seq_num 273) 2026-02-21T22:20:13.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:13.495 INFO:tasks.rados.rados.0.trial178.stdout:948: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:13.516 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2320374 2026-02-21T22:20:13.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:13.548 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:13.621 INFO:tasks.rados.rados.0.trial178.stdout:951: finishing write tid 1 to trial17818531-26 2026-02-21T22:20:13.621 INFO:tasks.rados.rados.0.trial178.stdout:951: finishing write tid 2 to trial17818531-26 2026-02-21T22:20:13.621 INFO:tasks.rados.rados.0.trial178.stdout:951: finishing write tid 3 to trial17818531-26 2026-02-21T22:20:13.621 INFO:tasks.rados.rados.0.trial178.stdout:951: finishing write tid 4 to trial17818531-26 2026-02-21T22:20:13.621 INFO:tasks.rados.rados.0.trial178.stdout:951: finishing write tid 5 to trial17818531-26 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:951: oid 26 updating version 0 to 406 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:951: oid 26 updating version 406 to 407 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:951: oid 26 updating version 407 to 408 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:951: oid 26 updating version 408 to 409 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 409 (ObjNum 345 snap 98 seq_num 345) dirty exists 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:951: left oid 26 (ObjNum 345 snap 98 seq_num 345) 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:948: done (4 left) 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:949: done (3 left) 2026-02-21T22:20:13.622 INFO:tasks.rados.rados.0.trial178.stdout:950: done (2 left) 2026-02-21T22:20:13.623 INFO:tasks.rados.rados.0.trial178.stdout:951: done (1 left) 2026-02-21T22:20:13.623 INFO:tasks.rados.rados.0.trial178.stdout:952: done (0 left) 2026-02-21T22:20:13.623 INFO:tasks.rados.rados.0.trial178.stdout:953: delete oid 48 current snap is 98 2026-02-21T22:20:13.623 INFO:tasks.rados.rados.0.trial178.stdout:953: done (0 left) 2026-02-21T22:20:13.624 INFO:tasks.rados.rados.0.trial178.stdout:954: snap_create 2026-02-21T22:20:14.430 INFO:tasks.rados.rados.0.trial178.stdout:954: done (0 left) 2026-02-21T22:20:14.430 INFO:tasks.rados.rados.0.trial178.stdout:955: write (excl) oid 5 current snap is 99 2026-02-21T22:20:14.430 INFO:tasks.rados.rados.0.trial178.stdout:955: seq_num 348 ranges {648836=622537,1981263=493111,3107319=741517} 2026-02-21T22:20:14.440 INFO:tasks.rados.rados.0.trial178.stdout:955: writing trial17818531-5 from 648836 to 1271373 tid 1 2026-02-21T22:20:14.448 INFO:tasks.rados.rados.0.trial178.stdout:955: writing trial17818531-5 from 1981263 to 2474374 tid 2 2026-02-21T22:20:14.457 INFO:tasks.rados.rados.0.trial178.stdout:955: writing trial17818531-5 from 3107319 to 3848836 tid 3 2026-02-21T22:20:14.458 INFO:tasks.rados.rados.0.trial178.stdout:956: write oid 34 current snap is 99 2026-02-21T22:20:14.458 INFO:tasks.rados.rados.0.trial178.stdout:956: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2026-02-21T22:20:14.466 INFO:tasks.rados.rados.0.trial178.stdout:956: writing trial17818531-34 from 665643 to 1363429 tid 1 2026-02-21T22:20:14.475 INFO:tasks.rados.rados.0.trial178.stdout:956: writing trial17818531-34 from 2039745 to 2792867 tid 2 2026-02-21T22:20:14.479 INFO:tasks.rados.rados.0.trial178.stdout:956: writing trial17818531-34 from 3534742 to 3865643 tid 3 2026-02-21T22:20:14.479 INFO:tasks.rados.rados.0.trial178.stdout:957: copy_from oid 50 from oid 9 current snap is 99 2026-02-21T22:20:14.480 INFO:tasks.rados.rados.0.trial178.stdout:955: finishing write tid 1 to trial17818531-5 2026-02-21T22:20:14.480 INFO:tasks.rados.rados.0.trial178.stdout:955: finishing write tid 2 to trial17818531-5 2026-02-21T22:20:14.480 INFO:tasks.rados.rados.0.trial178.stdout:958: write (excl) oid 16 current snap is 99 2026-02-21T22:20:14.480 INFO:tasks.rados.rados.0.trial178.stdout:958: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2026-02-21T22:20:14.484 INFO:tasks.rados.rados.0.trial178.stdout:958: writing trial17818531-16 from 682450 to 1171838 tid 1 2026-02-21T22:20:14.488 INFO:tasks.rados.rados.0.trial178.stdout:958: writing trial17818531-16 from 1814580 to 2311360 tid 2 2026-02-21T22:20:14.493 INFO:tasks.rados.rados.0.trial178.stdout:958: writing trial17818531-16 from 2878518 to 3666546 tid 3 2026-02-21T22:20:14.494 INFO:tasks.rados.rados.0.trial178.stdout:959: write oid 4 current snap is 99 2026-02-21T22:20:14.494 INFO:tasks.rados.rados.0.trial178.stdout:959: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2026-02-21T22:20:14.498 INFO:tasks.rados.rados.0.trial178.stdout:959: writing trial17818531-4 from 699257 to 1263894 tid 1 2026-02-21T22:20:14.503 INFO:tasks.rados.rados.0.trial178.stdout:959: writing trial17818531-4 from 1873062 to 2513500 tid 2 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:959: writing trial17818531-4 from 3305941 to 3899257 tid 3 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:960: snap_create 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: finishing write tid 3 to trial17818531-5 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: finishing write tid 4 to trial17818531-5 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: finishing write tid 5 to trial17818531-5 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: oid 5 updating version 0 to 263 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: oid 5 version 263 is already newer than 262 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: oid 5 updating version 263 to 264 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: oid 5 version 264 is already newer than 261 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 264 (ObjNum 348 snap 99 seq_num 348) dirty exists 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:955: left oid 5 (ObjNum 348 snap 99 seq_num 348) 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: finishing write tid 1 to trial17818531-34 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: finishing write tid 2 to trial17818531-34 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: finishing write tid 3 to trial17818531-34 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: finishing write tid 4 to trial17818531-34 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: finishing write tid 5 to trial17818531-34 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: oid 34 updating version 0 to 166 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: oid 34 version 166 is already newer than 164 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: oid 34 version 166 is already newer than 165 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: oid 34 version 166 is already newer than 163 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 166 (ObjNum 349 snap 99 seq_num 349) dirty exists 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:956: left oid 34 (ObjNum 349 snap 99 seq_num 349) 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: finishing write tid 1 to trial17818531-16 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: finishing write tid 2 to trial17818531-16 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:957: finishing copy_from racing read to trial17818531-50 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:957: finishing copy_from to trial17818531-50 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 345 (ObjNum 312 snap 91 seq_num 312) dirty exists 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: finishing write tid 3 to trial17818531-16 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: finishing write tid 4 to trial17818531-16 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: finishing write tid 5 to trial17818531-16 2026-02-21T22:20:14.509 INFO:tasks.rados.rados.0.trial178.stdout:958: oid 16 updating version 0 to 318 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:958: oid 16 updating version 318 to 319 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:958: oid 16 updating version 319 to 321 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:958: oid 16 version 321 is already newer than 320 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 321 (ObjNum 350 snap 99 seq_num 350) dirty exists 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:958: left oid 16 (ObjNum 350 snap 99 seq_num 350) 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: finishing write tid 1 to trial17818531-4 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: finishing write tid 2 to trial17818531-4 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: finishing write tid 3 to trial17818531-4 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: finishing write tid 4 to trial17818531-4 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: finishing write tid 5 to trial17818531-4 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: oid 4 updating version 0 to 323 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: oid 4 version 323 is already newer than 322 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: oid 4 updating version 323 to 324 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: oid 4 updating version 324 to 325 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 325 (ObjNum 351 snap 99 seq_num 351) dirty exists 2026-02-21T22:20:14.510 INFO:tasks.rados.rados.0.trial178.stdout:959: left oid 4 (ObjNum 351 snap 99 seq_num 351) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:955: done (5 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:956: done (4 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:957: done (3 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:958: done (2 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:959: done (1 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:960: done (0 left) 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:961: write (excl) oid 36 current snap is 100 2026-02-21T22:20:15.434 INFO:tasks.rados.rados.0.trial178.stdout:961: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2026-02-21T22:20:15.440 INFO:tasks.rados.rados.0.trial178.stdout:961: writing trial17818531-36 from 716064 to 1355950 tid 1 2026-02-21T22:20:15.444 INFO:tasks.rados.rados.0.trial178.stdout:961: writing trial17818531-36 from 1815191 to 2315640 tid 2 2026-02-21T22:20:15.448 INFO:tasks.rados.rados.0.trial178.stdout:961: writing trial17818531-36 from 2817011 to 3343936 tid 3 2026-02-21T22:20:15.448 INFO:tasks.rados.rados.0.trial178.stdout:962: read oid 37 snap -1 2026-02-21T22:20:15.448 INFO:tasks.rados.rados.0.trial178.stdout:962: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:15.448 INFO:tasks.rados.rados.0.trial178.stdout:963: write oid 32 current snap is 100 2026-02-21T22:20:15.448 INFO:tasks.rados.rados.0.trial178.stdout:963: seq_num 353 ranges {732871=715135,1873673=644107,3244434=688437} 2026-02-21T22:20:15.454 INFO:tasks.rados.rados.0.trial178.stdout:963: writing trial17818531-32 from 732871 to 1448006 tid 1 2026-02-21T22:20:15.458 INFO:tasks.rados.rados.0.trial178.stdout:963: writing trial17818531-32 from 1873673 to 2517780 tid 2 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:963: writing trial17818531-32 from 3244434 to 3932871 tid 3 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:964: delete oid 24 current snap is 100 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: finishing write tid 1 to trial17818531-36 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: finishing write tid 2 to trial17818531-36 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: finishing write tid 3 to trial17818531-36 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: finishing write tid 4 to trial17818531-36 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: finishing write tid 5 to trial17818531-36 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: oid 36 updating version 0 to 166 2026-02-21T22:20:15.464 INFO:tasks.rados.rados.0.trial178.stdout:961: oid 36 version 166 is already newer than 163 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:961: oid 36 version 166 is already newer than 164 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:961: oid 36 version 166 is already newer than 165 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 166 (ObjNum 352 snap 100 seq_num 352) dirty exists 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:961: left oid 36 (ObjNum 352 snap 100 seq_num 352) 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:962: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:15.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:15.539 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1282397 2026-02-21T22:20:15.572 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:15.618 INFO:tasks.rados.rados.0.trial178.stdout:963: finishing write tid 1 to trial17818531-32 2026-02-21T22:20:15.618 INFO:tasks.rados.rados.0.trial178.stdout:963: finishing write tid 2 to trial17818531-32 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: finishing write tid 3 to trial17818531-32 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:961: done (3 left) 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:962: done (2 left) 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:964: done (1 left) 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:965: read oid 46 snap -1 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:965: expect (ObjNum 320 snap 95 seq_num 320) 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:966: snap_create 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: finishing write tid 4 to trial17818531-32 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: finishing write tid 5 to trial17818531-32 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: oid 32 updating version 0 to 471 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: oid 32 updating version 471 to 472 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: oid 32 version 472 is already newer than 470 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: oid 32 updating version 472 to 473 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 473 (ObjNum 353 snap 100 seq_num 353) dirty exists 2026-02-21T22:20:15.619 INFO:tasks.rados.rados.0.trial178.stdout:963: left oid 32 (ObjNum 353 snap 100 seq_num 353) 2026-02-21T22:20:15.623 INFO:tasks.rados.rados.0.trial178.stdout:965: expect (ObjNum 320 snap 95 seq_num 320) 2026-02-21T22:20:15.623 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:15.696 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 578270 2026-02-21T22:20:16.436 INFO:tasks.rados.rados.0.trial178.stdout:963: done (2 left) 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:965: done (1 left) 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:966: done (0 left) 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:967: read oid 25 snap -1 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:967: expect (ObjNum 301 snap 89 seq_num 301) 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:968: read oid 20 snap -1 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:968: expect deleted 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:969: write (excl) oid 8 current snap is 101 2026-02-21T22:20:16.437 INFO:tasks.rados.rados.0.trial178.stdout:969: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2026-02-21T22:20:16.441 INFO:tasks.rados.rados.0.trial178.stdout:969: writing trial17818531-8 from 766485 to 1232118 tid 1 2026-02-21T22:20:16.446 INFO:tasks.rados.rados.0.trial178.stdout:969: writing trial17818531-8 from 1990637 to 2638413 tid 2 2026-02-21T22:20:16.451 INFO:tasks.rados.rados.0.trial178.stdout:969: writing trial17818531-8 from 3299280 to 3966485 tid 3 2026-02-21T22:20:16.452 INFO:tasks.rados.rados.0.trial178.stdout:970: write oid 13 current snap is 101 2026-02-21T22:20:16.452 INFO:tasks.rados.rados.0.trial178.stdout:970: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2026-02-21T22:20:16.456 INFO:tasks.rados.rados.0.trial178.stdout:970: writing trial17818531-13 from 783292 to 1324174 tid 1 2026-02-21T22:20:16.461 INFO:tasks.rados.rados.0.trial178.stdout:970: writing trial17818531-13 from 1932766 to 2440553 tid 2 2026-02-21T22:20:16.466 INFO:tasks.rados.rados.0.trial178.stdout:970: writing trial17818531-13 from 3210350 to 3782363 tid 3 2026-02-21T22:20:16.466 INFO:tasks.rados.rados.0.trial178.stdout:971: rollback oid 41 current snap is 101 2026-02-21T22:20:16.466 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 100 2026-02-21T22:20:16.466 INFO:tasks.rados.rados.0.trial178.stdout:967: expect (ObjNum 301 snap 89 seq_num 301) 2026-02-21T22:20:16.514 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1116154 2026-02-21T22:20:16.547 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: finishing write tid 1 to trial17818531-8 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: finishing write tid 2 to trial17818531-8 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: finishing write tid 3 to trial17818531-8 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:967: done (4 left) 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:968: done (3 left) 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:972: delete oid 33 current snap is 101 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: finishing write tid 4 to trial17818531-8 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: finishing write tid 5 to trial17818531-8 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: oid 8 updating version 0 to 475 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: oid 8 updating version 475 to 476 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: oid 8 updating version 476 to 478 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: oid 8 version 478 is already newer than 477 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 478 (ObjNum 355 snap 101 seq_num 355) dirty exists 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:969: left oid 8 (ObjNum 355 snap 101 seq_num 355) 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: finishing write tid 1 to trial17818531-13 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: finishing write tid 2 to trial17818531-13 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: finishing write tid 3 to trial17818531-13 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: finishing write tid 4 to trial17818531-13 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: finishing write tid 5 to trial17818531-13 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: oid 13 updating version 0 to 477 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: oid 13 version 477 is already newer than 475 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: oid 13 updating version 477 to 478 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: oid 13 version 478 is already newer than 476 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 478 (ObjNum 356 snap 101 seq_num 356) dirty exists 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:970: left oid 13 (ObjNum 356 snap 101 seq_num 356) 2026-02-21T22:20:16.590 INFO:tasks.rados.rados.0.trial178.stdout:971: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:971: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:971: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 293 (ObjNum 188 snap 56 seq_num 188) dirty exists 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:969: done (3 left) 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:970: done (2 left) 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:971: done (1 left) 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:972: done (0 left) 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:973: rollback oid 8 current snap is 101 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 97 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:974: rollback oid 28 current snap is 101 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 100 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:975: write oid 30 current snap is 101 2026-02-21T22:20:16.591 INFO:tasks.rados.rados.0.trial178.stdout:975: seq_num 358 ranges {416906=407733,1366083=511456} 2026-02-21T22:20:16.595 INFO:tasks.rados.rados.0.trial178.stdout:975: writing trial17818531-30 from 416906 to 824639 tid 1 2026-02-21T22:20:16.602 INFO:tasks.rados.rados.0.trial178.stdout:975: writing trial17818531-30 from 1366083 to 1877539 tid 2 2026-02-21T22:20:16.602 INFO:tasks.rados.rados.0.trial178.stdout:976: delete oid 39 current snap is 101 2026-02-21T22:20:16.602 INFO:tasks.rados.rados.0.trial178.stdout:974: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:20:16.602 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:20:16.603 INFO:tasks.rados.rados.0.trial178.stdout:973: finishing rollback tid 0 to trial17818531-8 2026-02-21T22:20:16.603 INFO:tasks.rados.rados.0.trial178.stdout:973: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:20:16.603 INFO:tasks.rados.rados.0.trial178.stdout:973: finishing rollback tid 2 to trial17818531-8 2026-02-21T22:20:16.603 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 481 (ObjNum 137 snap 33 seq_num 137) dirty exists 2026-02-21T22:20:16.603 INFO:tasks.rados.rados.0.trial178.stdout:975: finishing write tid 1 to trial17818531-30 2026-02-21T22:20:16.604 INFO:tasks.rados.rados.0.trial178.stdout:973: done (3 left) 2026-02-21T22:20:16.604 INFO:tasks.rados.rados.0.trial178.stdout:974: done (2 left) 2026-02-21T22:20:16.604 INFO:tasks.rados.rados.0.trial178.stdout:976: done (1 left) 2026-02-21T22:20:16.604 INFO:tasks.rados.rados.0.trial178.stdout:977: delete oid 48 current snap is 101 2026-02-21T22:20:16.605 INFO:tasks.rados.rados.0.trial178.stdout:977: done (1 left) 2026-02-21T22:20:16.605 INFO:tasks.rados.rados.0.trial178.stdout:978: delete oid 4 current snap is 101 2026-02-21T22:20:16.607 INFO:tasks.rados.rados.0.trial178.stdout:978: done (1 left) 2026-02-21T22:20:16.607 INFO:tasks.rados.rados.0.trial178.stdout:979: rollback oid 2 current snap is 101 2026-02-21T22:20:16.607 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 100 2026-02-21T22:20:16.607 INFO:tasks.rados.rados.0.trial178.stdout:980: snap_create 2026-02-21T22:20:16.608 INFO:tasks.rados.rados.0.trial178.stdout:979: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:20:16.608 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 0 (ObjNum 321 snap 0 seq_num 1453350989) dirty dne 2026-02-21T22:20:16.609 INFO:tasks.rados.rados.0.trial178.stdout:975: finishing write tid 2 to trial17818531-30 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: finishing write tid 3 to trial17818531-30 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: finishing write tid 4 to trial17818531-30 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: oid 30 updating version 0 to 266 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: oid 30 updating version 266 to 267 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: oid 30 updating version 267 to 268 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 268 (ObjNum 358 snap 101 seq_num 358) dirty exists 2026-02-21T22:20:16.610 INFO:tasks.rados.rados.0.trial178.stdout:975: left oid 30 (ObjNum 358 snap 101 seq_num 358) 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:975: done (2 left) 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:979: done (1 left) 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:980: done (0 left) 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:981: copy_from oid 10 from oid 29 current snap is 102 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:982: write oid 32 current snap is 102 2026-02-21T22:20:17.408 INFO:tasks.rados.rados.0.trial178.stdout:982: seq_num 362 ranges {484134=708729,1883658=200476} 2026-02-21T22:20:17.416 INFO:tasks.rados.rados.0.trial178.stdout:982: writing trial17818531-32 from 484134 to 1192863 tid 1 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:982: writing trial17818531-32 from 1883658 to 2084134 tid 2 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:983: snap_create 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:981: finishing copy_from racing read to trial17818531-10 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:982: finishing write tid 1 to trial17818531-32 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:981: finishing copy_from to trial17818531-10 2026-02-21T22:20:17.422 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 149 (ObjNum 299 snap 88 seq_num 299) dirty exists 2026-02-21T22:20:17.424 INFO:tasks.rados.rados.0.trial178.stdout:982: finishing write tid 2 to trial17818531-32 2026-02-21T22:20:17.424 INFO:tasks.rados.rados.0.trial178.stdout:982: finishing write tid 3 to trial17818531-32 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:982: finishing write tid 4 to trial17818531-32 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:982: oid 32 updating version 0 to 482 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:982: oid 32 version 482 is already newer than 480 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:982: oid 32 version 482 is already newer than 481 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 482 (ObjNum 362 snap 102 seq_num 362) dirty exists 2026-02-21T22:20:17.425 INFO:tasks.rados.rados.0.trial178.stdout:982: left oid 32 (ObjNum 362 snap 102 seq_num 362) 2026-02-21T22:20:18.412 INFO:tasks.rados.rados.0.trial178.stdout:981: done (2 left) 2026-02-21T22:20:18.412 INFO:tasks.rados.rados.0.trial178.stdout:982: done (1 left) 2026-02-21T22:20:18.412 INFO:tasks.rados.rados.0.trial178.stdout:983: done (0 left) 2026-02-21T22:20:18.413 INFO:tasks.rados.rados.0.trial178.stdout:984: read oid 34 snap -1 2026-02-21T22:20:18.413 INFO:tasks.rados.rados.0.trial178.stdout:984: expect (ObjNum 349 snap 99 seq_num 349) 2026-02-21T22:20:18.413 INFO:tasks.rados.rados.0.trial178.stdout:985: copy_from oid 18 from oid 40 current snap is 103 2026-02-21T22:20:18.413 INFO:tasks.rados.rados.0.trial178.stdout:986: delete oid 43 current snap is 103 2026-02-21T22:20:18.415 INFO:tasks.rados.rados.0.trial178.stdout:986: done (2 left) 2026-02-21T22:20:18.416 INFO:tasks.rados.rados.0.trial178.stdout:987: snap_remove snap 98 2026-02-21T22:20:18.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:18.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:19.412 INFO:tasks.rados.rados.0.trial178.stdout:987: done (2 left) 2026-02-21T22:20:19.412 INFO:tasks.rados.rados.0.trial178.stdout:988: copy_from oid 24 from oid 35 current snap is 103 2026-02-21T22:20:19.412 INFO:tasks.rados.rados.0.trial178.stdout:984: expect (ObjNum 349 snap 99 seq_num 349) 2026-02-21T22:20:19.510 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:19.549 INFO:tasks.rados.rados.0.trial178.stdout:985: finishing copy_from racing read to trial17818531-18 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:985: finishing copy_from to trial17818531-18 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 327 (ObjNum 324 snap 95 seq_num 324) dirty exists 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:988: finishing copy_from racing read to trial17818531-24 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:988: finishing copy_from to trial17818531-24 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:984: done (2 left) 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:985: done (1 left) 2026-02-21T22:20:19.550 INFO:tasks.rados.rados.0.trial178.stdout:988: done (0 left) 2026-02-21T22:20:19.551 INFO:tasks.rados.rados.0.trial178.stdout:989: copy_from oid 31 from oid 19 current snap is 103 2026-02-21T22:20:19.551 INFO:tasks.rados.rados.0.trial178.stdout:990: write oid 4 current snap is 103 2026-02-21T22:20:19.551 INFO:tasks.rados.rados.0.trial178.stdout:990: seq_num 364 ranges {517748=459227,1600622=406110} 2026-02-21T22:20:19.555 INFO:tasks.rados.rados.0.trial178.stdout:990: writing trial17818531-4 from 517748 to 976975 tid 1 2026-02-21T22:20:19.563 INFO:tasks.rados.rados.0.trial178.stdout:990: writing trial17818531-4 from 1600622 to 2006732 tid 2 2026-02-21T22:20:19.563 INFO:tasks.rados.rados.0.trial178.stdout:991: copy_from oid 25 from oid 12 current snap is 103 2026-02-21T22:20:19.563 INFO:tasks.rados.rados.0.trial178.stdout:989: finishing copy_from to trial17818531-31 2026-02-21T22:20:19.563 INFO:tasks.rados.rados.0.trial178.stdout:989: got expected ENOENT (src dne) 2026-02-21T22:20:19.563 INFO:tasks.rados.rados.0.trial178.stdout:989: finishing copy_from racing read to trial17818531-31 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:990: finishing write tid 1 to trial17818531-4 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:989: done (2 left) 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:992: read oid 11 snap 101 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:992: expect (ObjNum 260 snap 77 seq_num 260) 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:993: read oid 15 snap -1 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:993: expect deleted 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:994: read oid 27 snap -1 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:994: expect deleted 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:995: write oid 17 current snap is 103 2026-02-21T22:20:19.564 INFO:tasks.rados.rados.0.trial178.stdout:995: seq_num 365 ranges {534555=650829,1659104=475451} 2026-02-21T22:20:19.572 INFO:tasks.rados.rados.0.trial178.stdout:995: writing trial17818531-17 from 534555 to 1185384 tid 1 2026-02-21T22:20:19.579 INFO:tasks.rados.rados.0.trial178.stdout:995: writing trial17818531-17 from 1659104 to 2134555 tid 2 2026-02-21T22:20:19.579 INFO:tasks.rados.rados.0.trial178.stdout:996: write oid 26 current snap is 103 2026-02-21T22:20:19.579 INFO:tasks.rados.rados.0.trial178.stdout:996: seq_num 366 ranges {551362=726078,1717586=409779} 2026-02-21T22:20:19.587 INFO:tasks.rados.rados.0.trial178.stdout:996: writing trial17818531-26 from 551362 to 1277440 tid 1 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:996: writing trial17818531-26 from 1717586 to 2127365 tid 2 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:997: copy_from oid 34 from oid 6 current snap is 103 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: finishing write tid 2 to trial17818531-4 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: finishing write tid 3 to trial17818531-4 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: finishing write tid 4 to trial17818531-4 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: oid 4 updating version 0 to 327 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: oid 4 updating version 327 to 328 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: oid 4 updating version 328 to 329 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 329 (ObjNum 364 snap 103 seq_num 364) dirty exists 2026-02-21T22:20:19.594 INFO:tasks.rados.rados.0.trial178.stdout:990: left oid 4 (ObjNum 364 snap 103 seq_num 364) 2026-02-21T22:20:19.595 INFO:tasks.rados.rados.0.trial178.stdout:992: expect (ObjNum 260 snap 77 seq_num 260) 2026-02-21T22:20:19.624 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 352958 2026-02-21T22:20:19.682 INFO:tasks.rados.rados.0.trial178.stdout:995: finishing write tid 1 to trial17818531-17 2026-02-21T22:20:19.682 INFO:tasks.rados.rados.0.trial178.stdout:991: finishing copy_from racing read to trial17818531-25 2026-02-21T22:20:19.682 INFO:tasks.rados.rados.0.trial178.stdout:995: finishing write tid 2 to trial17818531-17 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:991: finishing copy_from to trial17818531-25 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 317 (ObjNum 339 snap 95 seq_num 339) dirty exists 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: finishing write tid 3 to trial17818531-17 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: finishing write tid 4 to trial17818531-17 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: oid 17 updating version 0 to 299 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: oid 17 version 299 is already newer than 298 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: oid 17 version 299 is already newer than 297 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 299 (ObjNum 365 snap 103 seq_num 365) dirty exists 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:995: left oid 17 (ObjNum 365 snap 103 seq_num 365) 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:996: finishing write tid 1 to trial17818531-26 2026-02-21T22:20:19.683 INFO:tasks.rados.rados.0.trial178.stdout:997: finishing copy_from to trial17818531-34 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:997: got expected ENOENT (src dne) 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:997: finishing copy_from racing read to trial17818531-34 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: finishing write tid 2 to trial17818531-26 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: finishing write tid 3 to trial17818531-26 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: finishing write tid 4 to trial17818531-26 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: oid 26 updating version 0 to 412 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: oid 26 version 412 is already newer than 410 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: oid 26 version 412 is already newer than 411 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 412 (ObjNum 366 snap 103 seq_num 366) dirty exists 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:996: left oid 26 (ObjNum 366 snap 103 seq_num 366) 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:990: done (7 left) 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:991: done (6 left) 2026-02-21T22:20:19.684 INFO:tasks.rados.rados.0.trial178.stdout:992: done (5 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:993: done (4 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:994: done (3 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:995: done (2 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:996: done (1 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:997: done (0 left) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:998: read oid 43 snap 99 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:998: expect (ObjNum 337 snap 95 seq_num 337) 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:999: write oid 16 current snap is 103 2026-02-21T22:20:19.685 INFO:tasks.rados.rados.0.trial178.stdout:999: seq_num 367 ranges {568169=401327,1376068=669790} 2026-02-21T22:20:19.688 INFO:tasks.rados.rados.0.trial178.stdout:999: writing trial17818531-16 from 568169 to 969496 tid 1 2026-02-21T22:20:19.695 INFO:tasks.rados.rados.0.trial178.stdout:999: writing trial17818531-16 from 1376068 to 2045858 tid 2 2026-02-21T22:20:19.696 INFO:tasks.rados.rados.0.trial178.stdout:1000: read oid 48 snap -1 2026-02-21T22:20:19.696 INFO:tasks.rados.rados.0.trial178.stdout:1000: expect deleted 2026-02-21T22:20:19.696 INFO:tasks.rados.rados.0.trial178.stdout:1001: write oid 32 current snap is 103 2026-02-21T22:20:19.696 INFO:tasks.rados.rados.0.trial178.stdout:1001: seq_num 368 ranges {584976=476576,1834550=350426} 2026-02-21T22:20:19.702 INFO:tasks.rados.rados.0.trial178.stdout:1001: writing trial17818531-32 from 584976 to 1061552 tid 1 2026-02-21T22:20:19.710 INFO:tasks.rados.rados.0.trial178.stdout:1001: writing trial17818531-32 from 1834550 to 2184976 tid 2 2026-02-21T22:20:19.710 INFO:tasks.rados.rados.0.trial178.stdout:1002: snap_create 2026-02-21T22:20:19.710 INFO:tasks.rados.rados.0.trial178.stdout:998: expect (ObjNum 337 snap 95 seq_num 337) 2026-02-21T22:20:19.710 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: finishing write tid 1 to trial17818531-16 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: finishing write tid 2 to trial17818531-16 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: finishing write tid 3 to trial17818531-16 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: finishing write tid 4 to trial17818531-16 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: oid 16 updating version 0 to 330 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: oid 16 updating version 330 to 331 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: oid 16 updating version 331 to 332 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 332 (ObjNum 367 snap 103 seq_num 367) dirty exists 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:999: left oid 16 (ObjNum 367 snap 103 seq_num 367) 2026-02-21T22:20:19.921 INFO:tasks.rados.rados.0.trial178.stdout:1001: finishing write tid 1 to trial17818531-32 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: finishing write tid 2 to trial17818531-32 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: finishing write tid 3 to trial17818531-32 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: finishing write tid 4 to trial17818531-32 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: oid 32 updating version 0 to 490 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: oid 32 version 490 is already newer than 488 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: oid 32 version 490 is already newer than 489 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 490 (ObjNum 368 snap 103 seq_num 368) dirty exists 2026-02-21T22:20:19.922 INFO:tasks.rados.rados.0.trial178.stdout:1001: left oid 32 (ObjNum 368 snap 103 seq_num 368) 2026-02-21T22:20:20.418 INFO:tasks.rados.rados.0.trial178.stdout:998: done (4 left) 2026-02-21T22:20:20.418 INFO:tasks.rados.rados.0.trial178.stdout:999: done (3 left) 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1000: done (2 left) 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1001: done (1 left) 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1002: done (0 left) 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1003: rollback oid 33 current snap is 104 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 103 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1004: write oid 25 current snap is 104 2026-02-21T22:20:20.419 INFO:tasks.rados.rados.0.trial178.stdout:1004: seq_num 369 ranges {601783=551825,1776679=425104} 2026-02-21T22:20:20.426 INFO:tasks.rados.rados.0.trial178.stdout:1004: writing trial17818531-25 from 601783 to 1153608 tid 1 2026-02-21T22:20:20.433 INFO:tasks.rados.rados.0.trial178.stdout:1004: writing trial17818531-25 from 1776679 to 2201783 tid 2 2026-02-21T22:20:20.433 INFO:tasks.rados.rados.0.trial178.stdout:1005: read oid 23 snap -1 2026-02-21T22:20:20.434 INFO:tasks.rados.rados.0.trial178.stdout:1005: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:20.434 INFO:tasks.rados.rados.0.trial178.stdout:1006: snap_remove snap 100 2026-02-21T22:20:21.419 INFO:tasks.rados.rados.0.trial178.stdout:1006: done (3 left) 2026-02-21T22:20:21.419 INFO:tasks.rados.rados.0.trial178.stdout:1007: write (excl) oid 11 current snap is 104 2026-02-21T22:20:21.419 INFO:tasks.rados.rados.0.trial178.stdout:1007: seq_num 370 ranges {618590=627074,1835161=383429} 2026-02-21T22:20:21.426 INFO:tasks.rados.rados.0.trial178.stdout:1007: writing trial17818531-11 from 618590 to 1245664 tid 1 2026-02-21T22:20:21.430 INFO:tasks.rados.rados.0.trial178.stdout:1007: writing trial17818531-11 from 1835161 to 2218590 tid 2 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1008: delete oid 4 current snap is 104 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1003: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 0 (ObjNum 241 snap 0 seq_num 101568752) dirty dne 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: finishing write tid 1 to trial17818531-25 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: finishing write tid 2 to trial17818531-25 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: finishing write tid 3 to trial17818531-25 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: finishing write tid 4 to trial17818531-25 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: oid 25 updating version 0 to 328 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: oid 25 version 328 is already newer than 327 2026-02-21T22:20:21.431 INFO:tasks.rados.rados.0.trial178.stdout:1004: oid 25 version 328 is already newer than 326 2026-02-21T22:20:21.432 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 328 (ObjNum 369 snap 104 seq_num 369) dirty exists 2026-02-21T22:20:21.432 INFO:tasks.rados.rados.0.trial178.stdout:1004: left oid 25 (ObjNum 369 snap 104 seq_num 369) 2026-02-21T22:20:21.432 INFO:tasks.rados.rados.0.trial178.stdout:1005: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: finishing write tid 1 to trial17818531-11 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: finishing write tid 2 to trial17818531-11 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: finishing write tid 3 to trial17818531-11 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: finishing write tid 4 to trial17818531-11 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: oid 11 updating version 0 to 497 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: oid 11 updating version 497 to 498 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: oid 11 version 498 is already newer than 496 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 498 (ObjNum 370 snap 104 seq_num 370) dirty exists 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: left oid 11 (ObjNum 370 snap 104 seq_num 370) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1003: done (4 left) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1004: done (3 left) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1005: done (2 left) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1007: done (1 left) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1008: done (0 left) 2026-02-21T22:20:21.572 INFO:tasks.rados.rados.0.trial178.stdout:1009: snap_create 2026-02-21T22:20:22.422 INFO:tasks.rados.rados.0.trial178.stdout:1009: done (0 left) 2026-02-21T22:20:22.423 INFO:tasks.rados.rados.0.trial178.stdout:1010: snap_remove snap 104 2026-02-21T22:20:23.423 INFO:tasks.rados.rados.0.trial178.stdout:1010: done (0 left) 2026-02-21T22:20:23.423 INFO:tasks.rados.rados.0.trial178.stdout:1011: write oid 15 current snap is 105 2026-02-21T22:20:23.424 INFO:tasks.rados.rados.0.trial178.stdout:1011: seq_num 372 ranges {652204=777572,1952125=300079} 2026-02-21T22:20:23.433 INFO:tasks.rados.rados.0.trial178.stdout:1011: writing trial17818531-15 from 652204 to 1429776 tid 1 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1011: writing trial17818531-15 from 1952125 to 2252204 tid 2 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1012: read oid 50 snap -1 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1012: expect (ObjNum 312 snap 91 seq_num 312) 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1013: read oid 25 snap -1 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1013: expect (ObjNum 369 snap 104 seq_num 369) 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1014: read oid 4 snap -1 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1014: expect deleted 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1015: write (excl) oid 33 current snap is 105 2026-02-21T22:20:23.439 INFO:tasks.rados.rados.0.trial178.stdout:1015: seq_num 373 ranges {669011=569174,2010607=258404} 2026-02-21T22:20:23.450 INFO:tasks.rados.rados.0.trial178.stdout:1015: writing trial17818531-33 from 669011 to 1238185 tid 1 2026-02-21T22:20:23.457 INFO:tasks.rados.rados.0.trial178.stdout:1015: writing trial17818531-33 from 2010607 to 2269011 tid 2 2026-02-21T22:20:23.457 INFO:tasks.rados.rados.0.trial178.stdout:1016: rollback oid 38 current snap is 105 2026-02-21T22:20:23.457 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 101 2026-02-21T22:20:23.457 INFO:tasks.rados.rados.0.trial178.stdout:1011: finishing write tid 1 to trial17818531-15 2026-02-21T22:20:23.457 INFO:tasks.rados.rados.0.trial178.stdout:1013: expect (ObjNum 369 snap 104 seq_num 369) 2026-02-21T22:20:23.525 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:20:23] 2026-02-21T22:20:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:20:23, mode upmap, max misplaced 0.050000 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', '.mgr', 'unique_pool_0'] 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:20:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:20:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:20:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:20:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:20:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:20:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:20:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:20:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:20:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:20:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:20:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:20:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:20:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:20:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:20:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:20:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:20:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:20:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:20:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:20:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:20:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:20:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "229", 2026-02-21T22:20:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:20:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:20:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 229, 2026-02-21T22:20:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 109, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:20:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0003878840693721065 of space, bias 1.0, pg target 0.11636522081163195 quantized to 32 (current 16) 2026-02-21T22:20:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:20:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:20:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:20:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:20:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:20:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:20:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:20:23.566 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1859116 2026-02-21T22:20:23.566 INFO:tasks.rados.rados.0.trial178.stdout:1011: finishing write tid 2 to trial17818531-15 2026-02-21T22:20:23.566 INFO:tasks.rados.rados.0.trial178.stdout:1011: finishing write tid 3 to trial17818531-15 2026-02-21T22:20:23.566 INFO:tasks.rados.rados.0.trial178.stdout:1011: finishing write tid 4 to trial17818531-15 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:1011: oid 15 updating version 0 to 305 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:1011: oid 15 version 305 is already newer than 304 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:1011: oid 15 updating version 305 to 306 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 306 (ObjNum 372 snap 105 seq_num 372) dirty exists 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:1011: left oid 15 (ObjNum 372 snap 105 seq_num 372) 2026-02-21T22:20:23.567 INFO:tasks.rados.rados.0.trial178.stdout:1012: expect (ObjNum 312 snap 91 seq_num 312) 2026-02-21T22:20:23.636 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1015: finishing write tid 1 to trial17818531-33 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1015: finishing write tid 2 to trial17818531-33 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1011: done (5 left) 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1012: done (4 left) 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1013: done (3 left) 2026-02-21T22:20:23.673 INFO:tasks.rados.rados.0.trial178.stdout:1014: done (2 left) 2026-02-21T22:20:23.674 INFO:tasks.rados.rados.0.trial178.stdout:1017: read oid 32 snap -1 2026-02-21T22:20:23.674 INFO:tasks.rados.rados.0.trial178.stdout:1017: expect (ObjNum 368 snap 103 seq_num 368) 2026-02-21T22:20:23.674 INFO:tasks.rados.rados.0.trial178.stdout:1018: write oid 44 current snap is 105 2026-02-21T22:20:23.674 INFO:tasks.rados.rados.0.trial178.stdout:1018: seq_num 374 ranges {685818=644423,2069089=216729} 2026-02-21T22:20:23.683 INFO:tasks.rados.rados.0.trial178.stdout:1018: writing trial17818531-44 from 685818 to 1330241 tid 1 2026-02-21T22:20:23.690 INFO:tasks.rados.rados.0.trial178.stdout:1018: writing trial17818531-44 from 2069089 to 2285818 tid 2 2026-02-21T22:20:23.690 INFO:tasks.rados.rados.0.trial178.stdout:1019: read oid 3 snap -1 2026-02-21T22:20:23.690 INFO:tasks.rados.rados.0.trial178.stdout:1019: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:20:23.690 INFO:tasks.rados.rados.0.trial178.stdout:1020: rollback oid 45 current snap is 105 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 103 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: finishing write tid 3 to trial17818531-33 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: finishing write tid 4 to trial17818531-33 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: oid 33 updating version 0 to 137 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: oid 33 updating version 137 to 138 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: oid 33 updating version 138 to 139 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 139 (ObjNum 373 snap 105 seq_num 373) dirty exists 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1015: left oid 33 (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1016: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1016: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:1016: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:20:23.691 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 143 (ObjNum 327 snap 95 seq_num 327) dirty exists 2026-02-21T22:20:23.692 INFO:tasks.rados.rados.0.trial178.stdout:1017: expect (ObjNum 368 snap 103 seq_num 368) 2026-02-21T22:20:23.734 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 399745 2026-02-21T22:20:23.820 INFO:tasks.rados.rados.0.trial178.stdout:1018: finishing write tid 1 to trial17818531-44 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1020: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 0 (ObjNum 241 snap 0 seq_num 101568752) dirty dne 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: finishing write tid 2 to trial17818531-44 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: finishing write tid 3 to trial17818531-44 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: finishing write tid 4 to trial17818531-44 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: oid 44 updating version 0 to 335 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: oid 44 updating version 335 to 336 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: oid 44 version 336 is already newer than 334 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 336 (ObjNum 374 snap 105 seq_num 374) dirty exists 2026-02-21T22:20:23.821 INFO:tasks.rados.rados.0.trial178.stdout:1018: left oid 44 (ObjNum 374 snap 105 seq_num 374) 2026-02-21T22:20:23.822 INFO:tasks.rados.rados.0.trial178.stdout:1019: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:20:23.825 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 779072 2026-02-21T22:20:23.913 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1780612 2026-02-21T22:20:23.934 INFO:tasks.rados.rados.0.trial178.stdout:1015: done (5 left) 2026-02-21T22:20:23.934 INFO:tasks.rados.rados.0.trial178.stdout:1016: done (4 left) 2026-02-21T22:20:23.934 INFO:tasks.rados.rados.0.trial178.stdout:1017: done (3 left) 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1018: done (2 left) 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1019: done (1 left) 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1020: done (0 left) 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1021: read oid 24 snap -1 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1021: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1022: rollback oid 31 current snap is 105 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 97 2026-02-21T22:20:23.935 INFO:tasks.rados.rados.0.trial178.stdout:1023: delete oid 8 current snap is 105 2026-02-21T22:20:23.936 INFO:tasks.rados.rados.0.trial178.stdout:1022: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:20:23.936 INFO:tasks.rados.rados.0.trial178.stdout:1022: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:20:23.937 INFO:tasks.rados.rados.0.trial178.stdout:1022: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:20:23.937 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 331 (ObjNum 239 snap 75 seq_num 239) dirty exists 2026-02-21T22:20:23.943 INFO:tasks.rados.rados.0.trial178.stdout:1021: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:23.948 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 869614 2026-02-21T22:20:24.075 INFO:tasks.rados.rados.0.trial178.stdout:1021: done (2 left) 2026-02-21T22:20:24.076 INFO:tasks.rados.rados.0.trial178.stdout:1022: done (1 left) 2026-02-21T22:20:24.076 INFO:tasks.rados.rados.0.trial178.stdout:1023: done (0 left) 2026-02-21T22:20:24.076 INFO:tasks.rados.rados.0.trial178.stdout:1024: write oid 46 current snap is 105 2026-02-21T22:20:24.076 INFO:tasks.rados.rados.0.trial178.stdout:1024: seq_num 376 ranges {719432=794921,2186053=133379} 2026-02-21T22:20:24.086 INFO:tasks.rados.rados.0.trial178.stdout:1024: writing trial17818531-46 from 719432 to 1514353 tid 1 2026-02-21T22:20:24.091 INFO:tasks.rados.rados.0.trial178.stdout:1024: writing trial17818531-46 from 2186053 to 2319432 tid 2 2026-02-21T22:20:24.091 INFO:tasks.rados.rados.0.trial178.stdout:1025: read oid 23 snap -1 2026-02-21T22:20:24.091 INFO:tasks.rados.rados.0.trial178.stdout:1025: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:24.091 INFO:tasks.rados.rados.0.trial178.stdout:1026: copy_from oid 10 from oid 2 current snap is 105 2026-02-21T22:20:24.091 INFO:tasks.rados.rados.0.trial178.stdout:1024: finishing write tid 1 to trial17818531-46 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1027: read oid 24 snap -1 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1027: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1028: read oid 25 snap -1 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1028: expect (ObjNum 369 snap 104 seq_num 369) 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1029: copy_from oid 42 from oid 21 current snap is 105 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1030: read oid 12 snap -1 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1030: expect (ObjNum 339 snap 95 seq_num 339) 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1031: copy_from oid 47 from oid 17 current snap is 105 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1032: write (excl) oid 38 current snap is 105 2026-02-21T22:20:24.092 INFO:tasks.rados.rados.0.trial178.stdout:1032: seq_num 377 ranges {736239=470170,1728182=571782} 2026-02-21T22:20:24.099 INFO:tasks.rados.rados.0.trial178.stdout:1032: writing trial17818531-38 from 736239 to 1206409 tid 1 2026-02-21T22:20:24.107 INFO:tasks.rados.rados.0.trial178.stdout:1032: writing trial17818531-38 from 1728182 to 2299964 tid 2 2026-02-21T22:20:24.107 INFO:tasks.rados.rados.0.trial178.stdout:1033: rollback oid 36 current snap is 105 2026-02-21T22:20:24.107 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 101 2026-02-21T22:20:24.107 INFO:tasks.rados.rados.0.trial178.stdout:1028: expect (ObjNum 369 snap 104 seq_num 369) 2026-02-21T22:20:24.139 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:24.172 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:24.205 INFO:tasks.rados.rados.0.trial178.stdout:1026: finishing copy_from to trial17818531-10 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1026: got expected ENOENT (src dne) 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1026: finishing copy_from racing read to trial17818531-10 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1024: finishing write tid 2 to trial17818531-46 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1024: finishing write tid 3 to trial17818531-46 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1026: done (9 left) 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1028: done (8 left) 2026-02-21T22:20:24.206 INFO:tasks.rados.rados.0.trial178.stdout:1034: snap_remove snap 102 2026-02-21T22:20:24.429 INFO:tasks.rados.rados.0.trial178.stdout:1034: done (8 left) 2026-02-21T22:20:24.429 INFO:tasks.rados.rados.0.trial178.stdout:1035: snap_remove snap 99 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: finishing write tid 4 to trial17818531-46 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: oid 46 updating version 0 to 178 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: oid 46 version 178 is already newer than 177 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: oid 46 updating version 178 to 179 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 179 (ObjNum 376 snap 105 seq_num 376) dirty exists 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: left oid 46 (ObjNum 376 snap 105 seq_num 376) 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:1025: expect (ObjNum 310 snap 91 seq_num 310) 2026-02-21T22:20:24.430 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:24.528 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2153270 2026-02-21T22:20:24.540 INFO:tasks.rados.rados.0.trial178.stdout:1030: expect (ObjNum 339 snap 95 seq_num 339) 2026-02-21T22:20:24.583 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3493534 2026-02-21T22:20:24.631 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:24.686 INFO:tasks.rados.rados.0.trial178.stdout:1027: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:24.690 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 676885 2026-02-21T22:20:24.827 INFO:tasks.rados.rados.0.trial178.stdout:1032: finishing write tid 1 to trial17818531-38 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: finishing write tid 2 to trial17818531-38 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: finishing write tid 3 to trial17818531-38 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: finishing write tid 4 to trial17818531-38 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: oid 38 updating version 0 to 146 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: oid 38 version 146 is already newer than 144 2026-02-21T22:20:24.828 INFO:tasks.rados.rados.0.trial178.stdout:1032: oid 38 version 146 is already newer than 145 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 146 (ObjNum 377 snap 105 seq_num 377) dirty exists 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:1032: left oid 38 (ObjNum 377 snap 105 seq_num 377) 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:1029: finishing copy_from racing read to trial17818531-42 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:1031: finishing copy_from to trial17818531-47 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 299 (ObjNum 365 snap 103 seq_num 365) dirty exists 2026-02-21T22:20:24.829 INFO:tasks.rados.rados.0.trial178.stdout:1031: finishing copy_from racing read to trial17818531-47 2026-02-21T22:20:24.830 INFO:tasks.rados.rados.0.trial178.stdout:1029: finishing copy_from to trial17818531-42 2026-02-21T22:20:24.830 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 403 (ObjNum 264 snap 81 seq_num 264) dirty exists 2026-02-21T22:20:24.830 INFO:tasks.rados.rados.0.trial178.stdout:1033: finishing rollback tid 0 to trial17818531-36 2026-02-21T22:20:24.830 INFO:tasks.rados.rados.0.trial178.stdout:1033: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:20:24.830 INFO:tasks.rados.rados.0.trial178.stdout:1033: finishing rollback tid 2 to trial17818531-36 2026-02-21T22:20:24.831 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 169 (ObjNum 352 snap 100 seq_num 352) dirty exists 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1024: done (8 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1025: done (7 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1027: done (6 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1029: done (5 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1030: done (4 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1031: done (3 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1032: done (2 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1033: done (1 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1035: done (0 left) 2026-02-21T22:20:25.430 INFO:tasks.rados.rados.0.trial178.stdout:1036: snap_remove snap 101 2026-02-21T22:20:26.431 INFO:tasks.rados.rados.0.trial178.stdout:1036: done (0 left) 2026-02-21T22:20:26.432 INFO:tasks.rados.rados.0.trial178.stdout:1037: write oid 44 current snap is 105 2026-02-21T22:20:26.432 INFO:tasks.rados.rados.0.trial178.stdout:1037: seq_num 378 ranges {753046=545419,1786664=431793} 2026-02-21T22:20:26.440 INFO:tasks.rados.rados.0.trial178.stdout:1037: writing trial17818531-44 from 753046 to 1298465 tid 1 2026-02-21T22:20:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1037: writing trial17818531-44 from 1786664 to 2218457 tid 2 2026-02-21T22:20:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1038: copy_from oid 8 from oid 26 current snap is 105 2026-02-21T22:20:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1037: finishing write tid 1 to trial17818531-44 2026-02-21T22:20:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1039: delete oid 36 current snap is 105 2026-02-21T22:20:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1039: done (2 left) 2026-02-21T22:20:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1040: snap_remove snap 103 2026-02-21T22:20:27.409 INFO:tasks.rados.rados.0.trial178.stdout:1040: done (2 left) 2026-02-21T22:20:27.409 INFO:tasks.rados.rados.0.trial178.stdout:1041: write oid 24 current snap is 105 2026-02-21T22:20:27.410 INFO:tasks.rados.rados.0.trial178.stdout:1041: seq_num 380 ranges {786660=695917,1903628=435462} 2026-02-21T22:20:27.420 INFO:tasks.rados.rados.0.trial178.stdout:1041: writing trial17818531-24 from 786660 to 1482577 tid 1 2026-02-21T22:20:27.426 INFO:tasks.rados.rados.0.trial178.stdout:1041: writing trial17818531-24 from 1903628 to 2339090 tid 2 2026-02-21T22:20:27.426 INFO:tasks.rados.rados.0.trial178.stdout:1042: write oid 49 current snap is 105 2026-02-21T22:20:27.426 INFO:tasks.rados.rados.0.trial178.stdout:1042: seq_num 381 ranges {403467=487519,1562110=579120} 2026-02-21T22:20:27.431 INFO:tasks.rados.rados.0.trial178.stdout:1042: writing trial17818531-49 from 403467 to 890986 tid 1 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1042: writing trial17818531-49 from 1562110 to 2141230 tid 2 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1043: rollback oid 47 current snap is 105 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 97 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1037: finishing write tid 2 to trial17818531-44 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1037: finishing write tid 3 to trial17818531-44 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1037: finishing write tid 4 to trial17818531-44 2026-02-21T22:20:27.441 INFO:tasks.rados.rados.0.trial178.stdout:1037: oid 44 updating version 0 to 338 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1037: oid 44 updating version 338 to 339 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1037: oid 44 version 339 is already newer than 337 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 339 (ObjNum 378 snap 105 seq_num 378) dirty exists 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1037: left oid 44 (ObjNum 378 snap 105 seq_num 378) 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1038: finishing copy_from racing read to trial17818531-8 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1038: finishing copy_from to trial17818531-8 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 412 (ObjNum 366 snap 103 seq_num 366) dirty exists 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1041: finishing write tid 1 to trial17818531-24 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1041: finishing write tid 2 to trial17818531-24 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1037: done (4 left) 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1038: done (3 left) 2026-02-21T22:20:27.442 INFO:tasks.rados.rados.0.trial178.stdout:1044: read oid 13 snap 97 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1044: expect (ObjNum 313 snap 92 seq_num 313) 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1045: snap_create 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: finishing write tid 3 to trial17818531-24 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: finishing write tid 4 to trial17818531-24 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: oid 24 updating version 0 to 521 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: oid 24 updating version 521 to 522 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: oid 24 version 522 is already newer than 516 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 522 (ObjNum 380 snap 105 seq_num 380) dirty exists 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1041: left oid 24 (ObjNum 380 snap 105 seq_num 380) 2026-02-21T22:20:27.443 INFO:tasks.rados.rados.0.trial178.stdout:1042: finishing write tid 1 to trial17818531-49 2026-02-21T22:20:27.444 INFO:tasks.rados.rados.0.trial178.stdout:1043: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:20:27.444 INFO:tasks.rados.rados.0.trial178.stdout:1043: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:20:27.444 INFO:tasks.rados.rados.0.trial178.stdout:1043: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:20:27.444 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 527 (ObjNum 232 snap 72 seq_num 232) dirty exists 2026-02-21T22:20:27.445 INFO:tasks.rados.rados.0.trial178.stdout:1042: finishing write tid 2 to trial17818531-49 2026-02-21T22:20:27.445 INFO:tasks.rados.rados.0.trial178.stdout:1042: finishing write tid 3 to trial17818531-49 2026-02-21T22:20:27.445 INFO:tasks.rados.rados.0.trial178.stdout:1042: finishing write tid 4 to trial17818531-49 2026-02-21T22:20:27.445 INFO:tasks.rados.rados.0.trial178.stdout:1042: oid 49 updating version 0 to 334 2026-02-21T22:20:27.445 INFO:tasks.rados.rados.0.trial178.stdout:1042: oid 49 version 334 is already newer than 332 2026-02-21T22:20:27.446 INFO:tasks.rados.rados.0.trial178.stdout:1042: oid 49 version 334 is already newer than 333 2026-02-21T22:20:27.446 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 334 (ObjNum 381 snap 105 seq_num 381) dirty exists 2026-02-21T22:20:27.446 INFO:tasks.rados.rados.0.trial178.stdout:1042: left oid 49 (ObjNum 381 snap 105 seq_num 381) 2026-02-21T22:20:27.447 INFO:tasks.rados.rados.0.trial178.stdout:1044: expect (ObjNum 313 snap 92 seq_num 313) 2026-02-21T22:20:27.521 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 542379 2026-02-21T22:20:28.440 INFO:tasks.rados.rados.0.trial178.stdout:1041: done (4 left) 2026-02-21T22:20:28.440 INFO:tasks.rados.rados.0.trial178.stdout:1042: done (3 left) 2026-02-21T22:20:28.440 INFO:tasks.rados.rados.0.trial178.stdout:1043: done (2 left) 2026-02-21T22:20:28.440 INFO:tasks.rados.rados.0.trial178.stdout:1044: done (1 left) 2026-02-21T22:20:28.440 INFO:tasks.rados.rados.0.trial178.stdout:1045: done (0 left) 2026-02-21T22:20:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1046: read oid 46 snap -1 2026-02-21T22:20:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1046: expect (ObjNum 376 snap 105 seq_num 376) 2026-02-21T22:20:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1047: copy_from oid 41 from oid 13 current snap is 106 2026-02-21T22:20:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1048: write oid 8 current snap is 106 2026-02-21T22:20:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1048: seq_num 382 ranges {420274=562768,1620592=439131} 2026-02-21T22:20:28.447 INFO:tasks.rados.rados.0.trial178.stdout:1048: writing trial17818531-8 from 420274 to 983042 tid 1 2026-02-21T22:20:28.454 INFO:tasks.rados.rados.0.trial178.stdout:1048: writing trial17818531-8 from 1620592 to 2059723 tid 2 2026-02-21T22:20:28.454 INFO:tasks.rados.rados.0.trial178.stdout:1049: read oid 31 snap -1 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:1049: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:1050: read oid 37 snap 105 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:1050: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:1051: rollback oid 23 current snap is 106 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 97 2026-02-21T22:20:28.455 INFO:tasks.rados.rados.0.trial178.stdout:1046: expect (ObjNum 376 snap 105 seq_num 376) 2026-02-21T22:20:28.478 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:28.502 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 214697 2026-02-21T22:20:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1048: finishing write tid 1 to trial17818531-8 2026-02-21T22:20:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1047: finishing copy_from racing read to trial17818531-41 2026-02-21T22:20:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1046: done (5 left) 2026-02-21T22:20:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1052: delete oid 43 current snap is 106 2026-02-21T22:20:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1049: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:20:28.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:28.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:28.591 INFO:tasks.rados.rados.0.trial178.stdout:1048: finishing write tid 2 to trial17818531-8 2026-02-21T22:20:28.591 INFO:tasks.rados.rados.0.trial178.stdout:1048: finishing write tid 3 to trial17818531-8 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1048: finishing write tid 4 to trial17818531-8 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1048: oid 8 updating version 0 to 530 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1048: oid 8 version 530 is already newer than 529 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1048: oid 8 updating version 530 to 531 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 531 (ObjNum 382 snap 106 seq_num 382) dirty exists 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1048: left oid 8 (ObjNum 382 snap 106 seq_num 382) 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:1050: expect (ObjNum 332 snap 95 seq_num 332) 2026-02-21T22:20:28.592 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1047: finishing copy_from to trial17818531-41 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 478 (ObjNum 356 snap 101 seq_num 356) dirty exists 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1047: done (5 left) 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1048: done (4 left) 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1049: done (3 left) 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1050: done (2 left) 2026-02-21T22:20:28.727 INFO:tasks.rados.rados.0.trial178.stdout:1052: done (1 left) 2026-02-21T22:20:28.728 INFO:tasks.rados.rados.0.trial178.stdout:1053: read oid 49 snap -1 2026-02-21T22:20:28.728 INFO:tasks.rados.rados.0.trial178.stdout:1053: expect (ObjNum 381 snap 105 seq_num 381) 2026-02-21T22:20:28.728 INFO:tasks.rados.rados.0.trial178.stdout:1054: write (excl) oid 42 current snap is 106 2026-02-21T22:20:28.728 INFO:tasks.rados.rados.0.trial178.stdout:1054: seq_num 384 ranges {453888=713266,1737556=442800} 2026-02-21T22:20:28.734 INFO:tasks.rados.rados.0.trial178.stdout:1054: writing trial17818531-42 from 453888 to 1167154 tid 1 2026-02-21T22:20:28.741 INFO:tasks.rados.rados.0.trial178.stdout:1054: writing trial17818531-42 from 1737556 to 2180356 tid 2 2026-02-21T22:20:28.741 INFO:tasks.rados.rados.0.trial178.stdout:1055: read oid 31 snap -1 2026-02-21T22:20:28.741 INFO:tasks.rados.rados.0.trial178.stdout:1055: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:1056: snap_create 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:1051: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:1051: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:1051: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 182 (ObjNum 310 snap 91 seq_num 310) dirty exists 2026-02-21T22:20:28.742 INFO:tasks.rados.rados.0.trial178.stdout:1053: expect (ObjNum 381 snap 105 seq_num 381) 2026-02-21T22:20:28.767 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1999065 2026-02-21T22:20:28.781 INFO:tasks.rados.rados.0.trial178.stdout:1054: finishing write tid 1 to trial17818531-42 2026-02-21T22:20:28.781 INFO:tasks.rados.rados.0.trial178.stdout:1055: expect (ObjNum 239 snap 75 seq_num 239) 2026-02-21T22:20:28.807 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1174195 2026-02-21T22:20:28.846 INFO:tasks.rados.rados.0.trial178.stdout:1054: finishing write tid 2 to trial17818531-42 2026-02-21T22:20:28.846 INFO:tasks.rados.rados.0.trial178.stdout:1054: finishing write tid 3 to trial17818531-42 2026-02-21T22:20:28.846 INFO:tasks.rados.rados.0.trial178.stdout:1054: finishing write tid 4 to trial17818531-42 2026-02-21T22:20:28.847 INFO:tasks.rados.rados.0.trial178.stdout:1054: oid 42 updating version 0 to 535 2026-02-21T22:20:28.847 INFO:tasks.rados.rados.0.trial178.stdout:1054: oid 42 version 535 is already newer than 533 2026-02-21T22:20:28.847 INFO:tasks.rados.rados.0.trial178.stdout:1054: oid 42 version 535 is already newer than 534 2026-02-21T22:20:28.847 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 535 (ObjNum 384 snap 106 seq_num 384) dirty exists 2026-02-21T22:20:28.847 INFO:tasks.rados.rados.0.trial178.stdout:1054: left oid 42 (ObjNum 384 snap 106 seq_num 384) 2026-02-21T22:20:29.442 INFO:tasks.rados.rados.0.trial178.stdout:1051: done (4 left) 2026-02-21T22:20:29.442 INFO:tasks.rados.rados.0.trial178.stdout:1053: done (3 left) 2026-02-21T22:20:29.443 INFO:tasks.rados.rados.0.trial178.stdout:1054: done (2 left) 2026-02-21T22:20:29.443 INFO:tasks.rados.rados.0.trial178.stdout:1055: done (1 left) 2026-02-21T22:20:29.443 INFO:tasks.rados.rados.0.trial178.stdout:1056: done (0 left) 2026-02-21T22:20:29.443 INFO:tasks.rados.rados.0.trial178.stdout:1057: snap_create 2026-02-21T22:20:30.441 INFO:tasks.rados.rados.0.trial178.stdout:1057: done (0 left) 2026-02-21T22:20:30.441 INFO:tasks.rados.rados.0.trial178.stdout:1058: read oid 5 snap 97 2026-02-21T22:20:30.441 INFO:tasks.rados.rados.0.trial178.stdout:1058: expect (ObjNum 270 snap 83 seq_num 270) 2026-02-21T22:20:30.441 INFO:tasks.rados.rados.0.trial178.stdout:1059: write oid 9 current snap is 108 2026-02-21T22:20:30.441 INFO:tasks.rados.rados.0.trial178.stdout:1059: seq_num 385 ranges {470695=788515,1679685=586458} 2026-02-21T22:20:30.450 INFO:tasks.rados.rados.0.trial178.stdout:1059: writing trial17818531-9 from 470695 to 1259210 tid 1 2026-02-21T22:20:30.457 INFO:tasks.rados.rados.0.trial178.stdout:1059: writing trial17818531-9 from 1679685 to 2266143 tid 2 2026-02-21T22:20:30.457 INFO:tasks.rados.rados.0.trial178.stdout:1060: write (excl) oid 45 current snap is 108 2026-02-21T22:20:30.457 INFO:tasks.rados.rados.0.trial178.stdout:1060: seq_num 386 ranges {487502=463764,1738167=446469} 2026-02-21T22:20:30.463 INFO:tasks.rados.rados.0.trial178.stdout:1060: writing trial17818531-45 from 487502 to 951266 tid 1 2026-02-21T22:20:30.471 INFO:tasks.rados.rados.0.trial178.stdout:1060: writing trial17818531-45 from 1738167 to 2184636 tid 2 2026-02-21T22:20:30.471 INFO:tasks.rados.rados.0.trial178.stdout:1061: rollback oid 1 current snap is 108 2026-02-21T22:20:30.471 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 107 2026-02-21T22:20:30.472 INFO:tasks.rados.rados.0.trial178.stdout:1058: expect (ObjNum 270 snap 83 seq_num 270) 2026-02-21T22:20:30.550 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:30.584 INFO:tasks.rados.rados.0.trial178.stdout:1059: finishing write tid 1 to trial17818531-9 2026-02-21T22:20:30.584 INFO:tasks.rados.rados.0.trial178.stdout:1060: finishing write tid 1 to trial17818531-45 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: finishing write tid 2 to trial17818531-9 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: finishing write tid 3 to trial17818531-9 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1058: done (3 left) 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1062: copy_from oid 41 from oid 28 current snap is 108 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: finishing write tid 4 to trial17818531-9 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: oid 9 updating version 0 to 369 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: oid 9 version 369 is already newer than 368 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: oid 9 updating version 369 to 370 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 370 (ObjNum 385 snap 108 seq_num 385) dirty exists 2026-02-21T22:20:30.585 INFO:tasks.rados.rados.0.trial178.stdout:1059: left oid 9 (ObjNum 385 snap 108 seq_num 385) 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: finishing write tid 2 to trial17818531-45 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: finishing write tid 3 to trial17818531-45 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1061: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1061: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: finishing write tid 4 to trial17818531-45 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: oid 45 updating version 0 to 391 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: oid 45 version 391 is already newer than 390 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: oid 45 updating version 391 to 392 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 392 (ObjNum 386 snap 108 seq_num 386) dirty exists 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1060: left oid 45 (ObjNum 386 snap 108 seq_num 386) 2026-02-21T22:20:30.586 INFO:tasks.rados.rados.0.trial178.stdout:1059: done (3 left) 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:1060: done (2 left) 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:1063: rollback oid 38 current snap is 108 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 107 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:1061: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 395 (ObjNum 331 snap 95 seq_num 331) dirty exists 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:1061: done (2 left) 2026-02-21T22:20:30.587 INFO:tasks.rados.rados.0.trial178.stdout:1064: snap_create 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:1063: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:1062: finishing copy_from to trial17818531-41 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:1062: got expected ENOENT (src dne) 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:1063: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:1063: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:20:30.588 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 158 (ObjNum 377 snap 105 seq_num 377) dirty exists 2026-02-21T22:20:30.589 INFO:tasks.rados.rados.0.trial178.stdout:1062: finishing copy_from racing read to trial17818531-41 2026-02-21T22:20:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1062: done (2 left) 2026-02-21T22:20:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1063: done (1 left) 2026-02-21T22:20:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1064: done (0 left) 2026-02-21T22:20:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1065: delete oid 37 current snap is 109 2026-02-21T22:20:31.452 INFO:tasks.rados.rados.0.trial178.stdout:1065: done (0 left) 2026-02-21T22:20:31.452 INFO:tasks.rados.rados.0.trial178.stdout:1066: delete oid 48 current snap is 109 2026-02-21T22:20:31.453 INFO:tasks.rados.rados.0.trial178.stdout:1066: done (0 left) 2026-02-21T22:20:31.453 INFO:tasks.rados.rados.0.trial178.stdout:1067: delete oid 20 current snap is 109 2026-02-21T22:20:31.456 INFO:tasks.rados.rados.0.trial178.stdout:1067: done (0 left) 2026-02-21T22:20:31.456 INFO:tasks.rados.rados.0.trial178.stdout:1068: write (excl) oid 4 current snap is 109 2026-02-21T22:20:31.456 INFO:tasks.rados.rados.0.trial178.stdout:1068: seq_num 390 ranges {554730=481113,1572095=737454} 2026-02-21T22:20:31.463 INFO:tasks.rados.rados.0.trial178.stdout:1068: writing trial17818531-4 from 554730 to 1035843 tid 1 2026-02-21T22:20:31.471 INFO:tasks.rados.rados.0.trial178.stdout:1068: writing trial17818531-4 from 1572095 to 2309549 tid 2 2026-02-21T22:20:31.471 INFO:tasks.rados.rados.0.trial178.stdout:1069: delete oid 25 current snap is 109 2026-02-21T22:20:31.471 INFO:tasks.rados.rados.0.trial178.stdout:1068: finishing write tid 1 to trial17818531-4 2026-02-21T22:20:31.473 INFO:tasks.rados.rados.0.trial178.stdout:1069: done (1 left) 2026-02-21T22:20:31.473 INFO:tasks.rados.rados.0.trial178.stdout:1070: snap_create 2026-02-21T22:20:31.474 INFO:tasks.rados.rados.0.trial178.stdout:1068: finishing write tid 2 to trial17818531-4 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: finishing write tid 3 to trial17818531-4 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: finishing write tid 4 to trial17818531-4 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: oid 4 updating version 0 to 341 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: oid 4 version 341 is already newer than 340 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: oid 4 updating version 341 to 342 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 342 (ObjNum 390 snap 109 seq_num 390) dirty exists 2026-02-21T22:20:31.485 INFO:tasks.rados.rados.0.trial178.stdout:1068: left oid 4 (ObjNum 390 snap 109 seq_num 390) 2026-02-21T22:20:32.410 INFO:tasks.rados.rados.0.trial178.stdout:1068: done (1 left) 2026-02-21T22:20:32.410 INFO:tasks.rados.rados.0.trial178.stdout:1070: done (0 left) 2026-02-21T22:20:32.410 INFO:tasks.rados.rados.0.trial178.stdout:1071: snap_remove snap 108 2026-02-21T22:20:33.410 INFO:tasks.rados.rados.0.trial178.stdout:1071: done (0 left) 2026-02-21T22:20:33.411 INFO:tasks.rados.rados.0.trial178.stdout:1072: snap_create 2026-02-21T22:20:33.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:33.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:34.413 INFO:tasks.rados.rados.0.trial178.stdout:1072: done (0 left) 2026-02-21T22:20:34.413 INFO:tasks.rados.rados.0.trial178.stdout:1073: write oid 12 current snap is 111 2026-02-21T22:20:34.413 INFO:tasks.rados.rados.0.trial178.stdout:1073: seq_num 392 ranges {588344=631611,1689059=741123} 2026-02-21T22:20:34.421 INFO:tasks.rados.rados.0.trial178.stdout:1073: writing trial17818531-12 from 588344 to 1219955 tid 1 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1073: writing trial17818531-12 from 1689059 to 2430182 tid 2 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1074: rollback oid 22 current snap is 111 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 97 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1073: finishing write tid 1 to trial17818531-12 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1075: read oid 42 snap -1 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1075: expect (ObjNum 384 snap 106 seq_num 384) 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1076: write oid 46 current snap is 111 2026-02-21T22:20:34.430 INFO:tasks.rados.rados.0.trial178.stdout:1076: seq_num 393 ranges {605151=706860,2031188=573963} 2026-02-21T22:20:34.439 INFO:tasks.rados.rados.0.trial178.stdout:1076: writing trial17818531-46 from 605151 to 1312011 tid 1 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1076: writing trial17818531-46 from 2031188 to 2605151 tid 2 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1077: copy_from oid 1 from oid 34 current snap is 111 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1073: finishing write tid 2 to trial17818531-12 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1073: finishing write tid 3 to trial17818531-12 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1073: finishing write tid 4 to trial17818531-12 2026-02-21T22:20:34.448 INFO:tasks.rados.rados.0.trial178.stdout:1073: oid 12 updating version 0 to 345 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1073: oid 12 version 345 is already newer than 343 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1073: oid 12 version 345 is already newer than 344 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 345 (ObjNum 392 snap 111 seq_num 392) dirty exists 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1073: left oid 12 (ObjNum 392 snap 111 seq_num 392) 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1074: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1074: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1074: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 348 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:20:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1075: expect (ObjNum 384 snap 106 seq_num 384) 2026-02-21T22:20:34.570 INFO:tasks.rados.rados.0.trial178.stdout:1076: finishing write tid 1 to trial17818531-46 2026-02-21T22:20:34.570 INFO:tasks.rados.rados.0.trial178.stdout:1076: finishing write tid 2 to trial17818531-46 2026-02-21T22:20:34.570 INFO:tasks.rados.rados.0.trial178.stdout:1076: finishing write tid 3 to trial17818531-46 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: finishing write tid 4 to trial17818531-46 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: oid 46 updating version 0 to 184 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: oid 46 updating version 184 to 185 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: oid 46 version 185 is already newer than 183 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 185 (ObjNum 393 snap 111 seq_num 393) dirty exists 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: left oid 46 (ObjNum 393 snap 111 seq_num 393) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1077: finishing copy_from racing read to trial17818531-1 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1077: finishing copy_from to trial17818531-1 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 166 (ObjNum 349 snap 99 seq_num 349) dirty exists 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1073: done (4 left) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1074: done (3 left) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1075: done (2 left) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1076: done (1 left) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1077: done (0 left) 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1078: write oid 26 current snap is 111 2026-02-21T22:20:34.571 INFO:tasks.rados.rados.0.trial178.stdout:1078: seq_num 394 ranges {621958=782109,2089670=532288} 2026-02-21T22:20:34.580 INFO:tasks.rados.rados.0.trial178.stdout:1078: writing trial17818531-26 from 621958 to 1404067 tid 1 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1078: writing trial17818531-26 from 2089670 to 2621958 tid 2 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1079: copy_from oid 7 from oid 35 current snap is 111 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1078: finishing write tid 1 to trial17818531-26 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1080: copy_from oid 30 from oid 48 current snap is 111 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1081: rollback oid 22 current snap is 111 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 110 2026-02-21T22:20:34.588 INFO:tasks.rados.rados.0.trial178.stdout:1082: delete oid 45 current snap is 111 2026-02-21T22:20:34.589 INFO:tasks.rados.rados.0.trial178.stdout:1080: finishing copy_from to trial17818531-30 2026-02-21T22:20:34.589 INFO:tasks.rados.rados.0.trial178.stdout:1080: got expected ENOENT (src dne) 2026-02-21T22:20:34.589 INFO:tasks.rados.rados.0.trial178.stdout:1080: finishing copy_from racing read to trial17818531-30 2026-02-21T22:20:34.590 INFO:tasks.rados.rados.0.trial178.stdout:1080: done (4 left) 2026-02-21T22:20:34.590 INFO:tasks.rados.rados.0.trial178.stdout:1082: done (3 left) 2026-02-21T22:20:34.590 INFO:tasks.rados.rados.0.trial178.stdout:1083: read oid 50 snap -1 2026-02-21T22:20:34.590 INFO:tasks.rados.rados.0.trial178.stdout:1083: expect (ObjNum 312 snap 91 seq_num 312) 2026-02-21T22:20:34.590 INFO:tasks.rados.rados.0.trial178.stdout:1084: read oid 9 snap 110 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1084: expect (ObjNum 385 snap 108 seq_num 385) 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1085: rollback oid 10 current snap is 111 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 106 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1086: rollback oid 1 current snap is 111 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 105 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1087: copy_from oid 27 from oid 28 current snap is 111 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1088: write oid 11 current snap is 111 2026-02-21T22:20:34.591 INFO:tasks.rados.rados.0.trial178.stdout:1088: seq_num 396 ranges {655572=648960,1922987=732585} 2026-02-21T22:20:34.601 INFO:tasks.rados.rados.0.trial178.stdout:1088: writing trial17818531-11 from 655572 to 1304532 tid 1 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1088: writing trial17818531-11 from 1922987 to 2655572 tid 2 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1089: read oid 38 snap -1 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1089: expect (ObjNum 377 snap 105 seq_num 377) 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1090: rollback oid 39 current snap is 111 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 109 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1081: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1091: copy_from oid 41 from oid 30 current snap is 111 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1081: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1081: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 351 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1078: finishing write tid 2 to trial17818531-26 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1078: finishing write tid 3 to trial17818531-26 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1081: done (11 left) 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1092: write (excl) oid 42 current snap is 111 2026-02-21T22:20:34.610 INFO:tasks.rados.rados.0.trial178.stdout:1092: seq_num 397 ranges {672379=724209,1865116=608472} 2026-02-21T22:20:34.619 INFO:tasks.rados.rados.0.trial178.stdout:1092: writing trial17818531-42 from 672379 to 1396588 tid 1 2026-02-21T22:20:34.627 INFO:tasks.rados.rados.0.trial178.stdout:1092: writing trial17818531-42 from 1865116 to 2473588 tid 2 2026-02-21T22:20:34.627 INFO:tasks.rados.rados.0.trial178.stdout:1093: write (excl) oid 29 current snap is 111 2026-02-21T22:20:34.627 INFO:tasks.rados.rados.0.trial178.stdout:1093: seq_num 398 ranges {689186=799458,1923598=752130} 2026-02-21T22:20:34.632 INFO:tasks.rados.rados.0.trial178.stdout:1093: writing trial17818531-29 from 689186 to 1488644 tid 1 2026-02-21T22:20:34.637 INFO:tasks.rados.rados.0.trial178.stdout:1093: writing trial17818531-29 from 1923598 to 2675728 tid 2 2026-02-21T22:20:34.637 INFO:tasks.rados.rados.0.trial178.stdout:1094: delete oid 5 current snap is 111 2026-02-21T22:20:34.637 INFO:tasks.rados.rados.0.trial178.stdout:1086: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1078: finishing write tid 4 to trial17818531-26 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1078: oid 26 updating version 0 to 415 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1078: oid 26 version 415 is already newer than 414 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1078: oid 26 version 415 is already newer than 413 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 415 (ObjNum 394 snap 111 seq_num 394) dirty exists 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1078: left oid 26 (ObjNum 394 snap 111 seq_num 394) 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1086: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:20:34.638 INFO:tasks.rados.rados.0.trial178.stdout:1086: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:20:34.639 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 400 (ObjNum 331 snap 95 seq_num 331) dirty exists 2026-02-21T22:20:34.639 INFO:tasks.rados.rados.0.trial178.stdout:1084: expect (ObjNum 385 snap 108 seq_num 385) 2026-02-21T22:20:34.639 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:34.678 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:34.753 INFO:tasks.rados.rados.0.trial178.stdout:1085: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:20:34.753 INFO:tasks.rados.rados.0.trial178.stdout:1085: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:20:34.753 INFO:tasks.rados.rados.0.trial178.stdout:1085: finishing rollback tid 2 to trial17818531-10 2026-02-21T22:20:34.753 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 418 (ObjNum 299 snap 88 seq_num 299) dirty exists 2026-02-21T22:20:34.753 INFO:tasks.rados.rados.0.trial178.stdout:1083: expect (ObjNum 312 snap 91 seq_num 312) 2026-02-21T22:20:34.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:34.856 INFO:tasks.rados.rados.0.trial178.stdout:1087: finishing copy_from to trial17818531-27 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1087: got expected ENOENT (src dne) 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1087: finishing copy_from racing read to trial17818531-27 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1079: finishing copy_from racing read to trial17818531-7 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1079: finishing copy_from to trial17818531-7 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1088: finishing write tid 1 to trial17818531-11 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1088: finishing write tid 2 to trial17818531-11 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1078: done (13 left) 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1079: done (12 left) 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1083: done (11 left) 2026-02-21T22:20:34.857 INFO:tasks.rados.rados.0.trial178.stdout:1084: done (10 left) 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1085: done (9 left) 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1086: done (8 left) 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1087: done (7 left) 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1094: done (6 left) 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1095: write oid 50 current snap is 111 2026-02-21T22:20:34.858 INFO:tasks.rados.rados.0.trial178.stdout:1095: seq_num 400 ranges {722800=549956,2040562=682238} 2026-02-21T22:20:34.865 INFO:tasks.rados.rados.0.trial178.stdout:1095: writing trial17818531-50 from 722800 to 1272756 tid 1 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1095: writing trial17818531-50 from 2040562 to 2722800 tid 2 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1096: rollback oid 6 current snap is 111 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 97 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1090: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 367 (ObjNum 0 snap 0 seq_num 304) dirty dne 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: finishing write tid 3 to trial17818531-11 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: finishing write tid 4 to trial17818531-11 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: oid 11 updating version 0 to 518 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: oid 11 version 518 is already newer than 516 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: oid 11 version 518 is already newer than 517 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 518 (ObjNum 396 snap 111 seq_num 396) dirty exists 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1088: left oid 11 (ObjNum 396 snap 111 seq_num 396) 2026-02-21T22:20:34.875 INFO:tasks.rados.rados.0.trial178.stdout:1089: expect (ObjNum 377 snap 105 seq_num 377) 2026-02-21T22:20:34.928 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2002741 2026-02-21T22:20:34.937 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:34.973 INFO:tasks.rados.rados.0.trial178.stdout:1091: finishing copy_from racing read to trial17818531-41 2026-02-21T22:20:34.973 INFO:tasks.rados.rados.0.trial178.stdout:1091: finishing copy_from to trial17818531-41 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 268 (ObjNum 358 snap 101 seq_num 358) dirty exists 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: finishing write tid 1 to trial17818531-42 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1093: finishing write tid 1 to trial17818531-29 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: finishing write tid 2 to trial17818531-42 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: finishing write tid 3 to trial17818531-42 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: finishing write tid 4 to trial17818531-42 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: oid 42 updating version 0 to 541 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: oid 42 version 541 is already newer than 539 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: oid 42 version 541 is already newer than 540 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 541 (ObjNum 397 snap 111 seq_num 397) dirty exists 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1092: left oid 42 (ObjNum 397 snap 111 seq_num 397) 2026-02-21T22:20:34.974 INFO:tasks.rados.rados.0.trial178.stdout:1093: finishing write tid 2 to trial17818531-29 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: finishing write tid 3 to trial17818531-29 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: finishing write tid 4 to trial17818531-29 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: oid 29 updating version 0 to 172 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: oid 29 version 172 is already newer than 170 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: oid 29 version 172 is already newer than 171 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 172 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1093: left oid 29 (ObjNum 398 snap 111 seq_num 398) 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1095: finishing write tid 1 to trial17818531-50 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1096: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 0 (ObjNum 225 snap 0 seq_num 2885557680) dirty dne 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1088: done (7 left) 2026-02-21T22:20:34.975 INFO:tasks.rados.rados.0.trial178.stdout:1089: done (6 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1090: done (5 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1091: done (4 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1092: done (3 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1093: done (2 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1096: done (1 left) 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1097: write oid 38 current snap is 111 2026-02-21T22:20:34.976 INFO:tasks.rados.rados.0.trial178.stdout:1097: seq_num 401 ranges {739607=625205,1982691=499457} 2026-02-21T22:20:34.982 INFO:tasks.rados.rados.0.trial178.stdout:1097: writing trial17818531-38 from 739607 to 1364812 tid 1 2026-02-21T22:20:34.990 INFO:tasks.rados.rados.0.trial178.stdout:1097: writing trial17818531-38 from 1982691 to 2482148 tid 2 2026-02-21T22:20:34.990 INFO:tasks.rados.rados.0.trial178.stdout:1098: read oid 7 snap 105 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1098: expect (ObjNum 334 snap 95 seq_num 334) 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1099: write oid 21 current snap is 111 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: finishing write tid 2 to trial17818531-50 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: finishing write tid 3 to trial17818531-50 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: finishing write tid 4 to trial17818531-50 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: oid 50 updating version 0 to 354 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: oid 50 version 354 is already newer than 353 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: oid 50 version 354 is already newer than 352 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 354 (ObjNum 400 snap 111 seq_num 400) dirty exists 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1095: left oid 50 (ObjNum 400 snap 111 seq_num 400) 2026-02-21T22:20:34.991 INFO:tasks.rados.rados.0.trial178.stdout:1097: finishing write tid 1 to trial17818531-38 2026-02-21T22:20:34.992 INFO:tasks.rados.rados.0.trial178.stdout:1099: seq_num 402 ranges {756414=700454,2041173=715241} 2026-02-21T22:20:35.000 INFO:tasks.rados.rados.0.trial178.stdout:1099: writing trial17818531-21 from 756414 to 1456868 tid 1 2026-02-21T22:20:35.009 INFO:tasks.rados.rados.0.trial178.stdout:1099: writing trial17818531-21 from 2041173 to 2756414 tid 2 2026-02-21T22:20:35.009 INFO:tasks.rados.rados.0.trial178.stdout:1095: done (3 left) 2026-02-21T22:20:35.009 INFO:tasks.rados.rados.0.trial178.stdout:1100: delete oid 46 current snap is 111 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: finishing write tid 2 to trial17818531-38 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: finishing write tid 3 to trial17818531-38 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: finishing write tid 4 to trial17818531-38 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: oid 38 updating version 0 to 162 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: oid 38 version 162 is already newer than 160 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: oid 38 version 162 is already newer than 161 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 162 (ObjNum 401 snap 111 seq_num 401) dirty exists 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1097: left oid 38 (ObjNum 401 snap 111 seq_num 401) 2026-02-21T22:20:35.010 INFO:tasks.rados.rados.0.trial178.stdout:1098: expect (ObjNum 334 snap 95 seq_num 334) 2026-02-21T22:20:35.012 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1095161 2026-02-21T22:20:35.053 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 615542 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: finishing write tid 1 to trial17818531-21 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: finishing write tid 2 to trial17818531-21 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: finishing write tid 3 to trial17818531-21 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: finishing write tid 4 to trial17818531-21 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: oid 21 updating version 0 to 543 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: oid 21 updating version 543 to 545 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: oid 21 version 545 is already newer than 544 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 545 (ObjNum 402 snap 111 seq_num 402) dirty exists 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: left oid 21 (ObjNum 402 snap 111 seq_num 402) 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1097: done (3 left) 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1098: done (2 left) 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1099: done (1 left) 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1100: done (0 left) 2026-02-21T22:20:35.102 INFO:tasks.rados.rados.0.trial178.stdout:1101: delete oid 5 current snap is 111 2026-02-21T22:20:35.103 INFO:tasks.rados.rados.0.trial178.stdout:1101: done (0 left) 2026-02-21T22:20:35.103 INFO:tasks.rados.rados.0.trial178.stdout:1102: copy_from oid 42 from oid 30 current snap is 111 2026-02-21T22:20:35.103 INFO:tasks.rados.rados.0.trial178.stdout:1103: delete oid 5 current snap is 111 2026-02-21T22:20:35.107 INFO:tasks.rados.rados.0.trial178.stdout:1103: done (1 left) 2026-02-21T22:20:35.108 INFO:tasks.rados.rados.0.trial178.stdout:1104: delete oid 8 current snap is 111 2026-02-21T22:20:35.109 INFO:tasks.rados.rados.0.trial178.stdout:1104: done (1 left) 2026-02-21T22:20:35.109 INFO:tasks.rados.rados.0.trial178.stdout:1105: read oid 28 snap -1 2026-02-21T22:20:35.109 INFO:tasks.rados.rados.0.trial178.stdout:1105: expect deleted 2026-02-21T22:20:35.109 INFO:tasks.rados.rados.0.trial178.stdout:1106: snap_create 2026-02-21T22:20:35.115 INFO:tasks.rados.rados.0.trial178.stdout:1102: finishing copy_from racing read to trial17818531-42 2026-02-21T22:20:35.115 INFO:tasks.rados.rados.0.trial178.stdout:1102: finishing copy_from to trial17818531-42 2026-02-21T22:20:35.115 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 268 (ObjNum 358 snap 101 seq_num 358) dirty exists 2026-02-21T22:20:35.418 INFO:tasks.rados.rados.0.trial178.stdout:1102: done (2 left) 2026-02-21T22:20:35.418 INFO:tasks.rados.rados.0.trial178.stdout:1105: done (1 left) 2026-02-21T22:20:35.418 INFO:tasks.rados.rados.0.trial178.stdout:1106: done (0 left) 2026-02-21T22:20:35.418 INFO:tasks.rados.rados.0.trial178.stdout:1107: snap_create 2026-02-21T22:20:36.419 INFO:tasks.rados.rados.0.trial178.stdout:1107: done (0 left) 2026-02-21T22:20:36.420 INFO:tasks.rados.rados.0.trial178.stdout:1108: write (excl) oid 39 current snap is 113 2026-02-21T22:20:36.420 INFO:tasks.rados.rados.0.trial178.stdout:1108: seq_num 407 ranges {440449=793052,1933583=510464} 2026-02-21T22:20:36.428 INFO:tasks.rados.rados.0.trial178.stdout:1108: writing trial17818531-39 from 440449 to 1233501 tid 1 2026-02-21T22:20:36.437 INFO:tasks.rados.rados.0.trial178.stdout:1108: writing trial17818531-39 from 1933583 to 2444047 tid 2 2026-02-21T22:20:36.437 INFO:tasks.rados.rados.0.trial178.stdout:1109: copy_from oid 25 from oid 43 current snap is 113 2026-02-21T22:20:36.437 INFO:tasks.rados.rados.0.trial178.stdout:1108: finishing write tid 1 to trial17818531-39 2026-02-21T22:20:36.437 INFO:tasks.rados.rados.0.trial178.stdout:1110: snap_create 2026-02-21T22:20:36.438 INFO:tasks.rados.rados.0.trial178.stdout:1109: finishing copy_from to trial17818531-25 2026-02-21T22:20:36.439 INFO:tasks.rados.rados.0.trial178.stdout:1109: got expected ENOENT (src dne) 2026-02-21T22:20:36.439 INFO:tasks.rados.rados.0.trial178.stdout:1109: finishing copy_from racing read to trial17818531-25 2026-02-21T22:20:36.439 INFO:tasks.rados.rados.0.trial178.stdout:1108: finishing write tid 2 to trial17818531-39 2026-02-21T22:20:36.439 INFO:tasks.rados.rados.0.trial178.stdout:1108: finishing write tid 3 to trial17818531-39 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:1108: finishing write tid 4 to trial17818531-39 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:1108: oid 39 updating version 0 to 372 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:1108: oid 39 updating version 372 to 373 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:1108: oid 39 version 373 is already newer than 371 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 373 (ObjNum 407 snap 113 seq_num 407) dirty exists 2026-02-21T22:20:36.440 INFO:tasks.rados.rados.0.trial178.stdout:1108: left oid 39 (ObjNum 407 snap 113 seq_num 407) 2026-02-21T22:20:37.411 INFO:tasks.rados.rados.0.trial178.stdout:1108: done (2 left) 2026-02-21T22:20:37.411 INFO:tasks.rados.rados.0.trial178.stdout:1109: done (1 left) 2026-02-21T22:20:37.411 INFO:tasks.rados.rados.0.trial178.stdout:1110: done (0 left) 2026-02-21T22:20:37.412 INFO:tasks.rados.rados.0.trial178.stdout:1111: copy_from oid 22 from oid 13 current snap is 114 2026-02-21T22:20:37.412 INFO:tasks.rados.rados.0.trial178.stdout:1112: delete oid 49 current snap is 114 2026-02-21T22:20:37.414 INFO:tasks.rados.rados.0.trial178.stdout:1112: done (1 left) 2026-02-21T22:20:37.414 INFO:tasks.rados.rados.0.trial178.stdout:1113: copy_from oid 46 from oid 38 current snap is 114 2026-02-21T22:20:37.414 INFO:tasks.rados.rados.0.trial178.stdout:1114: snap_remove snap 109 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1114: done (2 left) 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1115: rollback oid 29 current snap is 114 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 111 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1113: finishing copy_from racing read to trial17818531-46 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1113: finishing copy_from to trial17818531-46 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 162 (ObjNum 401 snap 111 seq_num 401) dirty exists 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1111: finishing copy_from racing read to trial17818531-22 2026-02-21T22:20:38.416 INFO:tasks.rados.rados.0.trial178.stdout:1111: finishing copy_from to trial17818531-22 2026-02-21T22:20:38.417 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 478 (ObjNum 356 snap 101 seq_num 356) dirty exists 2026-02-21T22:20:38.417 INFO:tasks.rados.rados.0.trial178.stdout:1111: done (2 left) 2026-02-21T22:20:38.417 INFO:tasks.rados.rados.0.trial178.stdout:1113: done (1 left) 2026-02-21T22:20:38.417 INFO:tasks.rados.rados.0.trial178.stdout:1116: write (excl) oid 48 current snap is 114 2026-02-21T22:20:38.417 INFO:tasks.rados.rados.0.trial178.stdout:1116: seq_num 409 ranges {474063=543550,1650547=514133,2765651=108412} 2026-02-21T22:20:38.422 INFO:tasks.rados.rados.0.trial178.stdout:1116: writing trial17818531-48 from 474063 to 1017613 tid 1 2026-02-21T22:20:38.430 INFO:tasks.rados.rados.0.trial178.stdout:1116: writing trial17818531-48 from 1650547 to 2164680 tid 2 2026-02-21T22:20:38.435 INFO:tasks.rados.rados.0.trial178.stdout:1116: writing trial17818531-48 from 2765651 to 2874063 tid 3 2026-02-21T22:20:38.435 INFO:tasks.rados.rados.0.trial178.stdout:1117: rollback oid 9 current snap is 114 2026-02-21T22:20:38.435 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 110 2026-02-21T22:20:38.435 INFO:tasks.rados.rados.0.trial178.stdout:1115: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:20:38.435 INFO:tasks.rados.rados.0.trial178.stdout:1115: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:1115: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 178 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:1116: finishing write tid 1 to trial17818531-48 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:1116: finishing write tid 2 to trial17818531-48 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:1115: done (2 left) 2026-02-21T22:20:38.436 INFO:tasks.rados.rados.0.trial178.stdout:1118: snap_create 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: finishing write tid 3 to trial17818531-48 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: finishing write tid 4 to trial17818531-48 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: finishing write tid 5 to trial17818531-48 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: oid 48 updating version 0 to 403 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: oid 48 version 403 is already newer than 401 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: oid 48 updating version 403 to 404 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:1116: oid 48 version 404 is already newer than 402 2026-02-21T22:20:38.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 404 (ObjNum 409 snap 114 seq_num 409) dirty exists 2026-02-21T22:20:38.438 INFO:tasks.rados.rados.0.trial178.stdout:1116: left oid 48 (ObjNum 409 snap 114 seq_num 409) 2026-02-21T22:20:38.438 INFO:tasks.rados.rados.0.trial178.stdout:1117: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:20:38.438 INFO:tasks.rados.rados.0.trial178.stdout:1117: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:20:38.438 INFO:tasks.rados.rados.0.trial178.stdout:1117: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:20:38.438 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 376 (ObjNum 385 snap 108 seq_num 385) dirty exists 2026-02-21T22:20:38.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:38.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:39.426 INFO:tasks.rados.rados.0.trial178.stdout:1116: done (2 left) 2026-02-21T22:20:39.426 INFO:tasks.rados.rados.0.trial178.stdout:1117: done (1 left) 2026-02-21T22:20:39.426 INFO:tasks.rados.rados.0.trial178.stdout:1118: done (0 left) 2026-02-21T22:20:39.426 INFO:tasks.rados.rados.0.trial178.stdout:1119: copy_from oid 23 from oid 26 current snap is 115 2026-02-21T22:20:39.427 INFO:tasks.rados.rados.0.trial178.stdout:1120: delete oid 30 current snap is 115 2026-02-21T22:20:39.428 INFO:tasks.rados.rados.0.trial178.stdout:1120: done (1 left) 2026-02-21T22:20:39.428 INFO:tasks.rados.rados.0.trial178.stdout:1121: snap_remove snap 112 2026-02-21T22:20:41.247 INFO:tasks.rados.rados.0.trial178.stdout:1121: done (1 left) 2026-02-21T22:20:41.247 INFO:tasks.rados.rados.0.trial178.stdout:1122: read oid 35 snap -1 2026-02-21T22:20:41.247 INFO:tasks.rados.rados.0.trial178.stdout:1122: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:41.247 INFO:tasks.rados.rados.0.trial178.stdout:1123: delete oid 47 current snap is 115 2026-02-21T22:20:41.247 INFO:tasks.rados.rados.0.trial178.stdout:1119: finishing copy_from racing read to trial17818531-23 2026-02-21T22:20:41.248 INFO:tasks.rados.rados.0.trial178.stdout:1119: finishing copy_from to trial17818531-23 2026-02-21T22:20:41.248 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 415 (ObjNum 394 snap 111 seq_num 394) dirty exists 2026-02-21T22:20:41.249 INFO:tasks.rados.rados.0.trial178.stdout:1119: done (2 left) 2026-02-21T22:20:41.249 INFO:tasks.rados.rados.0.trial178.stdout:1123: done (1 left) 2026-02-21T22:20:41.249 INFO:tasks.rados.rados.0.trial178.stdout:1124: snap_remove snap 106 2026-02-21T22:20:41.641 INFO:tasks.rados.rados.0.trial178.stdout:1124: done (1 left) 2026-02-21T22:20:41.641 INFO:tasks.rados.rados.0.trial178.stdout:1125: copy_from oid 34 from oid 29 current snap is 115 2026-02-21T22:20:41.641 INFO:tasks.rados.rados.0.trial178.stdout:1122: expect (ObjNum 104 snap 25 seq_num 104) 2026-02-21T22:20:41.642 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1125: finishing copy_from racing read to trial17818531-34 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1125: finishing copy_from to trial17818531-34 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 178 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1122: done (1 left) 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1125: done (0 left) 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1126: rollback oid 39 current snap is 115 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 107 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1127: write (excl) oid 44 current snap is 115 2026-02-21T22:20:41.794 INFO:tasks.rados.rados.0.trial178.stdout:1127: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2026-02-21T22:20:41.801 INFO:tasks.rados.rados.0.trial178.stdout:1127: writing trial17818531-44 from 524484 to 1010134 tid 1 2026-02-21T22:20:41.808 INFO:tasks.rados.rados.0.trial178.stdout:1127: writing trial17818531-44 from 1425993 to 2087453 tid 2 2026-02-21T22:20:41.814 INFO:tasks.rados.rados.0.trial178.stdout:1127: writing trial17818531-44 from 2847920 to 2924484 tid 3 2026-02-21T22:20:41.814 INFO:tasks.rados.rados.0.trial178.stdout:1128: read oid 32 snap 105 2026-02-21T22:20:41.814 INFO:tasks.rados.rados.0.trial178.stdout:1128: expect (ObjNum 368 snap 103 seq_num 368) 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1129: rollback oid 40 current snap is 115 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1126: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1126: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 378 (ObjNum 705 snap 0 seq_num 1453350627) dirty dne 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1127: finishing write tid 1 to trial17818531-44 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1127: finishing write tid 2 to trial17818531-44 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 110 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1126: done (3 left) 2026-02-21T22:20:41.815 INFO:tasks.rados.rados.0.trial178.stdout:1130: delete oid 50 current snap is 115 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: finishing write tid 3 to trial17818531-44 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: finishing write tid 4 to trial17818531-44 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: finishing write tid 5 to trial17818531-44 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: oid 44 updating version 0 to 346 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: oid 44 updating version 346 to 348 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: oid 44 updating version 348 to 349 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: oid 44 version 349 is already newer than 347 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 349 (ObjNum 412 snap 115 seq_num 412) dirty exists 2026-02-21T22:20:41.816 INFO:tasks.rados.rados.0.trial178.stdout:1127: left oid 44 (ObjNum 412 snap 115 seq_num 412) 2026-02-21T22:20:41.821 INFO:tasks.rados.rados.0.trial178.stdout:1128: expect (ObjNum 368 snap 103 seq_num 368) 2026-02-21T22:20:41.821 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:41.885 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:41.933 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 532872 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1129: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1129: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1129: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 481 (ObjNum 324 snap 95 seq_num 324) dirty exists 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1127: done (3 left) 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1128: done (2 left) 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1129: done (1 left) 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1130: done (0 left) 2026-02-21T22:20:41.972 INFO:tasks.rados.rados.0.trial178.stdout:1131: write (excl) oid 46 current snap is 115 2026-02-21T22:20:41.973 INFO:tasks.rados.rados.0.trial178.stdout:1131: seq_num 414 ranges {558098=636148,1826604=665129} 2026-02-21T22:20:41.979 INFO:tasks.rados.rados.0.trial178.stdout:1131: writing trial17818531-46 from 558098 to 1194246 tid 1 2026-02-21T22:20:41.988 INFO:tasks.rados.rados.0.trial178.stdout:1131: writing trial17818531-46 from 1826604 to 2491733 tid 2 2026-02-21T22:20:41.989 INFO:tasks.rados.rados.0.trial178.stdout:1132: write (excl) oid 22 current snap is 115 2026-02-21T22:20:41.989 INFO:tasks.rados.rados.0.trial178.stdout:1132: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2026-02-21T22:20:41.997 INFO:tasks.rados.rados.0.trial178.stdout:1132: writing trial17818531-22 from 574905 to 1286302 tid 1 2026-02-21T22:20:42.005 INFO:tasks.rados.rados.0.trial178.stdout:1132: writing trial17818531-22 from 1885086 to 2410226 tid 2 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1132: writing trial17818531-22 from 2930189 to 2974905 tid 3 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1133: read oid 39 snap -1 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1133: expect deleted 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1134: delete oid 8 current snap is 115 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1131: finishing write tid 1 to trial17818531-46 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1131: finishing write tid 2 to trial17818531-46 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1131: finishing write tid 3 to trial17818531-46 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1131: finishing write tid 4 to trial17818531-46 2026-02-21T22:20:42.010 INFO:tasks.rados.rados.0.trial178.stdout:1131: oid 46 updating version 0 to 418 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1131: oid 46 version 418 is already newer than 417 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1131: oid 46 version 418 is already newer than 416 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 418 (ObjNum 414 snap 115 seq_num 414) dirty exists 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1131: left oid 46 (ObjNum 414 snap 115 seq_num 414) 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1132: finishing write tid 1 to trial17818531-22 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1132: finishing write tid 2 to trial17818531-22 2026-02-21T22:20:42.011 INFO:tasks.rados.rados.0.trial178.stdout:1131: done (3 left) 2026-02-21T22:20:42.012 INFO:tasks.rados.rados.0.trial178.stdout:1133: done (2 left) 2026-02-21T22:20:42.012 INFO:tasks.rados.rados.0.trial178.stdout:1134: done (1 left) 2026-02-21T22:20:42.012 INFO:tasks.rados.rados.0.trial178.stdout:1135: write oid 15 current snap is 115 2026-02-21T22:20:42.012 INFO:tasks.rados.rados.0.trial178.stdout:1135: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2026-02-21T22:20:42.018 INFO:tasks.rados.rados.0.trial178.stdout:1135: writing trial17818531-15 from 608519 to 1070414 tid 1 2026-02-21T22:20:42.026 INFO:tasks.rados.rados.0.trial178.stdout:1135: writing trial17818531-15 from 1602050 to 2130859 tid 2 2026-02-21T22:20:42.032 INFO:tasks.rados.rados.0.trial178.stdout:1135: writing trial17818531-15 from 2585035 to 3008519 tid 3 2026-02-21T22:20:42.032 INFO:tasks.rados.rados.0.trial178.stdout:1136: snap_create 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: finishing write tid 3 to trial17818531-22 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: finishing write tid 4 to trial17818531-22 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: finishing write tid 5 to trial17818531-22 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: oid 22 updating version 0 to 484 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: oid 22 updating version 484 to 486 2026-02-21T22:20:42.033 INFO:tasks.rados.rados.0.trial178.stdout:1132: oid 22 version 486 is already newer than 483 2026-02-21T22:20:42.034 INFO:tasks.rados.rados.0.trial178.stdout:1132: oid 22 version 486 is already newer than 485 2026-02-21T22:20:42.034 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 486 (ObjNum 415 snap 115 seq_num 415) dirty exists 2026-02-21T22:20:42.034 INFO:tasks.rados.rados.0.trial178.stdout:1132: left oid 22 (ObjNum 415 snap 115 seq_num 415) 2026-02-21T22:20:42.034 INFO:tasks.rados.rados.0.trial178.stdout:1135: finishing write tid 1 to trial17818531-15 2026-02-21T22:20:42.034 INFO:tasks.rados.rados.0.trial178.stdout:1135: finishing write tid 2 to trial17818531-15 2026-02-21T22:20:42.035 INFO:tasks.rados.rados.0.trial178.stdout:1135: finishing write tid 3 to trial17818531-15 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: finishing write tid 4 to trial17818531-15 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: finishing write tid 5 to trial17818531-15 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: oid 15 updating version 0 to 479 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: oid 15 updating version 479 to 482 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: oid 15 version 482 is already newer than 480 2026-02-21T22:20:42.036 INFO:tasks.rados.rados.0.trial178.stdout:1135: oid 15 version 482 is already newer than 481 2026-02-21T22:20:42.037 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 482 (ObjNum 417 snap 115 seq_num 417) dirty exists 2026-02-21T22:20:42.037 INFO:tasks.rados.rados.0.trial178.stdout:1135: left oid 15 (ObjNum 417 snap 115 seq_num 417) 2026-02-21T22:20:42.414 INFO:tasks.rados.rados.0.trial178.stdout:1132: done (2 left) 2026-02-21T22:20:42.414 INFO:tasks.rados.rados.0.trial178.stdout:1135: done (1 left) 2026-02-21T22:20:42.414 INFO:tasks.rados.rados.0.trial178.stdout:1136: done (0 left) 2026-02-21T22:20:42.415 INFO:tasks.rados.rados.0.trial178.stdout:1137: copy_from oid 42 from oid 32 current snap is 116 2026-02-21T22:20:42.415 INFO:tasks.rados.rados.0.trial178.stdout:1138: write (excl) oid 22 current snap is 116 2026-02-21T22:20:42.415 INFO:tasks.rados.rados.0.trial178.stdout:1138: seq_num 418 ranges {625326=537144,1944179=672467} 2026-02-21T22:20:42.418 INFO:tasks.rados.rados.0.trial178.stdout:1138: writing trial17818531-22 from 625326 to 1162470 tid 1 2026-02-21T22:20:42.425 INFO:tasks.rados.rados.0.trial178.stdout:1138: writing trial17818531-22 from 1944179 to 2616646 tid 2 2026-02-21T22:20:42.425 INFO:tasks.rados.rados.0.trial178.stdout:1139: read oid 49 snap -1 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:1139: expect deleted 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:1140: snap_create 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:1138: finishing write tid 1 to trial17818531-22 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:1137: finishing copy_from racing read to trial17818531-42 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:1137: finishing copy_from to trial17818531-42 2026-02-21T22:20:42.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 490 (ObjNum 368 snap 103 seq_num 368) dirty exists 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: finishing write tid 2 to trial17818531-22 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: finishing write tid 3 to trial17818531-22 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: finishing write tid 4 to trial17818531-22 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: oid 22 updating version 0 to 487 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: oid 22 updating version 487 to 489 2026-02-21T22:20:42.428 INFO:tasks.rados.rados.0.trial178.stdout:1138: oid 22 version 489 is already newer than 488 2026-02-21T22:20:42.429 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 489 (ObjNum 418 snap 116 seq_num 418) dirty exists 2026-02-21T22:20:42.429 INFO:tasks.rados.rados.0.trial178.stdout:1138: left oid 22 (ObjNum 418 snap 116 seq_num 418) 2026-02-21T22:20:43.417 INFO:tasks.rados.rados.0.trial178.stdout:1137: done (3 left) 2026-02-21T22:20:43.417 INFO:tasks.rados.rados.0.trial178.stdout:1138: done (2 left) 2026-02-21T22:20:43.417 INFO:tasks.rados.rados.0.trial178.stdout:1139: done (1 left) 2026-02-21T22:20:43.417 INFO:tasks.rados.rados.0.trial178.stdout:1140: done (0 left) 2026-02-21T22:20:43.417 INFO:tasks.rados.rados.0.trial178.stdout:1141: delete oid 31 current snap is 117 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1141: done (0 left) 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1142: rollback oid 46 current snap is 117 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 115 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1143: copy_from oid 35 from oid 2 current snap is 117 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1144: read oid 15 snap -1 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1144: expect (ObjNum 417 snap 115 seq_num 417) 2026-02-21T22:20:43.419 INFO:tasks.rados.rados.0.trial178.stdout:1145: delete oid 19 current snap is 117 2026-02-21T22:20:43.420 INFO:tasks.rados.rados.0.trial178.stdout:1143: finishing copy_from to trial17818531-35 2026-02-21T22:20:43.420 INFO:tasks.rados.rados.0.trial178.stdout:1143: got expected ENOENT (src dne) 2026-02-21T22:20:43.422 INFO:tasks.rados.rados.0.trial178.stdout:1145: done (3 left) 2026-02-21T22:20:43.422 INFO:tasks.rados.rados.0.trial178.stdout:1146: delete oid 4 current snap is 117 2026-02-21T22:20:43.422 INFO:tasks.rados.rados.0.trial178.stdout:1143: finishing copy_from racing read to trial17818531-35 2026-02-21T22:20:43.424 INFO:tasks.rados.rados.0.trial178.stdout:1144: expect (ObjNum 417 snap 115 seq_num 417) 2026-02-21T22:20:43.530 INFO:tasks.rados.rados.0.trial178.stdout:1142: finishing rollback tid 0 to trial17818531-46 2026-02-21T22:20:43.530 INFO:tasks.rados.rados.0.trial178.stdout:1142: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1142: finishing rollback tid 2 to trial17818531-46 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 421 (ObjNum 414 snap 115 seq_num 414) dirty exists 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1142: done (3 left) 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1143: done (2 left) 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1144: done (1 left) 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1146: done (0 left) 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1147: write (excl) oid 18 current snap is 117 2026-02-21T22:20:43.531 INFO:tasks.rados.rados.0.trial178.stdout:1147: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2026-02-21T22:20:43.538 INFO:tasks.rados.rados.0.trial178.stdout:1147: writing trial17818531-18 from 692554 to 1247047 tid 1 2026-02-21T22:20:43.546 INFO:tasks.rados.rados.0.trial178.stdout:1147: writing trial17818531-18 from 1778107 to 2457912 tid 2 2026-02-21T22:20:43.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:43.549 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1147: writing trial17818531-18 from 2889444 to 3092554 tid 3 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1148: copy_from oid 10 from oid 40 current snap is 117 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1147: finishing write tid 1 to trial17818531-18 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1147: finishing write tid 2 to trial17818531-18 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1149: write (excl) oid 26 current snap is 117 2026-02-21T22:20:43.551 INFO:tasks.rados.rados.0.trial178.stdout:1149: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2026-02-21T22:20:43.560 INFO:tasks.rados.rados.0.trial178.stdout:1149: writing trial17818531-26 from 709361 to 1339103 tid 1 2026-02-21T22:20:43.567 INFO:tasks.rados.rados.0.trial178.stdout:1149: writing trial17818531-26 from 1836589 to 2376405 tid 2 2026-02-21T22:20:43.573 INFO:tasks.rados.rados.0.trial178.stdout:1149: writing trial17818531-26 from 3033220 to 3109361 tid 3 2026-02-21T22:20:43.573 INFO:tasks.rados.rados.0.trial178.stdout:1150: rollback oid 39 current snap is 117 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 114 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: finishing write tid 3 to trial17818531-18 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: finishing write tid 4 to trial17818531-18 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: finishing write tid 5 to trial17818531-18 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: oid 18 updating version 0 to 333 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: oid 18 version 333 is already newer than 331 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: oid 18 version 333 is already newer than 332 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: oid 18 version 333 is already newer than 330 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 333 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1147: left oid 18 (ObjNum 422 snap 117 seq_num 422) 2026-02-21T22:20:43.574 INFO:tasks.rados.rados.0.trial178.stdout:1148: finishing copy_from racing read to trial17818531-10 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1148: finishing copy_from to trial17818531-10 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 481 (ObjNum 324 snap 95 seq_num 324) dirty exists 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1149: finishing write tid 1 to trial17818531-26 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1149: finishing write tid 2 to trial17818531-26 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1147: done (3 left) 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1148: done (2 left) 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1151: copy_from oid 4 from oid 47 current snap is 117 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1152: write oid 40 current snap is 117 2026-02-21T22:20:43.575 INFO:tasks.rados.rados.0.trial178.stdout:1152: seq_num 424 ranges {726168=704991,1895071=683474} 2026-02-21T22:20:43.582 INFO:tasks.rados.rados.0.trial178.stdout:1152: writing trial17818531-40 from 726168 to 1431159 tid 1 2026-02-21T22:20:43.589 INFO:tasks.rados.rados.0.trial178.stdout:1152: writing trial17818531-40 from 1895071 to 2578545 tid 2 2026-02-21T22:20:43.589 INFO:tasks.rados.rados.0.trial178.stdout:1153: snap_create 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1149: finishing write tid 3 to trial17818531-26 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1151: finishing copy_from to trial17818531-4 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1151: got expected ENOENT (src dne) 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1151: finishing copy_from racing read to trial17818531-4 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1150: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1149: finishing write tid 4 to trial17818531-26 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1150: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 380 (ObjNum 407 snap 113 seq_num 407) dirty exists 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1149: finishing write tid 5 to trial17818531-26 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1149: oid 26 updating version 0 to 484 2026-02-21T22:20:43.590 INFO:tasks.rados.rados.0.trial178.stdout:1149: oid 26 version 484 is already newer than 482 2026-02-21T22:20:43.591 INFO:tasks.rados.rados.0.trial178.stdout:1149: oid 26 updating version 484 to 485 2026-02-21T22:20:43.591 INFO:tasks.rados.rados.0.trial178.stdout:1149: oid 26 version 485 is already newer than 483 2026-02-21T22:20:43.591 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 485 (ObjNum 423 snap 117 seq_num 423) dirty exists 2026-02-21T22:20:43.591 INFO:tasks.rados.rados.0.trial178.stdout:1149: left oid 26 (ObjNum 423 snap 117 seq_num 423) 2026-02-21T22:20:43.591 INFO:tasks.rados.rados.0.trial178.stdout:1152: finishing write tid 1 to trial17818531-40 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: finishing write tid 2 to trial17818531-40 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: finishing write tid 3 to trial17818531-40 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: finishing write tid 4 to trial17818531-40 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: oid 40 updating version 0 to 490 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: oid 40 updating version 490 to 492 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: oid 40 version 492 is already newer than 491 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 492 (ObjNum 424 snap 117 seq_num 424) dirty exists 2026-02-21T22:20:43.592 INFO:tasks.rados.rados.0.trial178.stdout:1152: left oid 40 (ObjNum 424 snap 117 seq_num 424) 2026-02-21T22:20:44.422 INFO:tasks.rados.rados.0.trial178.stdout:1149: done (4 left) 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1150: done (3 left) 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1151: done (2 left) 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1152: done (1 left) 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1153: done (0 left) 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1154: read oid 36 snap -1 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1154: expect deleted 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1155: copy_from oid 47 from oid 45 current snap is 118 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1156: write oid 2 current snap is 118 2026-02-21T22:20:44.423 INFO:tasks.rados.rados.0.trial178.stdout:1156: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2026-02-21T22:20:44.428 INFO:tasks.rados.rados.0.trial178.stdout:1156: writing trial17818531-2 from 742975 to 1523215 tid 1 2026-02-21T22:20:44.431 INFO:tasks.rados.rados.0.trial178.stdout:1156: writing trial17818531-2 from 1953553 to 2380685 tid 2 2026-02-21T22:20:44.434 INFO:tasks.rados.rados.0.trial178.stdout:1156: writing trial17818531-2 from 2971713 to 3142975 tid 3 2026-02-21T22:20:44.434 INFO:tasks.rados.rados.0.trial178.stdout:1157: read oid 33 snap 111 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1157: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1158: snap_create 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1155: finishing copy_from to trial17818531-47 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1155: got expected ENOENT (src dne) 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1155: finishing copy_from racing read to trial17818531-47 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1156: finishing write tid 1 to trial17818531-2 2026-02-21T22:20:44.435 INFO:tasks.rados.rados.0.trial178.stdout:1156: finishing write tid 2 to trial17818531-2 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: finishing write tid 3 to trial17818531-2 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: finishing write tid 4 to trial17818531-2 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: finishing write tid 5 to trial17818531-2 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: oid 2 updating version 0 to 487 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: oid 2 updating version 487 to 489 2026-02-21T22:20:44.436 INFO:tasks.rados.rados.0.trial178.stdout:1156: oid 2 version 489 is already newer than 486 2026-02-21T22:20:44.437 INFO:tasks.rados.rados.0.trial178.stdout:1156: oid 2 version 489 is already newer than 488 2026-02-21T22:20:44.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 489 (ObjNum 425 snap 118 seq_num 425) dirty exists 2026-02-21T22:20:44.437 INFO:tasks.rados.rados.0.trial178.stdout:1156: left oid 2 (ObjNum 425 snap 118 seq_num 425) 2026-02-21T22:20:44.437 INFO:tasks.rados.rados.0.trial178.stdout:1157: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:20:44.488 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1684209 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1154: done (4 left) 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1155: done (3 left) 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1156: done (2 left) 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1157: done (1 left) 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1158: done (0 left) 2026-02-21T22:20:45.422 INFO:tasks.rados.rados.0.trial178.stdout:1159: snap_remove snap 115 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1159: done (0 left) 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1160: read oid 31 snap 117 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1160: expect deleted 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1161: read oid 20 snap -1 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1161: expect deleted 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1162: copy_from oid 14 from oid 47 current snap is 119 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1163: read oid 49 snap -1 2026-02-21T22:20:46.424 INFO:tasks.rados.rados.0.trial178.stdout:1163: expect deleted 2026-02-21T22:20:46.425 INFO:tasks.rados.rados.0.trial178.stdout:1164: write oid 6 current snap is 119 2026-02-21T22:20:46.425 INFO:tasks.rados.rados.0.trial178.stdout:1164: seq_num 426 ranges {759782=571842,2012035=687143} 2026-02-21T22:20:46.432 INFO:tasks.rados.rados.0.trial178.stdout:1164: writing trial17818531-6 from 759782 to 1331624 tid 1 2026-02-21T22:20:46.442 INFO:tasks.rados.rados.0.trial178.stdout:1164: writing trial17818531-6 from 2012035 to 2699178 tid 2 2026-02-21T22:20:46.442 INFO:tasks.rados.rados.0.trial178.stdout:1165: snap_remove snap 116 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1165: done (5 left) 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1166: copy_from oid 41 from oid 39 current snap is 119 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1162: finishing copy_from to trial17818531-14 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1162: got expected ENOENT (src dne) 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1162: finishing copy_from racing read to trial17818531-14 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1164: finishing write tid 1 to trial17818531-6 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1164: finishing write tid 2 to trial17818531-6 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1164: finishing write tid 3 to trial17818531-6 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1164: finishing write tid 4 to trial17818531-6 2026-02-21T22:20:47.415 INFO:tasks.rados.rados.0.trial178.stdout:1164: oid 6 updating version 0 to 491 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1164: oid 6 version 491 is already newer than 490 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1164: oid 6 updating version 491 to 492 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 492 (ObjNum 426 snap 119 seq_num 426) dirty exists 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1164: left oid 6 (ObjNum 426 snap 119 seq_num 426) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1160: done (5 left) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1161: done (4 left) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1162: done (3 left) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1163: done (2 left) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1164: done (1 left) 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1167: copy_from oid 29 from oid 9 current snap is 119 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:1168: rollback oid 15 current snap is 119 2026-02-21T22:20:47.416 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 111 2026-02-21T22:20:47.417 INFO:tasks.rados.rados.0.trial178.stdout:1169: snap_create 2026-02-21T22:20:47.418 INFO:tasks.rados.rados.0.trial178.stdout:1168: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:20:47.427 INFO:tasks.rados.rados.0.trial178.stdout:1166: finishing copy_from racing read to trial17818531-41 2026-02-21T22:20:47.427 INFO:tasks.rados.rados.0.trial178.stdout:1168: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:20:47.428 INFO:tasks.rados.rados.0.trial178.stdout:1168: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:20:47.428 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 485 (ObjNum 372 snap 105 seq_num 372) dirty exists 2026-02-21T22:20:47.428 INFO:tasks.rados.rados.0.trial178.stdout:1166: finishing copy_from to trial17818531-41 2026-02-21T22:20:47.428 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 380 (ObjNum 407 snap 113 seq_num 407) dirty exists 2026-02-21T22:20:47.429 INFO:tasks.rados.rados.0.trial178.stdout:1167: finishing copy_from racing read to trial17818531-29 2026-02-21T22:20:47.432 INFO:tasks.rados.rados.0.trial178.stdout:1167: finishing copy_from to trial17818531-29 2026-02-21T22:20:47.432 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 376 (ObjNum 385 snap 108 seq_num 385) dirty exists 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1166: done (3 left) 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1167: done (2 left) 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1168: done (1 left) 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1169: done (0 left) 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1170: rollback oid 11 current snap is 120 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 114 2026-02-21T22:20:48.426 INFO:tasks.rados.rados.0.trial178.stdout:1171: snap_create 2026-02-21T22:20:48.429 INFO:tasks.rados.rados.0.trial178.stdout:1170: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:20:48.429 INFO:tasks.rados.rados.0.trial178.stdout:1170: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:20:48.429 INFO:tasks.rados.rados.0.trial178.stdout:1170: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:20:48.429 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 526 (ObjNum 396 snap 111 seq_num 396) dirty exists 2026-02-21T22:20:48.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:48.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:49.426 INFO:tasks.rados.rados.0.trial178.stdout:1170: done (1 left) 2026-02-21T22:20:49.426 INFO:tasks.rados.rados.0.trial178.stdout:1171: done (0 left) 2026-02-21T22:20:49.426 INFO:tasks.rados.rados.0.trial178.stdout:1172: write oid 29 current snap is 121 2026-02-21T22:20:49.426 INFO:tasks.rados.rados.0.trial178.stdout:1172: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2026-02-21T22:20:49.435 INFO:tasks.rados.rados.0.trial178.stdout:1172: writing trial17818531-29 from 776589 to 1423680 tid 1 2026-02-21T22:20:49.442 INFO:tasks.rados.rados.0.trial178.stdout:1172: writing trial17818531-29 from 2070517 to 2501318 tid 2 2026-02-21T22:20:49.447 INFO:tasks.rados.rados.0.trial178.stdout:1172: writing trial17818531-29 from 3026559 to 3176589 tid 3 2026-02-21T22:20:49.447 INFO:tasks.rados.rados.0.trial178.stdout:1173: rollback oid 1 current snap is 121 2026-02-21T22:20:49.448 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 105 2026-02-21T22:20:49.448 INFO:tasks.rados.rados.0.trial178.stdout:1172: finishing write tid 1 to trial17818531-29 2026-02-21T22:20:49.448 INFO:tasks.rados.rados.0.trial178.stdout:1172: finishing write tid 2 to trial17818531-29 2026-02-21T22:20:49.448 INFO:tasks.rados.rados.0.trial178.stdout:1174: snap_remove snap 120 2026-02-21T22:20:50.432 INFO:tasks.rados.rados.0.trial178.stdout:1174: done (2 left) 2026-02-21T22:20:50.433 INFO:tasks.rados.rados.0.trial178.stdout:1175: write oid 15 current snap is 121 2026-02-21T22:20:50.433 INFO:tasks.rados.rados.0.trial178.stdout:1175: seq_num 428 ranges {793396=722340,2128999=690812} 2026-02-21T22:20:50.442 INFO:tasks.rados.rados.0.trial178.stdout:1175: writing trial17818531-15 from 793396 to 1515736 tid 1 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1175: writing trial17818531-15 from 2128999 to 2819811 tid 2 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1176: delete oid 46 current snap is 121 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1172: finishing write tid 3 to trial17818531-29 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1172: finishing write tid 4 to trial17818531-29 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1172: finishing write tid 5 to trial17818531-29 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1172: oid 29 updating version 0 to 377 2026-02-21T22:20:50.452 INFO:tasks.rados.rados.0.trial178.stdout:1172: oid 29 updating version 377 to 379 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1172: oid 29 updating version 379 to 380 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1172: oid 29 version 380 is already newer than 378 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 380 (ObjNum 427 snap 121 seq_num 427) dirty exists 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1172: left oid 29 (ObjNum 427 snap 121 seq_num 427) 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1173: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1173: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1173: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 407 (ObjNum 331 snap 95 seq_num 331) dirty exists 2026-02-21T22:20:50.453 INFO:tasks.rados.rados.0.trial178.stdout:1175: finishing write tid 1 to trial17818531-15 2026-02-21T22:20:50.454 INFO:tasks.rados.rados.0.trial178.stdout:1172: done (3 left) 2026-02-21T22:20:50.454 INFO:tasks.rados.rados.0.trial178.stdout:1173: done (2 left) 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:1176: done (1 left) 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:1177: rollback oid 36 current snap is 121 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 117 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:1178: rollback oid 19 current snap is 121 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 118 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:1179: rollback oid 48 current snap is 121 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 105 2026-02-21T22:20:50.455 INFO:tasks.rados.rados.0.trial178.stdout:1180: snap_create 2026-02-21T22:20:50.456 INFO:tasks.rados.rados.0.trial178.stdout:1177: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:20:50.456 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 170 (ObjNum 529 snap 0 seq_num 1453350630) dirty dne 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1178: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 0 (ObjNum 209 snap 0 seq_num 4122735248) dirty dne 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: finishing write tid 2 to trial17818531-15 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: finishing write tid 3 to trial17818531-15 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: finishing write tid 4 to trial17818531-15 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: oid 15 updating version 0 to 488 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: oid 15 version 488 is already newer than 486 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: oid 15 version 488 is already newer than 487 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 488 (ObjNum 428 snap 121 seq_num 428) dirty exists 2026-02-21T22:20:50.457 INFO:tasks.rados.rados.0.trial178.stdout:1175: left oid 15 (ObjNum 428 snap 121 seq_num 428) 2026-02-21T22:20:50.459 INFO:tasks.rados.rados.0.trial178.stdout:1179: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:20:50.459 INFO:tasks.rados.rados.0.trial178.stdout:1179: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:20:50.460 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 409 (ObjNum 6577 snap 0 seq_num 99601968) dirty dne 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1175: done (4 left) 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1177: done (3 left) 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1178: done (2 left) 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1179: done (1 left) 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1180: done (0 left) 2026-02-21T22:20:51.436 INFO:tasks.rados.rados.0.trial178.stdout:1181: snap_create 2026-02-21T22:20:52.437 INFO:tasks.rados.rados.0.trial178.stdout:1181: done (0 left) 2026-02-21T22:20:52.437 INFO:tasks.rados.rados.0.trial178.stdout:1182: copy_from oid 14 from oid 31 current snap is 123 2026-02-21T22:20:52.438 INFO:tasks.rados.rados.0.trial178.stdout:1183: write (excl) oid 27 current snap is 123 2026-02-21T22:20:52.438 INFO:tasks.rados.rados.0.trial178.stdout:1183: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2026-02-21T22:20:52.443 INFO:tasks.rados.rados.0.trial178.stdout:1183: writing trial17818531-27 from 427010 to 899848 tid 1 2026-02-21T22:20:52.450 INFO:tasks.rados.rados.0.trial178.stdout:1183: writing trial17818531-27 from 1329610 to 2024091 tid 2 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1183: writing trial17818531-27 from 2708828 to 3227010 tid 3 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1184: delete oid 6 current snap is 123 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1182: finishing copy_from to trial17818531-14 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1182: got expected ENOENT (src dne) 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1182: finishing copy_from racing read to trial17818531-14 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1183: finishing write tid 1 to trial17818531-27 2026-02-21T22:20:52.459 INFO:tasks.rados.rados.0.trial178.stdout:1183: finishing write tid 2 to trial17818531-27 2026-02-21T22:20:52.461 INFO:tasks.rados.rados.0.trial178.stdout:1182: done (2 left) 2026-02-21T22:20:52.461 INFO:tasks.rados.rados.0.trial178.stdout:1184: done (1 left) 2026-02-21T22:20:52.461 INFO:tasks.rados.rados.0.trial178.stdout:1185: write oid 38 current snap is 123 2026-02-21T22:20:52.461 INFO:tasks.rados.rados.0.trial178.stdout:1185: seq_num 432 ranges {460624=623336,1846574=698150,3163674=96950} 2026-02-21T22:20:52.468 INFO:tasks.rados.rados.0.trial178.stdout:1185: writing trial17818531-38 from 460624 to 1083960 tid 1 2026-02-21T22:20:52.477 INFO:tasks.rados.rados.0.trial178.stdout:1185: writing trial17818531-38 from 1846574 to 2544724 tid 2 2026-02-21T22:20:52.482 INFO:tasks.rados.rados.0.trial178.stdout:1185: writing trial17818531-38 from 3163674 to 3260624 tid 3 2026-02-21T22:20:52.482 INFO:tasks.rados.rados.0.trial178.stdout:1186: write (excl) oid 12 current snap is 123 2026-02-21T22:20:52.482 INFO:tasks.rados.rados.0.trial178.stdout:1183: finishing write tid 3 to trial17818531-27 2026-02-21T22:20:52.482 INFO:tasks.rados.rados.0.trial178.stdout:1183: finishing write tid 4 to trial17818531-27 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: finishing write tid 5 to trial17818531-27 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: oid 27 updating version 0 to 382 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: oid 27 updating version 382 to 384 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: oid 27 version 384 is already newer than 381 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: oid 27 version 384 is already newer than 383 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 384 (ObjNum 430 snap 123 seq_num 430) dirty exists 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1183: left oid 27 (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1185: finishing write tid 1 to trial17818531-38 2026-02-21T22:20:52.483 INFO:tasks.rados.rados.0.trial178.stdout:1186: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2026-02-21T22:20:52.490 INFO:tasks.rados.rados.0.trial178.stdout:1186: writing trial17818531-12 from 477431 to 1176016 tid 1 2026-02-21T22:20:52.496 INFO:tasks.rados.rados.0.trial178.stdout:1186: writing trial17818531-12 from 1905056 to 2346864 tid 2 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1186: writing trial17818531-12 from 3074744 to 3277431 tid 3 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1183: done (2 left) 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1187: read oid 44 snap 119 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1187: expect (ObjNum 412 snap 115 seq_num 412) 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1188: copy_from oid 36 from oid 33 current snap is 123 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: finishing write tid 2 to trial17818531-38 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: finishing write tid 3 to trial17818531-38 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1189: copy_from oid 42 from oid 34 current snap is 123 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: finishing write tid 4 to trial17818531-38 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: finishing write tid 5 to trial17818531-38 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: oid 38 updating version 0 to 168 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: oid 38 updating version 168 to 169 2026-02-21T22:20:52.500 INFO:tasks.rados.rados.0.trial178.stdout:1185: oid 38 updating version 169 to 171 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1185: oid 38 version 171 is already newer than 170 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 171 (ObjNum 432 snap 123 seq_num 432) dirty exists 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1185: left oid 38 (ObjNum 432 snap 123 seq_num 432) 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1186: finishing write tid 1 to trial17818531-12 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1185: done (4 left) 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1190: rollback oid 38 current snap is 123 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 105 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1186: finishing write tid 2 to trial17818531-12 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1191: rollback oid 7 current snap is 123 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 7 to 122 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1192: copy_from oid 19 from oid 11 current snap is 123 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1193: read oid 27 snap -1 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1193: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:20:52.501 INFO:tasks.rados.rados.0.trial178.stdout:1194: snap_create 2026-02-21T22:20:52.502 INFO:tasks.rados.rados.0.trial178.stdout:1190: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:20:52.502 INFO:tasks.rados.rados.0.trial178.stdout:1190: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:20:52.502 INFO:tasks.rados.rados.0.trial178.stdout:1190: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:20:52.502 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 174 (ObjNum 377 snap 105 seq_num 377) dirty exists 2026-02-21T22:20:52.504 INFO:tasks.rados.rados.0.trial178.stdout:1191: finishing rollback tid 0 to trial17818531-7 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: finishing write tid 3 to trial17818531-12 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: finishing write tid 4 to trial17818531-12 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: finishing write tid 5 to trial17818531-12 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: oid 12 updating version 0 to 352 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: oid 12 updating version 352 to 354 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: oid 12 version 354 is already newer than 353 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: oid 12 version 354 is already newer than 351 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 354 (ObjNum 433 snap 123 seq_num 433) dirty exists 2026-02-21T22:20:52.506 INFO:tasks.rados.rados.0.trial178.stdout:1186: left oid 12 (ObjNum 433 snap 123 seq_num 433) 2026-02-21T22:20:52.507 INFO:tasks.rados.rados.0.trial178.stdout:1187: expect (ObjNum 412 snap 115 seq_num 412) 2026-02-21T22:20:52.556 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:52.648 INFO:tasks.rados.rados.0.trial178.stdout:1188: finishing copy_from racing read to trial17818531-36 2026-02-21T22:20:52.648 INFO:tasks.rados.rados.0.trial178.stdout:1193: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:20:52.671 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1192: finishing copy_from racing read to trial17818531-19 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1189: finishing copy_from racing read to trial17818531-42 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1191: finishing rollback tid 1 to trial17818531-7 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1191: finishing rollback tid 2 to trial17818531-7 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 579 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1188: finishing copy_from to trial17818531-36 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 139 (ObjNum 373 snap 105 seq_num 373) dirty exists 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1189: finishing copy_from to trial17818531-42 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 178 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:1192: finishing copy_from to trial17818531-19 2026-02-21T22:20:52.718 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 526 (ObjNum 396 snap 111 seq_num 396) dirty exists 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1186: done (8 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1187: done (7 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1188: done (6 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1189: done (5 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1190: done (4 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1191: done (3 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1192: done (2 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1193: done (1 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1194: done (0 left) 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:1195: rollback oid 4 current snap is 124 2026-02-21T22:20:53.444 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 119 2026-02-21T22:20:53.445 INFO:tasks.rados.rados.0.trial178.stdout:1196: snap_remove snap 121 2026-02-21T22:20:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:20:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:20:53.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:53.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:1196: done (1 left) 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:1197: read oid 30 snap -1 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:1197: expect deleted 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:1198: snap_create 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:1195: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:20:54.445 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 350 (ObjNum 129 snap 0 seq_num 99852624) dirty dne 2026-02-21T22:20:55.445 INFO:tasks.rados.rados.0.trial178.stdout:1195: done (2 left) 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:1197: done (1 left) 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:1198: done (0 left) 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:1199: rollback oid 33 current snap is 125 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 105 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:1200: write oid 13 current snap is 125 2026-02-21T22:20:55.446 INFO:tasks.rados.rados.0.trial178.stdout:1200: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2026-02-21T22:20:55.452 INFO:tasks.rados.rados.0.trial178.stdout:1200: writing trial17818531-13 from 494238 to 984425 tid 1 2026-02-21T22:20:55.461 INFO:tasks.rados.rados.0.trial178.stdout:1200: writing trial17818531-13 from 1563538 to 2265357 tid 2 2026-02-21T22:20:55.468 INFO:tasks.rados.rados.0.trial178.stdout:1200: writing trial17818531-13 from 2818520 to 3294238 tid 3 2026-02-21T22:20:55.468 INFO:tasks.rados.rados.0.trial178.stdout:1201: read oid 10 snap 123 2026-02-21T22:20:55.468 INFO:tasks.rados.rados.0.trial178.stdout:1201: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1202: copy_from oid 16 from oid 6 current snap is 125 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1199: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1199: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1199: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 180 (ObjNum 373 snap 105 seq_num 373) dirty exists 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1200: finishing write tid 1 to trial17818531-13 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1200: finishing write tid 2 to trial17818531-13 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1199: done (3 left) 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1203: read oid 40 snap -1 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1203: expect (ObjNum 424 snap 117 seq_num 424) 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1204: copy_from oid 20 from oid 15 current snap is 125 2026-02-21T22:20:55.469 INFO:tasks.rados.rados.0.trial178.stdout:1205: snap_remove snap 110 2026-02-21T22:20:56.453 INFO:tasks.rados.rados.0.trial178.stdout:1205: done (5 left) 2026-02-21T22:20:56.453 INFO:tasks.rados.rados.0.trial178.stdout:1206: read oid 30 snap 122 2026-02-21T22:20:56.453 INFO:tasks.rados.rados.0.trial178.stdout:1206: expect deleted 2026-02-21T22:20:56.453 INFO:tasks.rados.rados.0.trial178.stdout:1207: snap_create 2026-02-21T22:20:56.454 INFO:tasks.rados.rados.0.trial178.stdout:1202: finishing copy_from to trial17818531-16 2026-02-21T22:20:56.454 INFO:tasks.rados.rados.0.trial178.stdout:1202: got expected ENOENT (src dne) 2026-02-21T22:20:56.454 INFO:tasks.rados.rados.0.trial178.stdout:1202: finishing copy_from racing read to trial17818531-16 2026-02-21T22:20:56.454 INFO:tasks.rados.rados.0.trial178.stdout:1203: expect (ObjNum 424 snap 117 seq_num 424) 2026-02-21T22:20:56.454 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:56.615 INFO:tasks.rados.rados.0.trial178.stdout:1200: finishing write tid 3 to trial17818531-13 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: finishing write tid 4 to trial17818531-13 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: finishing write tid 5 to trial17818531-13 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: oid 13 updating version 0 to 528 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: oid 13 updating version 528 to 531 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: oid 13 version 531 is already newer than 529 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: oid 13 version 531 is already newer than 530 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 531 (ObjNum 434 snap 125 seq_num 434) dirty exists 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1200: left oid 13 (ObjNum 434 snap 125 seq_num 434) 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:1201: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:20:56.616 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:20:56.746 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1862621 2026-02-21T22:20:56.777 INFO:tasks.rados.rados.0.trial178.stdout:1204: finishing copy_from racing read to trial17818531-20 2026-02-21T22:20:56.777 INFO:tasks.rados.rados.0.trial178.stdout:1204: finishing copy_from to trial17818531-20 2026-02-21T22:20:56.777 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 488 (ObjNum 428 snap 121 seq_num 428) dirty exists 2026-02-21T22:20:57.415 INFO:tasks.rados.rados.0.trial178.stdout:1200: done (6 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1201: done (5 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1202: done (4 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1203: done (3 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1204: done (2 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1206: done (1 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1207: done (0 left) 2026-02-21T22:20:57.416 INFO:tasks.rados.rados.0.trial178.stdout:1208: snap_create 2026-02-21T22:20:58.416 INFO:tasks.rados.rados.0.trial178.stdout:1208: done (0 left) 2026-02-21T22:20:58.416 INFO:tasks.rados.rados.0.trial178.stdout:1209: snap_create 2026-02-21T22:20:58.550 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:20:58.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:20:59.417 INFO:tasks.rados.rados.0.trial178.stdout:1209: done (0 left) 2026-02-21T22:20:59.417 INFO:tasks.rados.rados.0.trial178.stdout:1210: write oid 30 current snap is 128 2026-02-21T22:20:59.417 INFO:tasks.rados.rados.0.trial178.stdout:1210: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2026-02-21T22:20:59.425 INFO:tasks.rados.rados.0.trial178.stdout:1210: writing trial17818531-30 from 511045 to 1076481 tid 1 2026-02-21T22:20:59.431 INFO:tasks.rados.rados.0.trial178.stdout:1210: writing trial17818531-30 from 1622020 to 2067497 tid 2 2026-02-21T22:20:59.440 INFO:tasks.rados.rados.0.trial178.stdout:1210: writing trial17818531-30 from 2729590 to 3286445 tid 3 2026-02-21T22:20:59.440 INFO:tasks.rados.rados.0.trial178.stdout:1211: snap_remove snap 97 2026-02-21T22:21:00.422 INFO:tasks.rados.rados.0.trial178.stdout:1211: done (1 left) 2026-02-21T22:21:00.422 INFO:tasks.rados.rados.0.trial178.stdout:1212: snap_remove snap 107 2026-02-21T22:21:01.425 INFO:tasks.rados.rados.0.trial178.stdout:1212: done (1 left) 2026-02-21T22:21:01.425 INFO:tasks.rados.rados.0.trial178.stdout:1213: rollback oid 38 current snap is 128 2026-02-21T22:21:01.425 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 113 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: finishing write tid 1 to trial17818531-30 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: finishing write tid 2 to trial17818531-30 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: finishing write tid 3 to trial17818531-30 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: finishing write tid 4 to trial17818531-30 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: finishing write tid 5 to trial17818531-30 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: oid 30 updating version 0 to 337 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: oid 30 version 337 is already newer than 336 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: oid 30 version 337 is already newer than 334 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: oid 30 version 337 is already newer than 335 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 337 (ObjNum 435 snap 128 seq_num 435) dirty exists 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: left oid 30 (ObjNum 435 snap 128 seq_num 435) 2026-02-21T22:21:01.426 INFO:tasks.rados.rados.0.trial178.stdout:1210: done (1 left) 2026-02-21T22:21:01.427 INFO:tasks.rados.rados.0.trial178.stdout:1214: copy_from oid 4 from oid 47 current snap is 128 2026-02-21T22:21:01.427 INFO:tasks.rados.rados.0.trial178.stdout:1215: snap_remove snap 117 2026-02-21T22:21:02.416 INFO:tasks.rados.rados.0.trial178.stdout:1215: done (2 left) 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1216: read oid 3 snap -1 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1216: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1217: write oid 39 current snap is 128 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1214: finishing copy_from to trial17818531-4 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1214: got expected ENOENT (src dne) 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1213: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1214: finishing copy_from racing read to trial17818531-4 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1213: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1213: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 194 (ObjNum 401 snap 111 seq_num 401) dirty exists 2026-02-21T22:21:02.417 INFO:tasks.rados.rados.0.trial178.stdout:1217: seq_num 436 ranges {527852=640685,1680502=589135,2757013=570839} 2026-02-21T22:21:02.422 INFO:tasks.rados.rados.0.trial178.stdout:1217: writing trial17818531-39 from 527852 to 1168537 tid 1 2026-02-21T22:21:02.428 INFO:tasks.rados.rados.0.trial178.stdout:1217: writing trial17818531-39 from 1680502 to 2269637 tid 2 2026-02-21T22:21:02.435 INFO:tasks.rados.rados.0.trial178.stdout:1217: writing trial17818531-39 from 2757013 to 3327852 tid 3 2026-02-21T22:21:02.435 INFO:tasks.rados.rados.0.trial178.stdout:1213: done (3 left) 2026-02-21T22:21:02.435 INFO:tasks.rados.rados.0.trial178.stdout:1214: done (2 left) 2026-02-21T22:21:02.435 INFO:tasks.rados.rados.0.trial178.stdout:1218: copy_from oid 36 from oid 50 current snap is 128 2026-02-21T22:21:02.435 INFO:tasks.rados.rados.0.trial178.stdout:1216: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:02.478 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3073718 2026-02-21T22:21:02.507 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2395866 2026-02-21T22:21:02.565 INFO:tasks.rados.rados.0.trial178.stdout:1217: finishing write tid 1 to trial17818531-39 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: finishing write tid 2 to trial17818531-39 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1218: finishing copy_from to trial17818531-36 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1218: got expected ENOENT (src dne) 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1218: finishing copy_from racing read to trial17818531-36 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: finishing write tid 3 to trial17818531-39 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: finishing write tid 4 to trial17818531-39 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: finishing write tid 5 to trial17818531-39 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: oid 39 updating version 0 to 387 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: oid 39 updating version 387 to 388 2026-02-21T22:21:02.566 INFO:tasks.rados.rados.0.trial178.stdout:1217: oid 39 version 388 is already newer than 385 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1217: oid 39 version 388 is already newer than 386 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 388 (ObjNum 436 snap 128 seq_num 436) dirty exists 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1217: left oid 39 (ObjNum 436 snap 128 seq_num 436) 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1216: done (2 left) 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1217: done (1 left) 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1218: done (0 left) 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1219: read oid 21 snap -1 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1219: expect (ObjNum 402 snap 111 seq_num 402) 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1220: copy_from oid 17 from oid 35 current snap is 128 2026-02-21T22:21:02.567 INFO:tasks.rados.rados.0.trial178.stdout:1221: snap_remove snap 127 2026-02-21T22:21:03.430 INFO:tasks.rados.rados.0.trial178.stdout:1221: done (2 left) 2026-02-21T22:21:03.430 INFO:tasks.rados.rados.0.trial178.stdout:1222: write (excl) oid 25 current snap is 128 2026-02-21T22:21:03.430 INFO:tasks.rados.rados.0.trial178.stdout:1222: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2026-02-21T22:21:03.438 INFO:tasks.rados.rados.0.trial178.stdout:1222: writing trial17818531-25 from 544659 to 1260593 tid 1 2026-02-21T22:21:03.444 INFO:tasks.rados.rados.0.trial178.stdout:1222: writing trial17818531-25 from 1738984 to 2188130 tid 2 2026-02-21T22:21:03.452 INFO:tasks.rados.rados.0.trial178.stdout:1222: writing trial17818531-25 from 2784436 to 3344659 tid 3 2026-02-21T22:21:03.453 INFO:tasks.rados.rados.0.trial178.stdout:1223: write (excl) oid 41 current snap is 128 2026-02-21T22:21:03.453 INFO:tasks.rados.rados.0.trial178.stdout:1223: seq_num 438 ranges {561466=791183,2081113=592804,3095506=265960} 2026-02-21T22:21:03.461 INFO:tasks.rados.rados.0.trial178.stdout:1223: writing trial17818531-41 from 561466 to 1352649 tid 1 2026-02-21T22:21:03.471 INFO:tasks.rados.rados.0.trial178.stdout:1223: writing trial17818531-41 from 2081113 to 2673917 tid 2 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1223: writing trial17818531-41 from 3095506 to 3361466 tid 3 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1224: read oid 33 snap 125 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1224: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1225: read oid 39 snap 122 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1225: expect (ObjNum 407 snap 113 seq_num 407) 2026-02-21T22:21:03.475 INFO:tasks.rados.rados.0.trial178.stdout:1226: snap_remove snap 119 2026-02-21T22:21:03.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:03.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:04.433 INFO:tasks.rados.rados.0.trial178.stdout:1226: done (6 left) 2026-02-21T22:21:04.433 INFO:tasks.rados.rados.0.trial178.stdout:1227: snap_remove snap 124 2026-02-21T22:21:05.434 INFO:tasks.rados.rados.0.trial178.stdout:1227: done (6 left) 2026-02-21T22:21:05.434 INFO:tasks.rados.rados.0.trial178.stdout:1228: copy_from oid 19 from oid 10 current snap is 128 2026-02-21T22:21:05.434 INFO:tasks.rados.rados.0.trial178.stdout:1219: expect (ObjNum 402 snap 111 seq_num 402) 2026-02-21T22:21:05.434 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:05.497 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2579519 2026-02-21T22:21:05.502 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 797014 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1220: finishing copy_from racing read to trial17818531-17 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1220: finishing copy_from to trial17818531-17 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 36 (ObjNum 104 snap 25 seq_num 104) dirty exists 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1222: finishing write tid 1 to trial17818531-25 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1222: finishing write tid 2 to trial17818531-25 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1222: finishing write tid 3 to trial17818531-25 2026-02-21T22:21:05.522 INFO:tasks.rados.rados.0.trial178.stdout:1222: finishing write tid 4 to trial17818531-25 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: finishing write tid 5 to trial17818531-25 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: oid 25 updating version 0 to 341 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: oid 25 version 341 is already newer than 340 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: oid 25 version 341 is already newer than 338 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: oid 25 version 341 is already newer than 339 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 341 (ObjNum 437 snap 128 seq_num 437) dirty exists 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1222: left oid 25 (ObjNum 437 snap 128 seq_num 437) 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1223: finishing write tid 1 to trial17818531-41 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1223: finishing write tid 2 to trial17818531-41 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1223: finishing write tid 3 to trial17818531-41 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1219: done (6 left) 2026-02-21T22:21:05.523 INFO:tasks.rados.rados.0.trial178.stdout:1220: done (5 left) 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1222: done (4 left) 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1229: copy_from oid 32 from oid 15 current snap is 128 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: finishing write tid 4 to trial17818531-41 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: finishing write tid 5 to trial17818531-41 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: oid 41 updating version 0 to 490 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: oid 41 updating version 490 to 492 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: oid 41 version 492 is already newer than 489 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: oid 41 version 492 is already newer than 491 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 492 (ObjNum 438 snap 128 seq_num 438) dirty exists 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1223: left oid 41 (ObjNum 438 snap 128 seq_num 438) 2026-02-21T22:21:05.524 INFO:tasks.rados.rados.0.trial178.stdout:1224: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:21:05.525 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1374919 2026-02-21T22:21:05.567 INFO:tasks.rados.rados.0.trial178.stdout:1225: expect (ObjNum 407 snap 113 seq_num 407) 2026-02-21T22:21:05.571 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 963068 2026-02-21T22:21:05.605 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:05.627 INFO:tasks.rados.rados.0.trial178.stdout:1228: finishing copy_from racing read to trial17818531-19 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1228: finishing copy_from to trial17818531-19 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 481 (ObjNum 324 snap 95 seq_num 324) dirty exists 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1229: finishing copy_from racing read to trial17818531-32 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1229: finishing copy_from to trial17818531-32 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 488 (ObjNum 428 snap 121 seq_num 428) dirty exists 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1223: done (4 left) 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1224: done (3 left) 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1225: done (2 left) 2026-02-21T22:21:05.628 INFO:tasks.rados.rados.0.trial178.stdout:1228: done (1 left) 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1229: done (0 left) 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1230: read oid 9 snap 123 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1230: expect (ObjNum 385 snap 108 seq_num 385) 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1231: read oid 24 snap -1 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1231: expect (ObjNum 380 snap 105 seq_num 380) 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1232: read oid 2 snap -1 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1232: expect (ObjNum 425 snap 118 seq_num 425) 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1233: write oid 20 current snap is 128 2026-02-21T22:21:05.629 INFO:tasks.rados.rados.0.trial178.stdout:1233: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2026-02-21T22:21:05.637 INFO:tasks.rados.rados.0.trial178.stdout:1233: writing trial17818531-20 from 578273 to 1044705 tid 1 2026-02-21T22:21:05.645 INFO:tasks.rados.rados.0.trial178.stdout:1233: writing trial17818531-20 from 1739595 to 2192410 tid 2 2026-02-21T22:21:05.654 INFO:tasks.rados.rados.0.trial178.stdout:1233: writing trial17818531-20 from 2722929 to 3378273 tid 3 2026-02-21T22:21:05.654 INFO:tasks.rados.rados.0.trial178.stdout:1234: copy_from oid 14 from oid 4 current snap is 128 2026-02-21T22:21:05.654 INFO:tasks.rados.rados.0.trial178.stdout:1230: expect (ObjNum 385 snap 108 seq_num 385) 2026-02-21T22:21:05.786 INFO:tasks.rados.rados.0.trial178.stdout:1232: expect (ObjNum 425 snap 118 seq_num 425) 2026-02-21T22:21:05.786 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:05.810 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:05.853 INFO:tasks.rados.rados.0.trial178.stdout:1231: expect (ObjNum 380 snap 105 seq_num 380) 2026-02-21T22:21:05.876 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1896930 2026-02-21T22:21:05.886 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1233: finishing write tid 1 to trial17818531-20 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1233: finishing write tid 2 to trial17818531-20 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1234: finishing copy_from to trial17818531-14 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1234: got expected ENOENT (src dne) 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1234: finishing copy_from racing read to trial17818531-14 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1233: finishing write tid 3 to trial17818531-20 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1233: finishing write tid 4 to trial17818531-20 2026-02-21T22:21:05.964 INFO:tasks.rados.rados.0.trial178.stdout:1233: finishing write tid 5 to trial17818531-20 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: oid 20 updating version 0 to 494 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: oid 20 version 494 is already newer than 493 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: oid 20 updating version 494 to 495 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: oid 20 updating version 495 to 496 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 496 (ObjNum 439 snap 128 seq_num 439) dirty exists 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: left oid 20 (ObjNum 439 snap 128 seq_num 439) 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1230: done (4 left) 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1231: done (3 left) 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1232: done (2 left) 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1233: done (1 left) 2026-02-21T22:21:05.965 INFO:tasks.rados.rados.0.trial178.stdout:1234: done (0 left) 2026-02-21T22:21:05.966 INFO:tasks.rados.rados.0.trial178.stdout:1235: snap_remove snap 113 2026-02-21T22:21:06.440 INFO:tasks.rados.rados.0.trial178.stdout:1235: done (0 left) 2026-02-21T22:21:06.441 INFO:tasks.rados.rados.0.trial178.stdout:1236: copy_from oid 14 from oid 3 current snap is 128 2026-02-21T22:21:06.441 INFO:tasks.rados.rados.0.trial178.stdout:1237: write oid 35 current snap is 128 2026-02-21T22:21:06.441 INFO:tasks.rados.rados.0.trial178.stdout:1237: seq_num 440 ranges {595080=541681,1798077=596473,3150352=244728} 2026-02-21T22:21:06.448 INFO:tasks.rados.rados.0.trial178.stdout:1237: writing trial17818531-35 from 595080 to 1136761 tid 1 2026-02-21T22:21:06.454 INFO:tasks.rados.rados.0.trial178.stdout:1237: writing trial17818531-35 from 1798077 to 2394550 tid 2 2026-02-21T22:21:06.460 INFO:tasks.rados.rados.0.trial178.stdout:1237: writing trial17818531-35 from 3150352 to 3395080 tid 3 2026-02-21T22:21:06.460 INFO:tasks.rados.rados.0.trial178.stdout:1238: write oid 34 current snap is 128 2026-02-21T22:21:06.461 INFO:tasks.rados.rados.0.trial178.stdout:1238: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2026-02-21T22:21:06.469 INFO:tasks.rados.rados.0.trial178.stdout:1238: writing trial17818531-34 from 611887 to 1345170 tid 1 2026-02-21T22:21:06.477 INFO:tasks.rados.rados.0.trial178.stdout:1238: writing trial17818531-34 from 1972912 to 2429396 tid 2 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1238: writing trial17818531-34 from 2894128 to 3411887 tid 3 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1239: delete oid 37 current snap is 128 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1237: finishing write tid 1 to trial17818531-35 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1236: finishing copy_from racing read to trial17818531-14 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1237: finishing write tid 2 to trial17818531-35 2026-02-21T22:21:06.485 INFO:tasks.rados.rados.0.trial178.stdout:1236: finishing copy_from to trial17818531-14 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 466 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: finishing write tid 3 to trial17818531-35 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: finishing write tid 4 to trial17818531-35 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: finishing write tid 5 to trial17818531-35 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: oid 35 updating version 0 to 568 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: oid 35 version 568 is already newer than 566 2026-02-21T22:21:06.486 INFO:tasks.rados.rados.0.trial178.stdout:1237: oid 35 version 568 is already newer than 563 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:1237: oid 35 version 568 is already newer than 567 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 568 (ObjNum 440 snap 128 seq_num 440) dirty exists 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:1237: left oid 35 (ObjNum 440 snap 128 seq_num 440) 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:1238: finishing write tid 1 to trial17818531-34 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:1238: finishing write tid 2 to trial17818531-34 2026-02-21T22:21:06.487 INFO:tasks.rados.rados.0.trial178.stdout:1236: done (3 left) 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1237: done (2 left) 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1239: done (1 left) 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1240: copy_from oid 22 from oid 23 current snap is 128 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1241: read oid 2 snap -1 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1241: expect (ObjNum 425 snap 118 seq_num 425) 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1242: write oid 9 current snap is 128 2026-02-21T22:21:06.488 INFO:tasks.rados.rados.0.trial178.stdout:1242: seq_num 443 ranges {645501=483781,1573523=460153,2548974=763384} 2026-02-21T22:21:06.489 INFO:tasks.rados.rados.0.trial178.stdout:1242: writing trial17818531-9 from 645501 to 1129282 tid 1 2026-02-21T22:21:06.492 INFO:tasks.rados.rados.0.trial178.stdout:1242: writing trial17818531-9 from 1573523 to 2033676 tid 2 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1242: writing trial17818531-9 from 2548974 to 3312358 tid 3 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1243: rollback oid 38 current snap is 128 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1238: finishing write tid 3 to trial17818531-34 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1238: finishing write tid 4 to trial17818531-34 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1238: finishing write tid 5 to trial17818531-34 2026-02-21T22:21:06.498 INFO:tasks.rados.rados.0.trial178.stdout:1238: oid 34 updating version 0 to 382 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:1238: oid 34 version 382 is already newer than 381 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:1238: oid 34 updating version 382 to 384 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:1238: oid 34 version 384 is already newer than 383 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 384 (ObjNum 441 snap 128 seq_num 441) dirty exists 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:1238: left oid 34 (ObjNum 441 snap 128 seq_num 441) 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:1241: expect (ObjNum 425 snap 118 seq_num 425) 2026-02-21T22:21:06.499 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1242: finishing write tid 1 to trial17818531-9 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1240: finishing copy_from racing read to trial17818531-22 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1242: finishing write tid 2 to trial17818531-9 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1240: finishing copy_from to trial17818531-22 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 415 (ObjNum 394 snap 111 seq_num 394) dirty exists 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1242: finishing write tid 3 to trial17818531-9 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1242: finishing write tid 4 to trial17818531-9 2026-02-21T22:21:06.567 INFO:tasks.rados.rados.0.trial178.stdout:1242: finishing write tid 5 to trial17818531-9 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: oid 9 updating version 0 to 469 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: oid 9 version 469 is already newer than 467 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: oid 9 updating version 469 to 470 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: oid 9 version 470 is already newer than 468 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 470 (ObjNum 443 snap 128 seq_num 443) dirty exists 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: left oid 9 (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 114 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1238: done (4 left) 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1240: done (3 left) 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1241: done (2 left) 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1242: done (1 left) 2026-02-21T22:21:06.568 INFO:tasks.rados.rados.0.trial178.stdout:1244: snap_remove snap 105 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1244: done (1 left) 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1245: read oid 36 snap -1 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1245: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1246: read oid 3 snap -1 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1246: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1247: write oid 23 current snap is 128 2026-02-21T22:21:07.419 INFO:tasks.rados.rados.0.trial178.stdout:1243: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:21:07.420 INFO:tasks.rados.rados.0.trial178.stdout:1243: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:21:07.420 INFO:tasks.rados.rados.0.trial178.stdout:1243: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:21:07.420 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 215 (ObjNum 401 snap 111 seq_num 401) dirty exists 2026-02-21T22:21:07.420 INFO:tasks.rados.rados.0.trial178.stdout:1247: seq_num 444 ranges {662308=559030,1632005=603811,2860044=602264} 2026-02-21T22:21:07.428 INFO:tasks.rados.rados.0.trial178.stdout:1247: writing trial17818531-23 from 662308 to 1221338 tid 1 2026-02-21T22:21:07.431 INFO:tasks.rados.rados.0.trial178.stdout:1247: writing trial17818531-23 from 1632005 to 2235816 tid 2 2026-02-21T22:21:07.438 INFO:tasks.rados.rados.0.trial178.stdout:1247: writing trial17818531-23 from 2860044 to 3462308 tid 3 2026-02-21T22:21:07.439 INFO:tasks.rados.rados.0.trial178.stdout:1243: done (3 left) 2026-02-21T22:21:07.439 INFO:tasks.rados.rados.0.trial178.stdout:1248: snap_remove snap 125 2026-02-21T22:21:08.423 INFO:tasks.rados.rados.0.trial178.stdout:1248: done (3 left) 2026-02-21T22:21:08.423 INFO:tasks.rados.rados.0.trial178.stdout:1249: copy_from oid 49 from oid 40 current snap is 128 2026-02-21T22:21:08.423 INFO:tasks.rados.rados.0.trial178.stdout:1246: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:08.468 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3278785 2026-02-21T22:21:08.533 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2116150 2026-02-21T22:21:08.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:08.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:08.552 INFO:tasks.rados.rados.0.trial178.stdout:1245: expect (ObjNum 373 snap 105 seq_num 373) 2026-02-21T22:21:08.581 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: finishing write tid 1 to trial17818531-23 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: finishing write tid 2 to trial17818531-23 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: finishing write tid 3 to trial17818531-23 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: finishing write tid 4 to trial17818531-23 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: finishing write tid 5 to trial17818531-23 2026-02-21T22:21:08.599 INFO:tasks.rados.rados.0.trial178.stdout:1247: oid 23 updating version 0 to 424 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1247: oid 23 updating version 424 to 426 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1247: oid 23 version 426 is already newer than 423 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1247: oid 23 version 426 is already newer than 425 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 426 (ObjNum 444 snap 128 seq_num 444) dirty exists 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1247: left oid 23 (ObjNum 444 snap 128 seq_num 444) 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1249: finishing copy_from racing read to trial17818531-49 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1249: finishing copy_from to trial17818531-49 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 492 (ObjNum 424 snap 117 seq_num 424) dirty exists 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1245: done (3 left) 2026-02-21T22:21:08.600 INFO:tasks.rados.rados.0.trial178.stdout:1246: done (2 left) 2026-02-21T22:21:08.601 INFO:tasks.rados.rados.0.trial178.stdout:1247: done (1 left) 2026-02-21T22:21:08.601 INFO:tasks.rados.rados.0.trial178.stdout:1249: done (0 left) 2026-02-21T22:21:08.601 INFO:tasks.rados.rados.0.trial178.stdout:1250: read oid 24 snap 118 2026-02-21T22:21:08.601 INFO:tasks.rados.rados.0.trial178.stdout:1250: expect (ObjNum 380 snap 105 seq_num 380) 2026-02-21T22:21:08.601 INFO:tasks.rados.rados.0.trial178.stdout:1251: delete oid 38 current snap is 128 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1251: done (1 left) 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1252: rollback oid 19 current snap is 128 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 122 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1253: rollback oid 47 current snap is 128 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 118 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1254: read oid 3 snap -1 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1254: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:08.602 INFO:tasks.rados.rados.0.trial178.stdout:1255: snap_remove snap 126 2026-02-21T22:21:09.448 INFO:tasks.rados.rados.0.trial178.stdout:1255: done (4 left) 2026-02-21T22:21:09.448 INFO:tasks.rados.rados.0.trial178.stdout:1256: snap_remove snap 114 2026-02-21T22:21:10.451 INFO:tasks.rados.rados.0.trial178.stdout:1256: done (4 left) 2026-02-21T22:21:10.451 INFO:tasks.rados.rados.0.trial178.stdout:1257: snap_create 2026-02-21T22:21:10.451 INFO:tasks.rados.rados.0.trial178.stdout:1250: expect (ObjNum 380 snap 105 seq_num 380) 2026-02-21T22:21:10.537 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:10.575 INFO:tasks.rados.rados.0.trial178.stdout:1252: finishing rollback tid 0 to trial17818531-19 2026-02-21T22:21:10.575 INFO:tasks.rados.rados.0.trial178.stdout:1252: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:21:10.575 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 648 (ObjNum 129 snap 0 seq_num 99852624) dirty dne 2026-02-21T22:21:10.575 INFO:tasks.rados.rados.0.trial178.stdout:1254: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:10.686 INFO:tasks.rados.rados.0.trial178.stdout:1253: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:21:10.686 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 558 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1250: done (4 left) 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1252: done (3 left) 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1253: done (2 left) 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1254: done (1 left) 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1257: done (0 left) 2026-02-21T22:21:11.451 INFO:tasks.rados.rados.0.trial178.stdout:1258: snap_create 2026-02-21T22:21:12.418 INFO:tasks.rados.rados.0.trial178.stdout:1258: done (0 left) 2026-02-21T22:21:12.418 INFO:tasks.rados.rados.0.trial178.stdout:1259: delete oid 43 current snap is 130 2026-02-21T22:21:12.421 INFO:tasks.rados.rados.0.trial178.stdout:1259: done (0 left) 2026-02-21T22:21:12.422 INFO:tasks.rados.rados.0.trial178.stdout:1260: snap_remove snap 118 2026-02-21T22:21:13.419 INFO:tasks.rados.rados.0.trial178.stdout:1260: done (0 left) 2026-02-21T22:21:13.420 INFO:tasks.rados.rados.0.trial178.stdout:1261: copy_from oid 43 from oid 18 current snap is 130 2026-02-21T22:21:13.420 INFO:tasks.rados.rados.0.trial178.stdout:1262: write (excl) oid 35 current snap is 130 2026-02-21T22:21:13.420 INFO:tasks.rados.rados.0.trial178.stdout:1262: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2026-02-21T22:21:13.429 INFO:tasks.rados.rados.0.trial178.stdout:1262: writing trial17818531-35 from 712729 to 1497506 tid 1 2026-02-21T22:21:13.436 INFO:tasks.rados.rados.0.trial178.stdout:1262: writing trial17818531-35 from 2091098 to 2558589 tid 2 2026-02-21T22:21:13.443 INFO:tasks.rados.rados.0.trial178.stdout:1262: writing trial17818531-35 from 3342313 to 3512729 tid 3 2026-02-21T22:21:13.443 INFO:tasks.rados.rados.0.trial178.stdout:1263: write oid 34 current snap is 130 2026-02-21T22:21:13.444 INFO:tasks.rados.rados.0.trial178.stdout:1263: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2026-02-21T22:21:13.451 INFO:tasks.rados.rados.0.trial178.stdout:1263: writing trial17818531-34 from 729536 to 1189562 tid 1 2026-02-21T22:21:13.459 INFO:tasks.rados.rados.0.trial178.stdout:1263: writing trial17818531-34 from 1749580 to 2360729 tid 2 2026-02-21T22:21:13.466 INFO:tasks.rados.rados.0.trial178.stdout:1263: writing trial17818531-34 from 2853383 to 3305833 tid 3 2026-02-21T22:21:13.466 INFO:tasks.rados.rados.0.trial178.stdout:1264: snap_create 2026-02-21T22:21:13.466 INFO:tasks.rados.rados.0.trial178.stdout:1261: finishing copy_from racing read to trial17818531-43 2026-02-21T22:21:13.466 INFO:tasks.rados.rados.0.trial178.stdout:1261: finishing copy_from to trial17818531-43 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 333 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: finishing write tid 1 to trial17818531-35 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: finishing write tid 2 to trial17818531-35 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: finishing write tid 3 to trial17818531-35 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: finishing write tid 4 to trial17818531-35 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: finishing write tid 5 to trial17818531-35 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: oid 35 updating version 0 to 603 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: oid 35 version 603 is already newer than 594 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: oid 35 updating version 603 to 604 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: oid 35 updating version 604 to 605 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 605 (ObjNum 447 snap 130 seq_num 447) dirty exists 2026-02-21T22:21:13.467 INFO:tasks.rados.rados.0.trial178.stdout:1262: left oid 35 (ObjNum 447 snap 130 seq_num 447) 2026-02-21T22:21:13.468 INFO:tasks.rados.rados.0.trial178.stdout:1263: finishing write tid 1 to trial17818531-34 2026-02-21T22:21:13.468 INFO:tasks.rados.rados.0.trial178.stdout:1263: finishing write tid 2 to trial17818531-34 2026-02-21T22:21:13.468 INFO:tasks.rados.rados.0.trial178.stdout:1263: finishing write tid 3 to trial17818531-34 2026-02-21T22:21:13.468 INFO:tasks.rados.rados.0.trial178.stdout:1263: finishing write tid 4 to trial17818531-34 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: finishing write tid 5 to trial17818531-34 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: oid 34 updating version 0 to 387 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: oid 34 updating version 387 to 388 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: oid 34 version 388 is already newer than 386 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: oid 34 version 388 is already newer than 385 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 388 (ObjNum 448 snap 130 seq_num 448) dirty exists 2026-02-21T22:21:13.469 INFO:tasks.rados.rados.0.trial178.stdout:1263: left oid 34 (ObjNum 448 snap 130 seq_num 448) 2026-02-21T22:21:13.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:13.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:14.423 INFO:tasks.rados.rados.0.trial178.stdout:1261: done (3 left) 2026-02-21T22:21:14.423 INFO:tasks.rados.rados.0.trial178.stdout:1262: done (2 left) 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:1263: done (1 left) 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:1264: done (0 left) 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:1265: rollback oid 6 current snap is 131 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 111 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:1266: write oid 1 current snap is 131 2026-02-21T22:21:14.424 INFO:tasks.rados.rados.0.trial178.stdout:1266: seq_num 449 ranges {746343=651628,1924415=754807,3397159=149184} 2026-02-21T22:21:14.429 INFO:tasks.rados.rados.0.trial178.stdout:1266: writing trial17818531-1 from 746343 to 1397971 tid 1 2026-02-21T22:21:14.434 INFO:tasks.rados.rados.0.trial178.stdout:1266: writing trial17818531-1 from 1924415 to 2679222 tid 2 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:1266: writing trial17818531-1 from 3397159 to 3546343 tid 3 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:1267: read oid 9 snap -1 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:1267: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:1268: rollback oid 49 current snap is 131 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 128 2026-02-21T22:21:14.437 INFO:tasks.rados.rados.0.trial178.stdout:1265: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 493 (ObjNum 129 snap 0 seq_num 99554544) dirty dne 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1266: finishing write tid 1 to trial17818531-1 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1266: finishing write tid 2 to trial17818531-1 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1265: done (3 left) 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1269: read oid 31 snap -1 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1269: expect deleted 2026-02-21T22:21:14.438 INFO:tasks.rados.rados.0.trial178.stdout:1270: delete oid 37 current snap is 131 2026-02-21T22:21:14.440 INFO:tasks.rados.rados.0.trial178.stdout:1268: finishing rollback tid 0 to trial17818531-49 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: finishing write tid 3 to trial17818531-1 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: finishing write tid 4 to trial17818531-1 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1268: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1268: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 495 (ObjNum 424 snap 117 seq_num 424) dirty exists 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: finishing write tid 5 to trial17818531-1 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: oid 1 updating version 0 to 412 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: oid 1 updating version 412 to 413 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: oid 1 version 413 is already newer than 410 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: oid 1 version 413 is already newer than 411 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 413 (ObjNum 449 snap 131 seq_num 449) dirty exists 2026-02-21T22:21:14.441 INFO:tasks.rados.rados.0.trial178.stdout:1266: left oid 1 (ObjNum 449 snap 131 seq_num 449) 2026-02-21T22:21:14.442 INFO:tasks.rados.rados.0.trial178.stdout:1266: done (4 left) 2026-02-21T22:21:14.442 INFO:tasks.rados.rados.0.trial178.stdout:1268: done (3 left) 2026-02-21T22:21:14.442 INFO:tasks.rados.rados.0.trial178.stdout:1269: done (2 left) 2026-02-21T22:21:14.442 INFO:tasks.rados.rados.0.trial178.stdout:1270: done (1 left) 2026-02-21T22:21:14.442 INFO:tasks.rados.rados.0.trial178.stdout:1271: snap_create 2026-02-21T22:21:14.444 INFO:tasks.rados.rados.0.trial178.stdout:1267: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:15.425 INFO:tasks.rados.rados.0.trial178.stdout:1267: done (1 left) 2026-02-21T22:21:15.425 INFO:tasks.rados.rados.0.trial178.stdout:1271: done (0 left) 2026-02-21T22:21:15.425 INFO:tasks.rados.rados.0.trial178.stdout:1272: copy_from oid 18 from oid 11 current snap is 132 2026-02-21T22:21:15.426 INFO:tasks.rados.rados.0.trial178.stdout:1273: write oid 30 current snap is 132 2026-02-21T22:21:15.426 INFO:tasks.rados.rados.0.trial178.stdout:1273: seq_num 451 ranges {779957=402126,1925026=758476,3335652=244305} 2026-02-21T22:21:15.433 INFO:tasks.rados.rados.0.trial178.stdout:1273: writing trial17818531-30 from 779957 to 1182083 tid 1 2026-02-21T22:21:15.443 INFO:tasks.rados.rados.0.trial178.stdout:1273: writing trial17818531-30 from 1925026 to 2683502 tid 2 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1273: writing trial17818531-30 from 3335652 to 3579957 tid 3 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1274: snap_remove snap 130 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1272: finishing copy_from racing read to trial17818531-18 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1272: finishing copy_from to trial17818531-18 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 526 (ObjNum 396 snap 111 seq_num 396) dirty exists 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1273: finishing write tid 1 to trial17818531-30 2026-02-21T22:21:15.450 INFO:tasks.rados.rados.0.trial178.stdout:1273: finishing write tid 2 to trial17818531-30 2026-02-21T22:21:16.430 INFO:tasks.rados.rados.0.trial178.stdout:1272: done (2 left) 2026-02-21T22:21:16.431 INFO:tasks.rados.rados.0.trial178.stdout:1274: done (1 left) 2026-02-21T22:21:16.431 INFO:tasks.rados.rados.0.trial178.stdout:1275: write (excl) oid 33 current snap is 132 2026-02-21T22:21:16.431 INFO:tasks.rados.rados.0.trial178.stdout:1275: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2026-02-21T22:21:16.440 INFO:tasks.rados.rados.0.trial178.stdout:1275: writing trial17818531-33 from 796764 to 1274139 tid 1 2026-02-21T22:21:16.449 INFO:tasks.rados.rados.0.trial178.stdout:1275: writing trial17818531-33 from 1983508 to 2601995 tid 2 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1275: writing trial17818531-33 from 3363075 to 3596764 tid 3 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1276: write oid 23 current snap is 132 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1273: finishing write tid 3 to trial17818531-30 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1273: finishing write tid 4 to trial17818531-30 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1273: finishing write tid 5 to trial17818531-30 2026-02-21T22:21:16.456 INFO:tasks.rados.rados.0.trial178.stdout:1273: oid 30 updating version 0 to 527 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1273: oid 30 updating version 527 to 530 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1273: oid 30 version 530 is already newer than 529 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1273: oid 30 version 530 is already newer than 528 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 530 (ObjNum 451 snap 132 seq_num 451) dirty exists 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1273: left oid 30 (ObjNum 451 snap 132 seq_num 451) 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1275: finishing write tid 1 to trial17818531-33 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1275: finishing write tid 2 to trial17818531-33 2026-02-21T22:21:16.457 INFO:tasks.rados.rados.0.trial178.stdout:1276: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2026-02-21T22:21:16.462 INFO:tasks.rados.rados.0.trial178.stdout:1276: writing trial17818531-23 from 413571 to 966195 tid 1 2026-02-21T22:21:16.469 INFO:tasks.rados.rados.0.trial178.stdout:1276: writing trial17818531-23 from 1641990 to 2404135 tid 2 2026-02-21T22:21:16.475 INFO:tasks.rados.rados.0.trial178.stdout:1276: writing trial17818531-23 from 2990498 to 3415661 tid 3 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1273: done (2 left) 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1277: write (excl) oid 13 current snap is 132 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: finishing write tid 3 to trial17818531-33 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: finishing write tid 4 to trial17818531-33 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: finishing write tid 5 to trial17818531-33 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: oid 33 updating version 0 to 235 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: oid 33 version 235 is already newer than 234 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: oid 33 updating version 235 to 236 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:1275: oid 33 updating version 236 to 237 2026-02-21T22:21:16.476 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 237 (ObjNum 452 snap 132 seq_num 452) dirty exists 2026-02-21T22:21:16.477 INFO:tasks.rados.rados.0.trial178.stdout:1275: left oid 33 (ObjNum 452 snap 132 seq_num 452) 2026-02-21T22:21:16.477 INFO:tasks.rados.rados.0.trial178.stdout:1276: finishing write tid 1 to trial17818531-23 2026-02-21T22:21:16.477 INFO:tasks.rados.rados.0.trial178.stdout:1276: finishing write tid 2 to trial17818531-23 2026-02-21T22:21:16.477 INFO:tasks.rados.rados.0.trial178.stdout:1277: seq_num 454 ranges {430378=627873,1700472=622156,3017921=612457} 2026-02-21T22:21:16.482 INFO:tasks.rados.rados.0.trial178.stdout:1277: writing trial17818531-13 from 430378 to 1058251 tid 1 2026-02-21T22:21:16.487 INFO:tasks.rados.rados.0.trial178.stdout:1277: writing trial17818531-13 from 1700472 to 2322628 tid 2 2026-02-21T22:21:16.492 INFO:tasks.rados.rados.0.trial178.stdout:1277: writing trial17818531-13 from 3017921 to 3630378 tid 3 2026-02-21T22:21:16.492 INFO:tasks.rados.rados.0.trial178.stdout:1275: done (2 left) 2026-02-21T22:21:16.492 INFO:tasks.rados.rados.0.trial178.stdout:1278: snap_remove snap 128 2026-02-21T22:21:17.420 INFO:tasks.rados.rados.0.trial178.stdout:1278: done (2 left) 2026-02-21T22:21:17.420 INFO:tasks.rados.rados.0.trial178.stdout:1279: read oid 42 snap -1 2026-02-21T22:21:17.420 INFO:tasks.rados.rados.0.trial178.stdout:1279: expect (ObjNum 398 snap 111 seq_num 398) 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1280: delete oid 11 current snap is 132 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: finishing write tid 3 to trial17818531-23 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: finishing write tid 4 to trial17818531-23 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: finishing write tid 5 to trial17818531-23 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: oid 23 updating version 0 to 428 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: oid 23 version 428 is already newer than 427 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: oid 23 updating version 428 to 430 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: oid 23 version 430 is already newer than 429 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 430 (ObjNum 453 snap 132 seq_num 453) dirty exists 2026-02-21T22:21:17.421 INFO:tasks.rados.rados.0.trial178.stdout:1276: left oid 23 (ObjNum 453 snap 132 seq_num 453) 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: finishing write tid 1 to trial17818531-13 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: finishing write tid 2 to trial17818531-13 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: finishing write tid 3 to trial17818531-13 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: finishing write tid 4 to trial17818531-13 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: finishing write tid 5 to trial17818531-13 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: oid 13 updating version 0 to 608 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: oid 13 version 608 is already newer than 607 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: oid 13 updating version 608 to 609 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: oid 13 updating version 609 to 610 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 610 (ObjNum 454 snap 132 seq_num 454) dirty exists 2026-02-21T22:21:17.422 INFO:tasks.rados.rados.0.trial178.stdout:1277: left oid 13 (ObjNum 454 snap 132 seq_num 454) 2026-02-21T22:21:17.425 INFO:tasks.rados.rados.0.trial178.stdout:1276: done (3 left) 2026-02-21T22:21:17.425 INFO:tasks.rados.rados.0.trial178.stdout:1277: done (2 left) 2026-02-21T22:21:17.425 INFO:tasks.rados.rados.0.trial178.stdout:1280: done (1 left) 2026-02-21T22:21:17.425 INFO:tasks.rados.rados.0.trial178.stdout:1281: delete oid 24 current snap is 132 2026-02-21T22:21:17.429 INFO:tasks.rados.rados.0.trial178.stdout:1279: expect (ObjNum 398 snap 111 seq_num 398) 2026-02-21T22:21:17.456 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2034352 2026-02-21T22:21:17.467 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 669039 2026-02-21T22:21:17.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:17.516 INFO:tasks.rados.rados.0.trial178.stdout:1279: done (1 left) 2026-02-21T22:21:17.516 INFO:tasks.rados.rados.0.trial178.stdout:1281: done (0 left) 2026-02-21T22:21:17.517 INFO:tasks.rados.rados.0.trial178.stdout:1282: read oid 10 snap 111 2026-02-21T22:21:17.517 INFO:tasks.rados.rados.0.trial178.stdout:1282: expect (ObjNum 299 snap 88 seq_num 299) 2026-02-21T22:21:17.517 INFO:tasks.rados.rados.0.trial178.stdout:1283: write (excl) oid 7 current snap is 132 2026-02-21T22:21:17.517 INFO:tasks.rados.rados.0.trial178.stdout:1283: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2026-02-21T22:21:17.523 INFO:tasks.rados.rados.0.trial178.stdout:1283: writing trial17818531-7 from 480799 to 1050772 tid 1 2026-02-21T22:21:17.532 INFO:tasks.rados.rados.0.trial178.stdout:1283: writing trial17818531-7 from 1475918 to 2245401 tid 2 2026-02-21T22:21:17.539 INFO:tasks.rados.rados.0.trial178.stdout:1283: writing trial17818531-7 from 2700190 to 3286794 tid 3 2026-02-21T22:21:17.539 INFO:tasks.rados.rados.0.trial178.stdout:1284: write oid 15 current snap is 132 2026-02-21T22:21:17.539 INFO:tasks.rados.rados.0.trial178.stdout:1284: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2026-02-21T22:21:17.547 INFO:tasks.rados.rados.0.trial178.stdout:1284: writing trial17818531-15 from 497606 to 1142828 tid 1 2026-02-21T22:21:17.557 INFO:tasks.rados.rados.0.trial178.stdout:1284: writing trial17818531-15 from 1934400 to 2563894 tid 2 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1284: writing trial17818531-15 from 3243966 to 3697606 tid 3 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1285: read oid 41 snap -1 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1285: expect (ObjNum 438 snap 128 seq_num 438) 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1286: read oid 12 snap 131 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1286: expect (ObjNum 433 snap 123 seq_num 433) 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1287: copy_from oid 47 from oid 42 current snap is 132 2026-02-21T22:21:17.564 INFO:tasks.rados.rados.0.trial178.stdout:1282: expect (ObjNum 299 snap 88 seq_num 299) 2026-02-21T22:21:17.605 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:17.630 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:17.654 INFO:tasks.rados.rados.0.trial178.stdout:1283: finishing write tid 1 to trial17818531-7 2026-02-21T22:21:17.654 INFO:tasks.rados.rados.0.trial178.stdout:1283: finishing write tid 2 to trial17818531-7 2026-02-21T22:21:17.654 INFO:tasks.rados.rados.0.trial178.stdout:1283: finishing write tid 3 to trial17818531-7 2026-02-21T22:21:17.654 INFO:tasks.rados.rados.0.trial178.stdout:1283: finishing write tid 4 to trial17818531-7 2026-02-21T22:21:17.654 INFO:tasks.rados.rados.0.trial178.stdout:1282: done (5 left) 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1288: snap_create 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: finishing write tid 5 to trial17818531-7 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: oid 7 updating version 0 to 668 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: oid 7 version 668 is already newer than 666 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: oid 7 version 668 is already newer than 665 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: oid 7 version 668 is already newer than 667 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 668 (ObjNum 457 snap 132 seq_num 457) dirty exists 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1283: left oid 7 (ObjNum 457 snap 132 seq_num 457) 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1284: finishing write tid 1 to trial17818531-15 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1284: finishing write tid 2 to trial17818531-15 2026-02-21T22:21:17.655 INFO:tasks.rados.rados.0.trial178.stdout:1286: expect (ObjNum 433 snap 123 seq_num 433) 2026-02-21T22:21:17.815 INFO:tasks.rados.rados.0.trial178.stdout:1287: finishing copy_from racing read to trial17818531-47 2026-02-21T22:21:17.815 INFO:tasks.rados.rados.0.trial178.stdout:1287: finishing copy_from to trial17818531-47 2026-02-21T22:21:17.815 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 178 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:21:17.816 INFO:tasks.rados.rados.0.trial178.stdout:1285: expect (ObjNum 438 snap 128 seq_num 438) 2026-02-21T22:21:17.832 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2595588 2026-02-21T22:21:17.841 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:17.876 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: finishing write tid 3 to trial17818531-15 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: finishing write tid 4 to trial17818531-15 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: finishing write tid 5 to trial17818531-15 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: oid 15 updating version 0 to 499 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: oid 15 version 499 is already newer than 498 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: oid 15 updating version 499 to 500 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:1284: oid 15 version 500 is already newer than 497 2026-02-21T22:21:17.911 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 500 (ObjNum 458 snap 132 seq_num 458) dirty exists 2026-02-21T22:21:17.912 INFO:tasks.rados.rados.0.trial178.stdout:1284: left oid 15 (ObjNum 458 snap 132 seq_num 458) 2026-02-21T22:21:18.436 INFO:tasks.rados.rados.0.trial178.stdout:1283: done (5 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1284: done (4 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1285: done (3 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1286: done (2 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1287: done (1 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1288: done (0 left) 2026-02-21T22:21:18.437 INFO:tasks.rados.rados.0.trial178.stdout:1289: delete oid 28 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1289: done (0 left) 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1290: rollback oid 46 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 123 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1291: copy_from oid 26 from oid 14 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1292: rollback oid 41 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 132 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1293: copy_from oid 17 from oid 43 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1294: write oid 24 current snap is 133 2026-02-21T22:21:18.438 INFO:tasks.rados.rados.0.trial178.stdout:1294: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2026-02-21T22:21:18.447 INFO:tasks.rados.rados.0.trial178.stdout:1294: writing trial17818531-24 from 531220 to 1326940 tid 1 2026-02-21T22:21:18.455 INFO:tasks.rados.rados.0.trial178.stdout:1294: writing trial17818531-24 from 1935011 to 2451821 tid 2 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1294: writing trial17818531-24 from 3066106 to 3486526 tid 3 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1295: read oid 13 snap -1 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1295: expect (ObjNum 454 snap 132 seq_num 454) 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1296: write (excl) oid 31 current snap is 133 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1290: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum 145 snap 0 seq_num 101036056) dirty dne 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1292: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:21:18.463 INFO:tasks.rados.rados.0.trial178.stdout:1292: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:21:18.464 INFO:tasks.rados.rados.0.trial178.stdout:1292: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:21:18.464 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 503 (ObjNum 438 snap 128 seq_num 438) dirty exists 2026-02-21T22:21:18.464 INFO:tasks.rados.rados.0.trial178.stdout:1294: finishing write tid 1 to trial17818531-24 2026-02-21T22:21:18.464 INFO:tasks.rados.rados.0.trial178.stdout:1291: finishing copy_from racing read to trial17818531-26 2026-02-21T22:21:18.464 INFO:tasks.rados.rados.0.trial178.stdout:1296: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2026-02-21T22:21:18.472 INFO:tasks.rados.rados.0.trial178.stdout:1296: writing trial17818531-31 from 548027 to 1018996 tid 1 2026-02-21T22:21:18.481 INFO:tasks.rados.rados.0.trial178.stdout:1296: writing trial17818531-31 from 1593493 to 2370314 tid 2 2026-02-21T22:21:18.490 INFO:tasks.rados.rados.0.trial178.stdout:1296: writing trial17818531-31 from 3093529 to 3725221 tid 3 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1290: done (6 left) 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1292: done (5 left) 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1297: read oid 48 snap -1 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1297: expect deleted 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1298: write (excl) oid 36 current snap is 133 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1291: finishing copy_from to trial17818531-26 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 466 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1293: finishing copy_from racing read to trial17818531-17 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1293: finishing copy_from to trial17818531-17 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 333 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: finishing write tid 2 to trial17818531-24 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: finishing write tid 3 to trial17818531-24 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: finishing write tid 4 to trial17818531-24 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: finishing write tid 5 to trial17818531-24 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: oid 24 updating version 0 to 673 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: oid 24 version 673 is already newer than 672 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: oid 24 version 673 is already newer than 671 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: oid 24 version 673 is already newer than 670 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 673 (ObjNum 460 snap 133 seq_num 460) dirty exists 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1294: left oid 24 (ObjNum 460 snap 133 seq_num 460) 2026-02-21T22:21:18.491 INFO:tasks.rados.rados.0.trial178.stdout:1295: expect (ObjNum 454 snap 132 seq_num 454) 2026-02-21T22:21:18.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:18.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:18.626 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:18.691 INFO:tasks.rados.rados.0.trial178.stdout:1296: finishing write tid 1 to trial17818531-31 2026-02-21T22:21:18.691 INFO:tasks.rados.rados.0.trial178.stdout:1296: finishing write tid 2 to trial17818531-31 2026-02-21T22:21:18.691 INFO:tasks.rados.rados.0.trial178.stdout:1296: finishing write tid 3 to trial17818531-31 2026-02-21T22:21:18.691 INFO:tasks.rados.rados.0.trial178.stdout:1296: finishing write tid 4 to trial17818531-31 2026-02-21T22:21:18.691 INFO:tasks.rados.rados.0.trial178.stdout:1296: finishing write tid 5 to trial17818531-31 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1296: oid 31 updating version 0 to 498 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1296: oid 31 version 498 is already newer than 497 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1296: oid 31 version 498 is already newer than 496 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1296: oid 31 updating version 498 to 499 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 499 (ObjNum 461 snap 133 seq_num 461) dirty exists 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1296: left oid 31 (ObjNum 461 snap 133 seq_num 461) 2026-02-21T22:21:18.692 INFO:tasks.rados.rados.0.trial178.stdout:1298: seq_num 462 ranges {564834=546218,1651975=520479,2720952=559317} 2026-02-21T22:21:18.695 INFO:tasks.rados.rados.0.trial178.stdout:1298: writing trial17818531-36 from 564834 to 1111052 tid 1 2026-02-21T22:21:18.699 INFO:tasks.rados.rados.0.trial178.stdout:1298: writing trial17818531-36 from 1651975 to 2172454 tid 2 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1298: writing trial17818531-36 from 2720952 to 3280269 tid 3 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1291: done (6 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1293: done (5 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1294: done (4 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1295: done (3 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1296: done (2 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1297: done (1 left) 2026-02-21T22:21:18.704 INFO:tasks.rados.rados.0.trial178.stdout:1299: write oid 19 current snap is 133 2026-02-21T22:21:18.705 INFO:tasks.rados.rados.0.trial178.stdout:1299: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2026-02-21T22:21:18.708 INFO:tasks.rados.rados.0.trial178.stdout:1299: writing trial17818531-19 from 581641 to 1203108 tid 1 2026-02-21T22:21:18.714 INFO:tasks.rados.rados.0.trial178.stdout:1299: writing trial17818531-19 from 1994104 to 2774594 tid 2 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1299: writing trial17818531-19 from 3432022 to 3781641 tid 3 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1300: rollback oid 38 current snap is 133 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 111 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: finishing write tid 1 to trial17818531-36 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: finishing write tid 2 to trial17818531-36 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: finishing write tid 3 to trial17818531-36 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: finishing write tid 4 to trial17818531-36 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: finishing write tid 5 to trial17818531-36 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: oid 36 updating version 0 to 174 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: oid 36 version 174 is already newer than 172 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: oid 36 version 174 is already newer than 171 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: oid 36 version 174 is already newer than 173 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 174 (ObjNum 462 snap 133 seq_num 462) dirty exists 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: left oid 36 (ObjNum 462 snap 133 seq_num 462) 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1299: finishing write tid 1 to trial17818531-19 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1299: finishing write tid 2 to trial17818531-19 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1298: done (2 left) 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1301: write (excl) oid 1 current snap is 133 2026-02-21T22:21:18.719 INFO:tasks.rados.rados.0.trial178.stdout:1301: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2026-02-21T22:21:18.723 INFO:tasks.rados.rados.0.trial178.stdout:1301: writing trial17818531-1 from 598448 to 1011517 tid 1 2026-02-21T22:21:18.727 INFO:tasks.rados.rados.0.trial178.stdout:1301: writing trial17818531-1 from 1768939 to 2293087 tid 2 2026-02-21T22:21:18.732 INFO:tasks.rados.rados.0.trial178.stdout:1301: writing trial17818531-1 from 2775798 to 3357659 tid 3 2026-02-21T22:21:18.733 INFO:tasks.rados.rados.0.trial178.stdout:1301: writing trial17818531-1 from 3781488 to 3798448 tid 4 2026-02-21T22:21:18.733 INFO:tasks.rados.rados.0.trial178.stdout:1302: read oid 9 snap -1 2026-02-21T22:21:18.733 INFO:tasks.rados.rados.0.trial178.stdout:1302: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:18.733 INFO:tasks.rados.rados.0.trial178.stdout:1303: snap_remove snap 122 2026-02-21T22:21:19.441 INFO:tasks.rados.rados.0.trial178.stdout:1303: done (4 left) 2026-02-21T22:21:19.441 INFO:tasks.rados.rados.0.trial178.stdout:1304: write oid 31 current snap is 133 2026-02-21T22:21:19.441 INFO:tasks.rados.rados.0.trial178.stdout:1304: seq_num 465 ranges {615255=488318,1827421=784159,3203221=612034} 2026-02-21T22:21:19.447 INFO:tasks.rados.rados.0.trial178.stdout:1304: writing trial17818531-31 from 615255 to 1103573 tid 1 2026-02-21T22:21:19.453 INFO:tasks.rados.rados.0.trial178.stdout:1304: writing trial17818531-31 from 1827421 to 2611580 tid 2 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1304: writing trial17818531-31 from 3203221 to 3815255 tid 3 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1305: copy_from oid 36 from oid 45 current snap is 133 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1300: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1300: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 242 (ObjNum 401 snap 111 seq_num 401) dirty exists 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: finishing write tid 3 to trial17818531-19 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: finishing write tid 4 to trial17818531-19 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: finishing write tid 5 to trial17818531-19 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: oid 19 updating version 0 to 676 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: oid 19 version 676 is already newer than 674 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: oid 19 updating version 676 to 677 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:1299: oid 19 version 677 is already newer than 675 2026-02-21T22:21:19.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 677 (ObjNum 463 snap 133 seq_num 463) dirty exists 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1299: left oid 19 (ObjNum 463 snap 133 seq_num 463) 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 1 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 2 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 3 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 4 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1299: done (5 left) 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1300: done (4 left) 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1306: copy_from oid 48 from oid 49 current snap is 133 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 5 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: finishing write tid 6 to trial17818531-1 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: oid 1 updating version 0 to 415 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: oid 1 updating version 415 to 416 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: oid 1 updating version 416 to 418 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: oid 1 version 418 is already newer than 417 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: oid 1 version 418 is already newer than 414 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 418 (ObjNum 464 snap 133 seq_num 464) dirty exists 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1301: left oid 1 (ObjNum 464 snap 133 seq_num 464) 2026-02-21T22:21:19.459 INFO:tasks.rados.rados.0.trial178.stdout:1302: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:19.622 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2479337 2026-02-21T22:21:19.634 INFO:tasks.rados.rados.0.trial178.stdout:1304: finishing write tid 1 to trial17818531-31 2026-02-21T22:21:19.634 INFO:tasks.rados.rados.0.trial178.stdout:1304: finishing write tid 2 to trial17818531-31 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1305: finishing copy_from to trial17818531-36 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1305: got expected ENOENT (src dne) 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1305: finishing copy_from racing read to trial17818531-36 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: finishing write tid 3 to trial17818531-31 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: finishing write tid 4 to trial17818531-31 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: finishing write tid 5 to trial17818531-31 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: oid 31 updating version 0 to 501 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: oid 31 version 501 is already newer than 500 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: oid 31 updating version 501 to 502 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: oid 31 updating version 502 to 503 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 503 (ObjNum 465 snap 133 seq_num 465) dirty exists 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1304: left oid 31 (ObjNum 465 snap 133 seq_num 465) 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1306: finishing copy_from racing read to trial17818531-48 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1306: finishing copy_from to trial17818531-48 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 495 (ObjNum 424 snap 117 seq_num 424) dirty exists 2026-02-21T22:21:19.635 INFO:tasks.rados.rados.0.trial178.stdout:1301: done (4 left) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1302: done (3 left) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1304: done (2 left) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1305: done (1 left) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1306: done (0 left) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1307: read oid 9 snap 132 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1307: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1308: write (excl) oid 39 current snap is 133 2026-02-21T22:21:19.636 INFO:tasks.rados.rados.0.trial178.stdout:1308: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2026-02-21T22:21:19.644 INFO:tasks.rados.rados.0.trial178.stdout:1308: writing trial17818531-39 from 632062 to 1195629 tid 1 2026-02-21T22:21:19.653 INFO:tasks.rados.rados.0.trial178.stdout:1308: writing trial17818531-39 from 1885903 to 2413720 tid 2 2026-02-21T22:21:19.661 INFO:tasks.rados.rados.0.trial178.stdout:1308: writing trial17818531-39 from 2830644 to 3551402 tid 3 2026-02-21T22:21:19.661 INFO:tasks.rados.rados.0.trial178.stdout:1309: write oid 50 current snap is 133 2026-02-21T22:21:19.661 INFO:tasks.rados.rados.0.trial178.stdout:1309: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2026-02-21T22:21:19.670 INFO:tasks.rados.rados.0.trial178.stdout:1309: writing trial17818531-50 from 648869 to 1287685 tid 1 2026-02-21T22:21:19.679 INFO:tasks.rados.rados.0.trial178.stdout:1309: writing trial17818531-50 from 1828032 to 2615860 tid 2 2026-02-21T22:21:19.685 INFO:tasks.rados.rados.0.trial178.stdout:1309: writing trial17818531-50 from 3141714 to 3673744 tid 3 2026-02-21T22:21:19.685 INFO:tasks.rados.rados.0.trial178.stdout:1310: copy_from oid 35 from oid 43 current snap is 133 2026-02-21T22:21:19.685 INFO:tasks.rados.rados.0.trial178.stdout:1307: expect (ObjNum 443 snap 128 seq_num 443) 2026-02-21T22:21:19.700 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1185054 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: finishing write tid 1 to trial17818531-39 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: finishing write tid 2 to trial17818531-39 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: finishing write tid 3 to trial17818531-39 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: finishing write tid 4 to trial17818531-39 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: finishing write tid 5 to trial17818531-39 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: oid 39 updating version 0 to 473 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: oid 39 updating version 473 to 474 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: oid 39 version 474 is already newer than 471 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: oid 39 version 474 is already newer than 472 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 474 (ObjNum 466 snap 133 seq_num 466) dirty exists 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1308: left oid 39 (ObjNum 466 snap 133 seq_num 466) 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: finishing write tid 1 to trial17818531-50 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: finishing write tid 2 to trial17818531-50 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: finishing write tid 3 to trial17818531-50 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: finishing write tid 4 to trial17818531-50 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: finishing write tid 5 to trial17818531-50 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: oid 50 updating version 0 to 494 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: oid 50 version 494 is already newer than 493 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: oid 50 updating version 494 to 496 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: oid 50 version 496 is already newer than 495 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 496 (ObjNum 467 snap 133 seq_num 467) dirty exists 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1309: left oid 50 (ObjNum 467 snap 133 seq_num 467) 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1310: finishing copy_from racing read to trial17818531-35 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:1310: finishing copy_from to trial17818531-35 2026-02-21T22:21:19.869 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 333 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:21:19.870 INFO:tasks.rados.rados.0.trial178.stdout:1307: done (3 left) 2026-02-21T22:21:19.870 INFO:tasks.rados.rados.0.trial178.stdout:1308: done (2 left) 2026-02-21T22:21:19.870 INFO:tasks.rados.rados.0.trial178.stdout:1309: done (1 left) 2026-02-21T22:21:19.870 INFO:tasks.rados.rados.0.trial178.stdout:1310: done (0 left) 2026-02-21T22:21:19.870 INFO:tasks.rados.rados.0.trial178.stdout:1311: delete oid 49 current snap is 133 2026-02-21T22:21:19.871 INFO:tasks.rados.rados.0.trial178.stdout:1311: done (0 left) 2026-02-21T22:21:19.871 INFO:tasks.rados.rados.0.trial178.stdout:1312: write (excl) oid 16 current snap is 133 2026-02-21T22:21:19.871 INFO:tasks.rados.rados.0.trial178.stdout:1312: seq_num 469 ranges {682483=789314,1944996=791497,3196560=554574} 2026-02-21T22:21:19.881 INFO:tasks.rados.rados.0.trial178.stdout:1312: writing trial17818531-16 from 682483 to 1471797 tid 1 2026-02-21T22:21:19.890 INFO:tasks.rados.rados.0.trial178.stdout:1312: writing trial17818531-16 from 1944996 to 2736493 tid 2 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1312: writing trial17818531-16 from 3196560 to 3751134 tid 3 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1313: rollback oid 4 current snap is 133 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 131 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1312: finishing write tid 1 to trial17818531-16 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1312: finishing write tid 2 to trial17818531-16 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1314: write (excl) oid 38 current snap is 133 2026-02-21T22:21:19.897 INFO:tasks.rados.rados.0.trial178.stdout:1314: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2026-02-21T22:21:19.905 INFO:tasks.rados.rados.0.trial178.stdout:1314: writing trial17818531-38 from 699290 to 1163853 tid 1 2026-02-21T22:21:19.911 INFO:tasks.rados.rados.0.trial178.stdout:1314: writing trial17818531-38 from 1603478 to 2138633 tid 2 2026-02-21T22:21:19.921 INFO:tasks.rados.rados.0.trial178.stdout:1314: writing trial17818531-38 from 2823983 to 3589829 tid 3 2026-02-21T22:21:19.921 INFO:tasks.rados.rados.0.trial178.stdout:1315: write (excl) oid 21 current snap is 133 2026-02-21T22:21:19.921 INFO:tasks.rados.rados.0.trial178.stdout:1315: seq_num 471 ranges {716097=539812,1945607=795166,3535053=381044} 2026-02-21T22:21:19.926 INFO:tasks.rados.rados.0.trial178.stdout:1315: writing trial17818531-21 from 716097 to 1255909 tid 1 2026-02-21T22:21:19.932 INFO:tasks.rados.rados.0.trial178.stdout:1315: writing trial17818531-21 from 1945607 to 2740773 tid 2 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:1315: writing trial17818531-21 from 3535053 to 3916097 tid 3 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:1316: read oid 14 snap 111 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:1316: expect (ObjNum 333 snap 95 seq_num 333) 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:1317: rollback oid 5 current snap is 133 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 123 2026-02-21T22:21:19.937 INFO:tasks.rados.rados.0.trial178.stdout:1312: finishing write tid 3 to trial17818531-16 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: finishing write tid 4 to trial17818531-16 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: finishing write tid 5 to trial17818531-16 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: oid 16 updating version 0 to 356 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: oid 16 updating version 356 to 357 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: oid 16 updating version 357 to 358 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: oid 16 version 358 is already newer than 355 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 358 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: left oid 16 (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1313: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 350 (ObjNum 129 snap 0 seq_num 99498856) dirty dne 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: finishing write tid 1 to trial17818531-38 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: finishing write tid 2 to trial17818531-38 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1312: done (5 left) 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1313: done (4 left) 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1318: delete oid 9 current snap is 133 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: finishing write tid 3 to trial17818531-38 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: finishing write tid 4 to trial17818531-38 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: finishing write tid 5 to trial17818531-38 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: oid 38 updating version 0 to 247 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: oid 38 updating version 247 to 250 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: oid 38 version 250 is already newer than 248 2026-02-21T22:21:19.938 INFO:tasks.rados.rados.0.trial178.stdout:1314: oid 38 version 250 is already newer than 249 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 250 (ObjNum 470 snap 133 seq_num 470) dirty exists 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:1314: left oid 38 (ObjNum 470 snap 133 seq_num 470) 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:1315: finishing write tid 1 to trial17818531-21 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:1315: finishing write tid 2 to trial17818531-21 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:1317: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:21:19.939 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: finishing write tid 3 to trial17818531-21 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: finishing write tid 4 to trial17818531-21 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: finishing write tid 5 to trial17818531-21 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: oid 21 updating version 0 to 685 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: oid 21 updating version 685 to 687 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: oid 21 updating version 687 to 688 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: oid 21 version 688 is already newer than 686 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 688 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:21:19.943 INFO:tasks.rados.rados.0.trial178.stdout:1315: left oid 21 (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:21:19.944 INFO:tasks.rados.rados.0.trial178.stdout:1316: expect (ObjNum 333 snap 95 seq_num 333) 2026-02-21T22:21:20.014 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1672291 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1314: done (4 left) 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1315: done (3 left) 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1316: done (2 left) 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1317: done (1 left) 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1318: done (0 left) 2026-02-21T22:21:20.097 INFO:tasks.rados.rados.0.trial178.stdout:1319: delete oid 19 current snap is 133 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1319: done (0 left) 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1320: rollback oid 49 current snap is 133 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 111 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1321: read oid 16 snap -1 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1321: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1322: read oid 39 snap -1 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1322: expect (ObjNum 466 snap 133 seq_num 466) 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1323: rollback oid 18 current snap is 133 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 129 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1324: read oid 5 snap -1 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1324: expect deleted 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1325: rollback oid 28 current snap is 133 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 131 2026-02-21T22:21:20.098 INFO:tasks.rados.rados.0.trial178.stdout:1326: delete oid 29 current snap is 133 2026-02-21T22:21:20.099 INFO:tasks.rados.rados.0.trial178.stdout:1320: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:21:20.099 INFO:tasks.rados.rados.0.trial178.stdout:1320: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:21:20.099 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 506 (ObjNum 381 snap 105 seq_num 381) dirty exists 2026-02-21T22:21:20.099 INFO:tasks.rados.rados.0.trial178.stdout:1325: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:21:20.099 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 0 (ObjNum 49 snap 0 seq_num 4122730800) dirty dne 2026-02-21T22:21:20.101 INFO:tasks.rados.rados.0.trial178.stdout:1323: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:21:20.102 INFO:tasks.rados.rados.0.trial178.stdout:1323: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:21:20.102 INFO:tasks.rados.rados.0.trial178.stdout:1323: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:21:20.102 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 533 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1320: done (6 left) 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1323: done (5 left) 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1324: done (4 left) 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1325: done (3 left) 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1326: done (2 left) 2026-02-21T22:21:20.104 INFO:tasks.rados.rados.0.trial178.stdout:1327: read oid 44 snap -1 2026-02-21T22:21:20.105 INFO:tasks.rados.rados.0.trial178.stdout:1327: expect (ObjNum 412 snap 115 seq_num 412) 2026-02-21T22:21:20.105 INFO:tasks.rados.rados.0.trial178.stdout:1328: delete oid 37 current snap is 133 2026-02-21T22:21:20.105 INFO:tasks.rados.rados.0.trial178.stdout:1322: expect (ObjNum 466 snap 133 seq_num 466) 2026-02-21T22:21:20.204 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 359083 2026-02-21T22:21:20.252 INFO:tasks.rados.rados.0.trial178.stdout:1321: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:21:20.262 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 928148 2026-02-21T22:21:20.327 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:20.371 INFO:tasks.rados.rados.0.trial178.stdout:1327: expect (ObjNum 412 snap 115 seq_num 412) 2026-02-21T22:21:20.412 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 180894 2026-02-21T22:21:20.456 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:20.502 INFO:tasks.rados.rados.0.trial178.stdout:1321: done (3 left) 2026-02-21T22:21:20.502 INFO:tasks.rados.rados.0.trial178.stdout:1322: done (2 left) 2026-02-21T22:21:20.502 INFO:tasks.rados.rados.0.trial178.stdout:1327: done (1 left) 2026-02-21T22:21:20.502 INFO:tasks.rados.rados.0.trial178.stdout:1328: done (0 left) 2026-02-21T22:21:20.502 INFO:tasks.rados.rados.0.trial178.stdout:1329: write (excl) oid 18 current snap is 133 2026-02-21T22:21:20.503 INFO:tasks.rados.rados.0.trial178.stdout:1329: seq_num 476 ranges {400132=632410,1438017=546162} 2026-02-21T22:21:20.509 INFO:tasks.rados.rados.0.trial178.stdout:1329: writing trial17818531-18 from 400132 to 1032542 tid 1 2026-02-21T22:21:20.516 INFO:tasks.rados.rados.0.trial178.stdout:1329: writing trial17818531-18 from 1438017 to 1984179 tid 2 2026-02-21T22:21:20.516 INFO:tasks.rados.rados.0.trial178.stdout:1330: read oid 8 snap -1 2026-02-21T22:21:20.516 INFO:tasks.rados.rados.0.trial178.stdout:1330: expect deleted 2026-02-21T22:21:20.516 INFO:tasks.rados.rados.0.trial178.stdout:1331: snap_remove snap 111 2026-02-21T22:21:21.446 INFO:tasks.rados.rados.0.trial178.stdout:1331: done (2 left) 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1332: delete oid 19 current snap is 133 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: finishing write tid 1 to trial17818531-18 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: finishing write tid 2 to trial17818531-18 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: finishing write tid 3 to trial17818531-18 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: finishing write tid 4 to trial17818531-18 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: oid 18 updating version 0 to 536 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: oid 18 version 536 is already newer than 534 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:1329: oid 18 version 536 is already newer than 535 2026-02-21T22:21:21.447 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 536 (ObjNum 476 snap 133 seq_num 476) dirty exists 2026-02-21T22:21:21.448 INFO:tasks.rados.rados.0.trial178.stdout:1329: left oid 18 (ObjNum 476 snap 133 seq_num 476) 2026-02-21T22:21:21.452 INFO:tasks.rados.rados.0.trial178.stdout:1329: done (2 left) 2026-02-21T22:21:21.452 INFO:tasks.rados.rados.0.trial178.stdout:1330: done (1 left) 2026-02-21T22:21:21.452 INFO:tasks.rados.rados.0.trial178.stdout:1332: done (0 left) 2026-02-21T22:21:21.452 INFO:tasks.rados.rados.0.trial178.stdout:1333: delete oid 18 current snap is 133 2026-02-21T22:21:21.454 INFO:tasks.rados.rados.0.trial178.stdout:1333: done (0 left) 2026-02-21T22:21:21.454 INFO:tasks.rados.rados.0.trial178.stdout:1334: rollback oid 43 current snap is 133 2026-02-21T22:21:21.455 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 129 2026-02-21T22:21:21.455 INFO:tasks.rados.rados.0.trial178.stdout:1335: delete oid 37 current snap is 133 2026-02-21T22:21:21.456 INFO:tasks.rados.rados.0.trial178.stdout:1335: done (1 left) 2026-02-21T22:21:21.456 INFO:tasks.rados.rados.0.trial178.stdout:1336: snap_remove snap 123 2026-02-21T22:21:22.420 INFO:tasks.rados.rados.0.trial178.stdout:1336: done (1 left) 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1337: read oid 49 snap -1 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1337: expect (ObjNum 381 snap 105 seq_num 381) 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1338: read oid 38 snap -1 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1338: expect (ObjNum 470 snap 133 seq_num 470) 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1339: copy_from oid 10 from oid 46 current snap is 133 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1334: finishing rollback tid 0 to trial17818531-43 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1334: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 335 (ObjNum 129 snap 0 seq_num 99769992) dirty dne 2026-02-21T22:21:22.421 INFO:tasks.rados.rados.0.trial178.stdout:1334: done (3 left) 2026-02-21T22:21:22.422 INFO:tasks.rados.rados.0.trial178.stdout:1340: snap_create 2026-02-21T22:21:22.426 INFO:tasks.rados.rados.0.trial178.stdout:1337: expect (ObjNum 381 snap 105 seq_num 381) 2026-02-21T22:21:22.467 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2054146 2026-02-21T22:21:22.491 INFO:tasks.rados.rados.0.trial178.stdout:1338: expect (ObjNum 470 snap 133 seq_num 470) 2026-02-21T22:21:22.645 INFO:tasks.rados.rados.0.trial178.stdout:1339: finishing copy_from to trial17818531-10 2026-02-21T22:21:22.645 INFO:tasks.rados.rados.0.trial178.stdout:1339: got expected ENOENT (src dne) 2026-02-21T22:21:22.645 INFO:tasks.rados.rados.0.trial178.stdout:1339: finishing copy_from racing read to trial17818531-10 2026-02-21T22:21:23.421 INFO:tasks.rados.rados.0.trial178.stdout:1337: done (3 left) 2026-02-21T22:21:23.421 INFO:tasks.rados.rados.0.trial178.stdout:1338: done (2 left) 2026-02-21T22:21:23.421 INFO:tasks.rados.rados.0.trial178.stdout:1339: done (1 left) 2026-02-21T22:21:23.421 INFO:tasks.rados.rados.0.trial178.stdout:1340: done (0 left) 2026-02-21T22:21:23.421 INFO:tasks.rados.rados.0.trial178.stdout:1341: snap_remove snap 131 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:21:23] 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:21:23, mode upmap, max misplaced 0.050000 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'rbd', 'unique_pool_0'] 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:21:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:21:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:21:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:21:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:21:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:21:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:21:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:21:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:21:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:21:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:21:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:21:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:21:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:21:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:21:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:21:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:21:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:21:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:21:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:21:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:21:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:21:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:21:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "288", 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:21:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:21:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:21:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:21:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 288, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 138, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:21:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:21:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:21:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:21:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:21:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0004840638902452257 of space, bias 1.0, pg target 0.14521916707356772 quantized to 32 (current 16) 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:21:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:21:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:21:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:21:24.424 INFO:tasks.rados.rados.0.trial178.stdout:1341: done (0 left) 2026-02-21T22:21:24.425 INFO:tasks.rados.rados.0.trial178.stdout:1342: rollback oid 6 current snap is 134 2026-02-21T22:21:24.425 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 133 2026-02-21T22:21:24.425 INFO:tasks.rados.rados.0.trial178.stdout:1343: rollback oid 19 current snap is 134 2026-02-21T22:21:24.425 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 133 2026-02-21T22:21:24.425 INFO:tasks.rados.rados.0.trial178.stdout:1344: read oid 9 snap -1 2026-02-21T22:21:24.426 INFO:tasks.rados.rados.0.trial178.stdout:1344: expect deleted 2026-02-21T22:21:24.426 INFO:tasks.rados.rados.0.trial178.stdout:1345: delete oid 36 current snap is 134 2026-02-21T22:21:24.427 INFO:tasks.rados.rados.0.trial178.stdout:1342: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:21:24.427 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 0 (ObjNum 65 snap 0 seq_num 99560784) dirty dne 2026-02-21T22:21:24.427 INFO:tasks.rados.rados.0.trial178.stdout:1343: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:21:24.427 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 0 (ObjNum 1201 snap 0 seq_num 99535984) dirty dne 2026-02-21T22:21:24.428 INFO:tasks.rados.rados.0.trial178.stdout:1342: done (3 left) 2026-02-21T22:21:24.428 INFO:tasks.rados.rados.0.trial178.stdout:1343: done (2 left) 2026-02-21T22:21:24.428 INFO:tasks.rados.rados.0.trial178.stdout:1344: done (1 left) 2026-02-21T22:21:24.428 INFO:tasks.rados.rados.0.trial178.stdout:1345: done (0 left) 2026-02-21T22:21:24.429 INFO:tasks.rados.rados.0.trial178.stdout:1346: write (excl) oid 41 current snap is 134 2026-02-21T22:21:24.429 INFO:tasks.rados.rados.0.trial178.stdout:1346: seq_num 481 ranges {484167=725008,1730427=353740} 2026-02-21T22:21:24.436 INFO:tasks.rados.rados.0.trial178.stdout:1346: writing trial17818531-41 from 484167 to 1209175 tid 1 2026-02-21T22:21:24.442 INFO:tasks.rados.rados.0.trial178.stdout:1346: writing trial17818531-41 from 1730427 to 2084167 tid 2 2026-02-21T22:21:24.442 INFO:tasks.rados.rados.0.trial178.stdout:1347: delete oid 42 current snap is 134 2026-02-21T22:21:24.442 INFO:tasks.rados.rados.0.trial178.stdout:1346: finishing write tid 1 to trial17818531-41 2026-02-21T22:21:24.444 INFO:tasks.rados.rados.0.trial178.stdout:1347: done (1 left) 2026-02-21T22:21:24.444 INFO:tasks.rados.rados.0.trial178.stdout:1348: delete oid 30 current snap is 134 2026-02-21T22:21:24.445 INFO:tasks.rados.rados.0.trial178.stdout:1346: finishing write tid 2 to trial17818531-41 2026-02-21T22:21:24.445 INFO:tasks.rados.rados.0.trial178.stdout:1346: finishing write tid 3 to trial17818531-41 2026-02-21T22:21:24.445 INFO:tasks.rados.rados.0.trial178.stdout:1346: finishing write tid 4 to trial17818531-41 2026-02-21T22:21:24.445 INFO:tasks.rados.rados.0.trial178.stdout:1346: oid 41 updating version 0 to 504 2026-02-21T22:21:24.446 INFO:tasks.rados.rados.0.trial178.stdout:1346: oid 41 updating version 504 to 505 2026-02-21T22:21:24.446 INFO:tasks.rados.rados.0.trial178.stdout:1346: oid 41 updating version 505 to 506 2026-02-21T22:21:24.446 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 506 (ObjNum 481 snap 134 seq_num 481) dirty exists 2026-02-21T22:21:24.446 INFO:tasks.rados.rados.0.trial178.stdout:1346: left oid 41 (ObjNum 481 snap 134 seq_num 481) 2026-02-21T22:21:24.447 INFO:tasks.rados.rados.0.trial178.stdout:1346: done (1 left) 2026-02-21T22:21:24.447 INFO:tasks.rados.rados.0.trial178.stdout:1348: done (0 left) 2026-02-21T22:21:24.447 INFO:tasks.rados.rados.0.trial178.stdout:1349: copy_from oid 1 from oid 35 current snap is 134 2026-02-21T22:21:24.447 INFO:tasks.rados.rados.0.trial178.stdout:1350: delete oid 32 current snap is 134 2026-02-21T22:21:24.451 INFO:tasks.rados.rados.0.trial178.stdout:1350: done (1 left) 2026-02-21T22:21:24.451 INFO:tasks.rados.rados.0.trial178.stdout:1351: read oid 43 snap -1 2026-02-21T22:21:24.452 INFO:tasks.rados.rados.0.trial178.stdout:1351: expect deleted 2026-02-21T22:21:24.452 INFO:tasks.rados.rados.0.trial178.stdout:1352: read oid 24 snap -1 2026-02-21T22:21:24.452 INFO:tasks.rados.rados.0.trial178.stdout:1352: expect (ObjNum 460 snap 133 seq_num 460) 2026-02-21T22:21:24.452 INFO:tasks.rados.rados.0.trial178.stdout:1353: write (excl) oid 44 current snap is 134 2026-02-21T22:21:24.452 INFO:tasks.rados.rados.0.trial178.stdout:1353: seq_num 485 ranges {551395=626004,1848002=303393} 2026-02-21T22:21:24.462 INFO:tasks.rados.rados.0.trial178.stdout:1353: writing trial17818531-44 from 551395 to 1177399 tid 1 2026-02-21T22:21:24.469 INFO:tasks.rados.rados.0.trial178.stdout:1353: writing trial17818531-44 from 1848002 to 2151395 tid 2 2026-02-21T22:21:24.469 INFO:tasks.rados.rados.0.trial178.stdout:1354: copy_from oid 33 from oid 19 current snap is 134 2026-02-21T22:21:24.469 INFO:tasks.rados.rados.0.trial178.stdout:1349: finishing copy_from racing read to trial17818531-1 2026-02-21T22:21:24.469 INFO:tasks.rados.rados.0.trial178.stdout:1352: expect (ObjNum 460 snap 133 seq_num 460) 2026-02-21T22:21:24.470 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:24.544 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:24.572 INFO:tasks.rados.rados.0.trial178.stdout:1349: finishing copy_from to trial17818531-1 2026-02-21T22:21:24.572 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 333 (ObjNum 422 snap 117 seq_num 422) dirty exists 2026-02-21T22:21:24.572 INFO:tasks.rados.rados.0.trial178.stdout:1353: finishing write tid 1 to trial17818531-44 2026-02-21T22:21:24.572 INFO:tasks.rados.rados.0.trial178.stdout:1354: finishing copy_from to trial17818531-33 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1354: got expected ENOENT (src dne) 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1354: finishing copy_from racing read to trial17818531-33 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: finishing write tid 2 to trial17818531-44 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: finishing write tid 3 to trial17818531-44 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: finishing write tid 4 to trial17818531-44 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: oid 44 updating version 0 to 361 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: oid 44 version 361 is already newer than 360 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: oid 44 version 361 is already newer than 359 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 361 (ObjNum 485 snap 134 seq_num 485) dirty exists 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1353: left oid 44 (ObjNum 485 snap 134 seq_num 485) 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1349: done (4 left) 2026-02-21T22:21:24.573 INFO:tasks.rados.rados.0.trial178.stdout:1351: done (3 left) 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1352: done (2 left) 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1353: done (1 left) 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1354: done (0 left) 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1355: read oid 10 snap 133 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1355: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:21:24.574 INFO:tasks.rados.rados.0.trial178.stdout:1356: snap_remove snap 132 2026-02-21T22:21:25.428 INFO:tasks.rados.rados.0.trial178.stdout:1356: done (1 left) 2026-02-21T22:21:25.428 INFO:tasks.rados.rados.0.trial178.stdout:1357: copy_from oid 28 from oid 27 current snap is 134 2026-02-21T22:21:25.428 INFO:tasks.rados.rados.0.trial178.stdout:1355: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:21:25.428 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:25.503 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:25.578 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2242141 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1357: finishing copy_from racing read to trial17818531-28 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1357: finishing copy_from to trial17818531-28 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 384 (ObjNum 430 snap 123 seq_num 430) dirty exists 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1355: done (1 left) 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1357: done (0 left) 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1358: copy_from oid 47 from oid 21 current snap is 134 2026-02-21T22:21:25.608 INFO:tasks.rados.rados.0.trial178.stdout:1359: snap_create 2026-02-21T22:21:25.623 INFO:tasks.rados.rados.0.trial178.stdout:1358: finishing copy_from racing read to trial17818531-47 2026-02-21T22:21:25.626 INFO:tasks.rados.rados.0.trial178.stdout:1358: finishing copy_from to trial17818531-47 2026-02-21T22:21:25.626 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 688 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:21:26.433 INFO:tasks.rados.rados.0.trial178.stdout:1358: done (1 left) 2026-02-21T22:21:26.433 INFO:tasks.rados.rados.0.trial178.stdout:1359: done (0 left) 2026-02-21T22:21:26.434 INFO:tasks.rados.rados.0.trial178.stdout:1360: write oid 49 current snap is 135 2026-02-21T22:21:26.434 INFO:tasks.rados.rados.0.trial178.stdout:1360: seq_num 486 ranges {568202=701253,1906484=261718} 2026-02-21T22:21:26.442 INFO:tasks.rados.rados.0.trial178.stdout:1360: writing trial17818531-49 from 568202 to 1269455 tid 1 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1360: writing trial17818531-49 from 1906484 to 2168202 tid 2 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1361: rollback oid 6 current snap is 135 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 133 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1360: finishing write tid 1 to trial17818531-49 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1362: read oid 9 snap -1 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1362: expect deleted 2026-02-21T22:21:26.448 INFO:tasks.rados.rados.0.trial178.stdout:1363: delete oid 31 current snap is 135 2026-02-21T22:21:26.450 INFO:tasks.rados.rados.0.trial178.stdout:1360: finishing write tid 2 to trial17818531-49 2026-02-21T22:21:26.451 INFO:tasks.rados.rados.0.trial178.stdout:1360: finishing write tid 3 to trial17818531-49 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1361: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: finishing write tid 4 to trial17818531-49 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: oid 49 updating version 0 to 507 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: oid 49 updating version 507 to 508 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: oid 49 updating version 508 to 509 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 509 (ObjNum 486 snap 135 seq_num 486) dirty exists 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: left oid 49 (ObjNum 486 snap 135 seq_num 486) 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1360: done (3 left) 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1361: done (2 left) 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1362: done (1 left) 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1363: done (0 left) 2026-02-21T22:21:26.456 INFO:tasks.rados.rados.0.trial178.stdout:1364: snap_create 2026-02-21T22:21:27.421 INFO:tasks.rados.rados.0.trial178.stdout:1364: done (0 left) 2026-02-21T22:21:27.421 INFO:tasks.rados.rados.0.trial178.stdout:1365: read oid 12 snap -1 2026-02-21T22:21:27.421 INFO:tasks.rados.rados.0.trial178.stdout:1365: expect (ObjNum 433 snap 123 seq_num 433) 2026-02-21T22:21:27.421 INFO:tasks.rados.rados.0.trial178.stdout:1366: write (excl) oid 24 current snap is 136 2026-02-21T22:21:27.421 INFO:tasks.rados.rados.0.trial178.stdout:1366: seq_num 488 ranges {601816=568104,1623448=451823} 2026-02-21T22:21:27.428 INFO:tasks.rados.rados.0.trial178.stdout:1366: writing trial17818531-24 from 601816 to 1169920 tid 1 2026-02-21T22:21:27.435 INFO:tasks.rados.rados.0.trial178.stdout:1366: writing trial17818531-24 from 1623448 to 2075271 tid 2 2026-02-21T22:21:27.435 INFO:tasks.rados.rados.0.trial178.stdout:1367: rollback oid 2 current snap is 136 2026-02-21T22:21:27.436 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 135 2026-02-21T22:21:27.436 INFO:tasks.rados.rados.0.trial178.stdout:1365: expect (ObjNum 433 snap 123 seq_num 433) 2026-02-21T22:21:27.436 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:27.556 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: finishing write tid 1 to trial17818531-24 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: finishing write tid 2 to trial17818531-24 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1367: finishing rollback tid 0 to trial17818531-2 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1367: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: finishing write tid 3 to trial17818531-24 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: finishing write tid 4 to trial17818531-24 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: oid 24 updating version 0 to 725 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: oid 24 updating version 725 to 726 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: oid 24 version 726 is already newer than 724 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 726 (ObjNum 488 snap 136 seq_num 488) dirty exists 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: left oid 24 (ObjNum 488 snap 136 seq_num 488) 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1367: finishing rollback tid 2 to trial17818531-2 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 496 (ObjNum 425 snap 118 seq_num 425) dirty exists 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1365: done (2 left) 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1366: done (1 left) 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1367: done (0 left) 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1368: read oid 9 snap -1 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1368: expect deleted 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1369: read oid 31 snap 129 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1369: expect deleted 2026-02-21T22:21:27.611 INFO:tasks.rados.rados.0.trial178.stdout:1370: delete oid 24 current snap is 136 2026-02-21T22:21:27.612 INFO:tasks.rados.rados.0.trial178.stdout:1368: done (2 left) 2026-02-21T22:21:27.612 INFO:tasks.rados.rados.0.trial178.stdout:1369: done (1 left) 2026-02-21T22:21:27.612 INFO:tasks.rados.rados.0.trial178.stdout:1370: done (0 left) 2026-02-21T22:21:27.612 INFO:tasks.rados.rados.0.trial178.stdout:1371: delete oid 33 current snap is 136 2026-02-21T22:21:27.616 INFO:tasks.rados.rados.0.trial178.stdout:1371: done (0 left) 2026-02-21T22:21:27.616 INFO:tasks.rados.rados.0.trial178.stdout:1372: rollback oid 1 current snap is 136 2026-02-21T22:21:27.616 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 129 2026-02-21T22:21:27.616 INFO:tasks.rados.rados.0.trial178.stdout:1373: write (excl) oid 15 current snap is 136 2026-02-21T22:21:27.616 INFO:tasks.rados.rados.0.trial178.stdout:1373: seq_num 491 ranges {652237=793851,2198894=53343} 2026-02-21T22:21:27.625 INFO:tasks.rados.rados.0.trial178.stdout:1373: writing trial17818531-15 from 652237 to 1446088 tid 1 2026-02-21T22:21:27.632 INFO:tasks.rados.rados.0.trial178.stdout:1373: writing trial17818531-15 from 2198894 to 2252237 tid 2 2026-02-21T22:21:27.632 INFO:tasks.rados.rados.0.trial178.stdout:1374: read oid 5 snap 133 2026-02-21T22:21:27.632 INFO:tasks.rados.rados.0.trial178.stdout:1374: expect deleted 2026-02-21T22:21:27.632 INFO:tasks.rados.rados.0.trial178.stdout:1375: write (excl) oid 47 current snap is 136 2026-02-21T22:21:27.632 INFO:tasks.rados.rados.0.trial178.stdout:1375: seq_num 492 ranges {669044=469100,1741023=459161} 2026-02-21T22:21:27.639 INFO:tasks.rados.rados.0.trial178.stdout:1375: writing trial17818531-47 from 669044 to 1138144 tid 1 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:1375: writing trial17818531-47 from 1741023 to 2200184 tid 2 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:1376: rollback oid 46 current snap is 136 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 133 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:1372: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:1377: snap_create 2026-02-21T22:21:27.648 INFO:tasks.rados.rados.0.trial178.stdout:1372: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1372: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 498 (ObjNum 331 snap 95 seq_num 331) dirty exists 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: finishing write tid 1 to trial17818531-15 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: finishing write tid 2 to trial17818531-15 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: finishing write tid 3 to trial17818531-15 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: finishing write tid 4 to trial17818531-15 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: oid 15 updating version 0 to 508 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: oid 15 updating version 508 to 509 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: oid 15 version 509 is already newer than 507 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 509 (ObjNum 491 snap 136 seq_num 491) dirty exists 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1373: left oid 15 (ObjNum 491 snap 136 seq_num 491) 2026-02-21T22:21:27.649 INFO:tasks.rados.rados.0.trial178.stdout:1375: finishing write tid 1 to trial17818531-47 2026-02-21T22:21:27.650 INFO:tasks.rados.rados.0.trial178.stdout:1376: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:21:27.650 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum 129 snap 0 seq_num 99593240) dirty dne 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: finishing write tid 2 to trial17818531-47 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: finishing write tid 3 to trial17818531-47 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: finishing write tid 4 to trial17818531-47 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: oid 47 updating version 0 to 729 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: oid 47 updating version 729 to 730 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: oid 47 updating version 730 to 731 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 731 (ObjNum 492 snap 136 seq_num 492) dirty exists 2026-02-21T22:21:27.651 INFO:tasks.rados.rados.0.trial178.stdout:1375: left oid 47 (ObjNum 492 snap 136 seq_num 492) 2026-02-21T22:21:28.441 INFO:tasks.rados.rados.0.trial178.stdout:1372: done (5 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1373: done (4 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1374: done (3 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1375: done (2 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1376: done (1 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1377: done (0 left) 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1378: rollback oid 6 current snap is 137 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 133 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1379: copy_from oid 24 from oid 2 current snap is 137 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:1380: rollback oid 26 current snap is 137 2026-02-21T22:21:28.442 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 133 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1381: read oid 36 snap -1 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1381: expect deleted 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1382: read oid 18 snap -1 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1382: expect deleted 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1383: write oid 8 current snap is 137 2026-02-21T22:21:28.443 INFO:tasks.rados.rados.0.trial178.stdout:1383: seq_num 493 ranges {685851=544349,1799505=486346} 2026-02-21T22:21:28.449 INFO:tasks.rados.rados.0.trial178.stdout:1383: writing trial17818531-8 from 685851 to 1230200 tid 1 2026-02-21T22:21:28.457 INFO:tasks.rados.rados.0.trial178.stdout:1383: writing trial17818531-8 from 1799505 to 2285851 tid 2 2026-02-21T22:21:28.457 INFO:tasks.rados.rados.0.trial178.stdout:1384: write (excl) oid 25 current snap is 137 2026-02-21T22:21:28.457 INFO:tasks.rados.rados.0.trial178.stdout:1384: seq_num 494 ranges {702658=619598,1857987=444671} 2026-02-21T22:21:28.466 INFO:tasks.rados.rados.0.trial178.stdout:1384: writing trial17818531-25 from 702658 to 1322256 tid 1 2026-02-21T22:21:28.473 INFO:tasks.rados.rados.0.trial178.stdout:1384: writing trial17818531-25 from 1857987 to 2302658 tid 2 2026-02-21T22:21:28.473 INFO:tasks.rados.rados.0.trial178.stdout:1385: write (excl) oid 23 current snap is 137 2026-02-21T22:21:28.473 INFO:tasks.rados.rados.0.trial178.stdout:1385: seq_num 495 ranges {719465=411200,1632822=686643} 2026-02-21T22:21:28.481 INFO:tasks.rados.rados.0.trial178.stdout:1385: writing trial17818531-23 from 719465 to 1130665 tid 1 2026-02-21T22:21:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1385: writing trial17818531-23 from 1632822 to 2319465 tid 2 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1386: snap_create 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1378: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 0 (ObjNum 0 snap 0 seq_num 304) dirty dne 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1380: finishing rollback tid 0 to trial17818531-26 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1380: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1380: finishing rollback tid 2 to trial17818531-26 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 499 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1379: finishing copy_from racing read to trial17818531-24 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1379: finishing copy_from to trial17818531-24 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 496 (ObjNum 425 snap 118 seq_num 425) dirty exists 2026-02-21T22:21:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1383: finishing write tid 1 to trial17818531-8 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: finishing write tid 2 to trial17818531-8 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: finishing write tid 3 to trial17818531-8 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: finishing write tid 4 to trial17818531-8 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: oid 8 updating version 0 to 733 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: oid 8 updating version 733 to 734 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: oid 8 updating version 734 to 735 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 735 (ObjNum 493 snap 137 seq_num 493) dirty exists 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1383: left oid 8 (ObjNum 493 snap 137 seq_num 493) 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1384: finishing write tid 1 to trial17818531-25 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1384: finishing write tid 2 to trial17818531-25 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1384: finishing write tid 3 to trial17818531-25 2026-02-21T22:21:28.490 INFO:tasks.rados.rados.0.trial178.stdout:1384: finishing write tid 4 to trial17818531-25 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:1384: oid 25 updating version 0 to 512 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:1384: oid 25 updating version 512 to 513 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:1384: oid 25 version 513 is already newer than 511 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 513 (ObjNum 494 snap 137 seq_num 494) dirty exists 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:1384: left oid 25 (ObjNum 494 snap 137 seq_num 494) 2026-02-21T22:21:28.491 INFO:tasks.rados.rados.0.trial178.stdout:1385: finishing write tid 1 to trial17818531-23 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: finishing write tid 2 to trial17818531-23 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: finishing write tid 3 to trial17818531-23 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: finishing write tid 4 to trial17818531-23 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: oid 23 updating version 0 to 431 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: oid 23 updating version 431 to 432 2026-02-21T22:21:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1385: oid 23 updating version 432 to 433 2026-02-21T22:21:28.493 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 433 (ObjNum 495 snap 137 seq_num 495) dirty exists 2026-02-21T22:21:28.493 INFO:tasks.rados.rados.0.trial178.stdout:1385: left oid 23 (ObjNum 495 snap 137 seq_num 495) 2026-02-21T22:21:28.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:28.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:29.440 INFO:tasks.rados.rados.0.trial178.stdout:1378: done (8 left) 2026-02-21T22:21:29.440 INFO:tasks.rados.rados.0.trial178.stdout:1379: done (7 left) 2026-02-21T22:21:29.440 INFO:tasks.rados.rados.0.trial178.stdout:1380: done (6 left) 2026-02-21T22:21:29.440 INFO:tasks.rados.rados.0.trial178.stdout:1381: done (5 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1382: done (4 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1383: done (3 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1384: done (2 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1385: done (1 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1386: done (0 left) 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1387: copy_from oid 27 from oid 16 current snap is 138 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1388: write (excl) oid 5 current snap is 138 2026-02-21T22:21:29.441 INFO:tasks.rados.rados.0.trial178.stdout:1388: seq_num 496 ranges {736272=486449,1974951=361321} 2026-02-21T22:21:29.445 INFO:tasks.rados.rados.0.trial178.stdout:1388: writing trial17818531-5 from 736272 to 1222721 tid 1 2026-02-21T22:21:29.449 INFO:tasks.rados.rados.0.trial178.stdout:1388: writing trial17818531-5 from 1974951 to 2336272 tid 2 2026-02-21T22:21:29.449 INFO:tasks.rados.rados.0.trial178.stdout:1389: snap_create 2026-02-21T22:21:29.449 INFO:tasks.rados.rados.0.trial178.stdout:1388: finishing write tid 1 to trial17818531-5 2026-02-21T22:21:29.451 INFO:tasks.rados.rados.0.trial178.stdout:1388: finishing write tid 2 to trial17818531-5 2026-02-21T22:21:29.451 INFO:tasks.rados.rados.0.trial178.stdout:1388: finishing write tid 3 to trial17818531-5 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:1388: finishing write tid 4 to trial17818531-5 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:1388: oid 5 updating version 0 to 541 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:1388: oid 5 version 541 is already newer than 540 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:1388: oid 5 version 541 is already newer than 539 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 541 (ObjNum 496 snap 138 seq_num 496) dirty exists 2026-02-21T22:21:29.452 INFO:tasks.rados.rados.0.trial178.stdout:1388: left oid 5 (ObjNum 496 snap 138 seq_num 496) 2026-02-21T22:21:29.459 INFO:tasks.rados.rados.0.trial178.stdout:1387: finishing copy_from racing read to trial17818531-27 2026-02-21T22:21:29.459 INFO:tasks.rados.rados.0.trial178.stdout:1387: finishing copy_from to trial17818531-27 2026-02-21T22:21:29.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 358 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1387: done (2 left) 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1388: done (1 left) 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1389: done (0 left) 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1390: rollback oid 46 current snap is 139 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 136 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1391: copy_from oid 5 from oid 47 current snap is 139 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1392: write oid 8 current snap is 139 2026-02-21T22:21:30.443 INFO:tasks.rados.rados.0.trial178.stdout:1392: seq_num 497 ranges {753079=561698,2033433=319646} 2026-02-21T22:21:30.452 INFO:tasks.rados.rados.0.trial178.stdout:1392: writing trial17818531-8 from 753079 to 1314777 tid 1 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1392: writing trial17818531-8 from 2033433 to 2353079 tid 2 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1393: rollback oid 48 current snap is 139 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 136 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1390: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1392: finishing write tid 1 to trial17818531-8 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1391: finishing copy_from racing read to trial17818531-5 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1391: finishing copy_from to trial17818531-5 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 731 (ObjNum 492 snap 136 seq_num 492) dirty exists 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1390: done (3 left) 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1391: done (2 left) 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1394: write oid 4 current snap is 139 2026-02-21T22:21:30.460 INFO:tasks.rados.rados.0.trial178.stdout:1394: seq_num 498 ranges {769886=636947,2091915=277971} 2026-02-21T22:21:30.469 INFO:tasks.rados.rados.0.trial178.stdout:1394: writing trial17818531-4 from 769886 to 1406833 tid 1 2026-02-21T22:21:30.476 INFO:tasks.rados.rados.0.trial178.stdout:1394: writing trial17818531-4 from 2091915 to 2369886 tid 2 2026-02-21T22:21:30.476 INFO:tasks.rados.rados.0.trial178.stdout:1395: rollback oid 21 current snap is 139 2026-02-21T22:21:30.476 INFO:tasks.rados.rados.0.trial178.stdout:1392: finishing write tid 2 to trial17818531-8 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: finishing write tid 3 to trial17818531-8 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: finishing write tid 4 to trial17818531-8 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: oid 8 updating version 0 to 739 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: oid 8 version 739 is already newer than 738 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: oid 8 version 739 is already newer than 737 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 739 (ObjNum 497 snap 139 seq_num 497) dirty exists 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1392: left oid 8 (ObjNum 497 snap 139 seq_num 497) 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1393: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1393: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1393: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:21:30.477 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 501 (ObjNum 424 snap 117 seq_num 424) dirty exists 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1394: finishing write tid 1 to trial17818531-4 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 137 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1392: done (3 left) 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1393: done (2 left) 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1396: copy_from oid 20 from oid 27 current snap is 139 2026-02-21T22:21:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1397: snap_create 2026-02-21T22:21:30.479 INFO:tasks.rados.rados.0.trial178.stdout:1394: finishing write tid 2 to trial17818531-4 2026-02-21T22:21:30.479 INFO:tasks.rados.rados.0.trial178.stdout:1394: finishing write tid 3 to trial17818531-4 2026-02-21T22:21:30.479 INFO:tasks.rados.rados.0.trial178.stdout:1394: finishing write tid 4 to trial17818531-4 2026-02-21T22:21:30.480 INFO:tasks.rados.rados.0.trial178.stdout:1394: oid 4 updating version 0 to 362 2026-02-21T22:21:30.480 INFO:tasks.rados.rados.0.trial178.stdout:1394: oid 4 updating version 362 to 364 2026-02-21T22:21:30.480 INFO:tasks.rados.rados.0.trial178.stdout:1394: oid 4 version 364 is already newer than 363 2026-02-21T22:21:30.480 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 364 (ObjNum 498 snap 139 seq_num 498) dirty exists 2026-02-21T22:21:30.480 INFO:tasks.rados.rados.0.trial178.stdout:1394: left oid 4 (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:21:30.483 INFO:tasks.rados.rados.0.trial178.stdout:1395: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:21:30.483 INFO:tasks.rados.rados.0.trial178.stdout:1395: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:21:30.483 INFO:tasks.rados.rados.0.trial178.stdout:1395: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:21:30.483 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 743 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:21:30.494 INFO:tasks.rados.rados.0.trial178.stdout:1396: finishing copy_from racing read to trial17818531-20 2026-02-21T22:21:30.494 INFO:tasks.rados.rados.0.trial178.stdout:1396: finishing copy_from to trial17818531-20 2026-02-21T22:21:30.495 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 358 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:21:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1394: done (3 left) 2026-02-21T22:21:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1395: done (2 left) 2026-02-21T22:21:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1396: done (1 left) 2026-02-21T22:21:31.447 INFO:tasks.rados.rados.0.trial178.stdout:1397: done (0 left) 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:1398: rollback oid 42 current snap is 140 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 133 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:1399: read oid 38 snap -1 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:1399: expect (ObjNum 470 snap 133 seq_num 470) 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:1400: copy_from oid 47 from oid 4 current snap is 140 2026-02-21T22:21:31.448 INFO:tasks.rados.rados.0.trial178.stdout:1401: delete oid 29 current snap is 140 2026-02-21T22:21:31.450 INFO:tasks.rados.rados.0.trial178.stdout:1401: done (3 left) 2026-02-21T22:21:31.450 INFO:tasks.rados.rados.0.trial178.stdout:1402: rollback oid 36 current snap is 140 2026-02-21T22:21:31.450 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 135 2026-02-21T22:21:31.450 INFO:tasks.rados.rados.0.trial178.stdout:1403: write (excl) oid 37 current snap is 140 2026-02-21T22:21:31.450 INFO:tasks.rados.rados.0.trial178.stdout:1403: seq_num 500 ranges {403500=787445,1692526=473837} 2026-02-21T22:21:31.458 INFO:tasks.rados.rados.0.trial178.stdout:1403: writing trial17818531-37 from 403500 to 1190945 tid 1 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:1403: writing trial17818531-37 from 1692526 to 2166363 tid 2 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:1404: delete oid 24 current snap is 140 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:1398: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:1398: finishing rollback tid 2 to trial17818531-42 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 745 (ObjNum 398 snap 111 seq_num 398) dirty exists 2026-02-21T22:21:31.465 INFO:tasks.rados.rados.0.trial178.stdout:1402: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:21:31.466 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 175 (ObjNum 33 snap 0 seq_num 101031200) dirty dne 2026-02-21T22:21:31.466 INFO:tasks.rados.rados.0.trial178.stdout:1399: expect (ObjNum 470 snap 133 seq_num 470) 2026-02-21T22:21:31.466 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:31.536 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1400: finishing copy_from racing read to trial17818531-47 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1403: finishing write tid 1 to trial17818531-37 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1403: finishing write tid 2 to trial17818531-37 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1403: finishing write tid 3 to trial17818531-37 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1403: finishing write tid 4 to trial17818531-37 2026-02-21T22:21:31.644 INFO:tasks.rados.rados.0.trial178.stdout:1403: oid 37 updating version 0 to 503 2026-02-21T22:21:31.645 INFO:tasks.rados.rados.0.trial178.stdout:1403: oid 37 version 503 is already newer than 502 2026-02-21T22:21:31.645 INFO:tasks.rados.rados.0.trial178.stdout:1403: oid 37 updating version 503 to 504 2026-02-21T22:21:31.645 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 504 (ObjNum 500 snap 140 seq_num 500) dirty exists 2026-02-21T22:21:31.645 INFO:tasks.rados.rados.0.trial178.stdout:1403: left oid 37 (ObjNum 500 snap 140 seq_num 500) 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1400: finishing copy_from to trial17818531-47 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 364 (ObjNum 498 snap 139 seq_num 498) dirty exists 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1398: done (5 left) 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1399: done (4 left) 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1400: done (3 left) 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1402: done (2 left) 2026-02-21T22:21:31.675 INFO:tasks.rados.rados.0.trial178.stdout:1403: done (1 left) 2026-02-21T22:21:31.676 INFO:tasks.rados.rados.0.trial178.stdout:1404: done (0 left) 2026-02-21T22:21:31.676 INFO:tasks.rados.rados.0.trial178.stdout:1405: snap_create 2026-02-21T22:21:32.422 INFO:tasks.rados.rados.0.trial178.stdout:1405: done (0 left) 2026-02-21T22:21:32.422 INFO:tasks.rados.rados.0.trial178.stdout:1406: snap_create 2026-02-21T22:21:33.422 INFO:tasks.rados.rados.0.trial178.stdout:1406: done (0 left) 2026-02-21T22:21:33.423 INFO:tasks.rados.rados.0.trial178.stdout:1407: write (excl) oid 38 current snap is 142 2026-02-21T22:21:33.423 INFO:tasks.rados.rados.0.trial178.stdout:1407: seq_num 502 ranges {437114=654296,1525843=477506} 2026-02-21T22:21:33.430 INFO:tasks.rados.rados.0.trial178.stdout:1407: writing trial17818531-38 from 437114 to 1091410 tid 1 2026-02-21T22:21:33.436 INFO:tasks.rados.rados.0.trial178.stdout:1407: writing trial17818531-38 from 1525843 to 2003349 tid 2 2026-02-21T22:21:33.436 INFO:tasks.rados.rados.0.trial178.stdout:1408: snap_create 2026-02-21T22:21:33.437 INFO:tasks.rados.rados.0.trial178.stdout:1407: finishing write tid 1 to trial17818531-38 2026-02-21T22:21:33.439 INFO:tasks.rados.rados.0.trial178.stdout:1407: finishing write tid 2 to trial17818531-38 2026-02-21T22:21:33.439 INFO:tasks.rados.rados.0.trial178.stdout:1407: finishing write tid 3 to trial17818531-38 2026-02-21T22:21:33.439 INFO:tasks.rados.rados.0.trial178.stdout:1407: finishing write tid 4 to trial17818531-38 2026-02-21T22:21:33.439 INFO:tasks.rados.rados.0.trial178.stdout:1407: oid 38 updating version 0 to 388 2026-02-21T22:21:33.439 INFO:tasks.rados.rados.0.trial178.stdout:1407: oid 38 version 388 is already newer than 387 2026-02-21T22:21:33.440 INFO:tasks.rados.rados.0.trial178.stdout:1407: oid 38 version 388 is already newer than 386 2026-02-21T22:21:33.440 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 388 (ObjNum 502 snap 142 seq_num 502) dirty exists 2026-02-21T22:21:33.440 INFO:tasks.rados.rados.0.trial178.stdout:1407: left oid 38 (ObjNum 502 snap 142 seq_num 502) 2026-02-21T22:21:33.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:33.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:34.423 INFO:tasks.rados.rados.0.trial178.stdout:1407: done (1 left) 2026-02-21T22:21:34.423 INFO:tasks.rados.rados.0.trial178.stdout:1408: done (0 left) 2026-02-21T22:21:34.424 INFO:tasks.rados.rados.0.trial178.stdout:1409: delete oid 29 current snap is 143 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1409: done (0 left) 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1410: read oid 34 snap 141 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1410: expect (ObjNum 448 snap 130 seq_num 448) 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1411: read oid 3 snap -1 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1411: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:34.427 INFO:tasks.rados.rados.0.trial178.stdout:1412: read oid 1 snap -1 2026-02-21T22:21:34.428 INFO:tasks.rados.rados.0.trial178.stdout:1412: expect (ObjNum 331 snap 95 seq_num 331) 2026-02-21T22:21:34.428 INFO:tasks.rados.rados.0.trial178.stdout:1413: write oid 19 current snap is 143 2026-02-21T22:21:34.428 INFO:tasks.rados.rados.0.trial178.stdout:1413: seq_num 504 ranges {470728=404794,1526454=481175} 2026-02-21T22:21:34.434 INFO:tasks.rados.rados.0.trial178.stdout:1413: writing trial17818531-19 from 470728 to 875522 tid 1 2026-02-21T22:21:34.443 INFO:tasks.rados.rados.0.trial178.stdout:1413: writing trial17818531-19 from 1526454 to 2007629 tid 2 2026-02-21T22:21:34.443 INFO:tasks.rados.rados.0.trial178.stdout:1414: read oid 36 snap 136 2026-02-21T22:21:34.443 INFO:tasks.rados.rados.0.trial178.stdout:1414: expect deleted 2026-02-21T22:21:34.443 INFO:tasks.rados.rados.0.trial178.stdout:1415: write oid 48 current snap is 143 2026-02-21T22:21:34.443 INFO:tasks.rados.rados.0.trial178.stdout:1415: seq_num 505 ranges {487535=480043,1584936=624833} 2026-02-21T22:21:34.449 INFO:tasks.rados.rados.0.trial178.stdout:1415: writing trial17818531-48 from 487535 to 967578 tid 1 2026-02-21T22:21:34.458 INFO:tasks.rados.rados.0.trial178.stdout:1415: writing trial17818531-48 from 1584936 to 2209769 tid 2 2026-02-21T22:21:34.458 INFO:tasks.rados.rados.0.trial178.stdout:1416: read oid 40 snap -1 2026-02-21T22:21:34.458 INFO:tasks.rados.rados.0.trial178.stdout:1416: expect (ObjNum 424 snap 117 seq_num 424) 2026-02-21T22:21:34.458 INFO:tasks.rados.rados.0.trial178.stdout:1417: read oid 10 snap -1 2026-02-21T22:21:34.458 INFO:tasks.rados.rados.0.trial178.stdout:1417: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:21:34.459 INFO:tasks.rados.rados.0.trial178.stdout:1418: write oid 46 current snap is 143 2026-02-21T22:21:34.459 INFO:tasks.rados.rados.0.trial178.stdout:1418: seq_num 506 ranges {504342=555292,1643418=484844} 2026-02-21T22:21:34.465 INFO:tasks.rados.rados.0.trial178.stdout:1418: writing trial17818531-46 from 504342 to 1059634 tid 1 2026-02-21T22:21:34.473 INFO:tasks.rados.rados.0.trial178.stdout:1418: writing trial17818531-46 from 1643418 to 2128262 tid 2 2026-02-21T22:21:34.473 INFO:tasks.rados.rados.0.trial178.stdout:1419: read oid 43 snap -1 2026-02-21T22:21:34.473 INFO:tasks.rados.rados.0.trial178.stdout:1419: expect deleted 2026-02-21T22:21:34.473 INFO:tasks.rados.rados.0.trial178.stdout:1420: copy_from oid 14 from oid 18 current snap is 143 2026-02-21T22:21:34.473 INFO:tasks.rados.rados.0.trial178.stdout:1410: expect (ObjNum 448 snap 130 seq_num 448) 2026-02-21T22:21:34.563 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3415283 2026-02-21T22:21:34.614 INFO:tasks.rados.rados.0.trial178.stdout:1411: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:34.635 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2530726 2026-02-21T22:21:34.649 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 266626 2026-02-21T22:21:34.720 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3519770 2026-02-21T22:21:34.724 INFO:tasks.rados.rados.0.trial178.stdout:1412: expect (ObjNum 331 snap 95 seq_num 331) 2026-02-21T22:21:34.724 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:34.799 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1077566 2026-02-21T22:21:34.897 INFO:tasks.rados.rados.0.trial178.stdout:1413: finishing write tid 1 to trial17818531-19 2026-02-21T22:21:34.898 INFO:tasks.rados.rados.0.trial178.stdout:1410: done (10 left) 2026-02-21T22:21:34.898 INFO:tasks.rados.rados.0.trial178.stdout:1411: done (9 left) 2026-02-21T22:21:34.898 INFO:tasks.rados.rados.0.trial178.stdout:1412: done (8 left) 2026-02-21T22:21:34.898 INFO:tasks.rados.rados.0.trial178.stdout:1414: done (7 left) 2026-02-21T22:21:34.898 INFO:tasks.rados.rados.0.trial178.stdout:1421: snap_remove snap 133 2026-02-21T22:21:35.566 INFO:tasks.rados.rados.0.trial178.stdout:1421: done (7 left) 2026-02-21T22:21:35.566 INFO:tasks.rados.rados.0.trial178.stdout:1422: snap_create 2026-02-21T22:21:35.566 INFO:tasks.rados.rados.0.trial178.stdout:1413: finishing write tid 2 to trial17818531-19 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: finishing write tid 3 to trial17818531-19 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: finishing write tid 4 to trial17818531-19 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: oid 19 updating version 0 to 751 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: oid 19 updating version 751 to 752 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: oid 19 version 752 is already newer than 750 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 752 (ObjNum 504 snap 143 seq_num 504) dirty exists 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1413: left oid 19 (ObjNum 504 snap 143 seq_num 504) 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1415: finishing write tid 1 to trial17818531-48 2026-02-21T22:21:35.567 INFO:tasks.rados.rados.0.trial178.stdout:1416: expect (ObjNum 424 snap 117 seq_num 424) 2026-02-21T22:21:35.712 INFO:tasks.rados.rados.0.trial178.stdout:1415: finishing write tid 2 to trial17818531-48 2026-02-21T22:21:35.712 INFO:tasks.rados.rados.0.trial178.stdout:1415: finishing write tid 3 to trial17818531-48 2026-02-21T22:21:35.712 INFO:tasks.rados.rados.0.trial178.stdout:1415: finishing write tid 4 to trial17818531-48 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:1415: oid 48 updating version 0 to 505 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:1415: oid 48 updating version 505 to 507 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:1415: oid 48 version 507 is already newer than 506 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 507 (ObjNum 505 snap 143 seq_num 505) dirty exists 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:1415: left oid 48 (ObjNum 505 snap 143 seq_num 505) 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:1417: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:21:35.713 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:35.869 INFO:tasks.rados.rados.0.trial178.stdout:1418: finishing write tid 1 to trial17818531-46 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1420: finishing copy_from to trial17818531-14 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1420: got expected ENOENT (src dne) 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1420: finishing copy_from racing read to trial17818531-14 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: finishing write tid 2 to trial17818531-46 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: finishing write tid 3 to trial17818531-46 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: finishing write tid 4 to trial17818531-46 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: oid 46 updating version 0 to 436 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: oid 46 version 436 is already newer than 434 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:1418: oid 46 version 436 is already newer than 435 2026-02-21T22:21:35.870 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 436 (ObjNum 506 snap 143 seq_num 506) dirty exists 2026-02-21T22:21:35.871 INFO:tasks.rados.rados.0.trial178.stdout:1418: left oid 46 (ObjNum 506 snap 143 seq_num 506) 2026-02-21T22:21:36.312 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:21:36.312 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:21:36.312 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:21:36.312 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:21:36.313 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:21:36.313 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:21:36.313 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:21:36.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: get_monmap_and_config 2026-02-21T22:21:36.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: monmap: 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:36.395364+0000 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:36.395364+0000 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 1 Processor -- start 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 1 -- start start 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: init 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-b con 0x7f5d6c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-a con 0x7f5d6c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-c con 0x7f5d6c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c086a10 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c08b130 con 0x7f5d6c08d8e0 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c08bd60 con 0x7f5d6c08ce00 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d72523640 10 monclient(hunting): authenticate will time out at 1505.631665s 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d6bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08ce00 0x7f5d6c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44722/0 (socket says 10.20.193.102:44722) 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d6b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5d6c08b7f0 0x7f5d6c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54912/0 (socket says 10.20.193.102:54912) 2026-02-21T22:21:36.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.389+0000 7f5d6bfff640 1 -- 10.20.193.102:0/3184240050 learned_addr learned my addr 10.20.193.102:0/3184240050 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.393+0000 7f5d6b7fe640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.102:6789/0 conn(0x7f5d6c08b7f0 legacy=0x7f5d6c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.393+0000 7f5d6bfff640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08ce00 legacy=0x7f5d6c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.393+0000 7f5d70a99640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.178:6789/0 conn(0x7f5d6c08d8e0 legacy=0x7f5d6c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6192721 0 0) 0x7f5d6c086a10 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient(hunting): my global_id is 34278 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d50003620 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720359720 0 0) 0x7f5d50003620 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08ce00 legacy=0x7f5d6c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.178:6789/0 conn(0x7f5d6c08d8e0 legacy=0x7f5d6c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 monclient: found mon.noname-a 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c08b370 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d5c003110 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 5 monclient: authenticate success, global_id 34278 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: dump: 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.413 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5d5c0033e0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 1 -- 10.20.193.102:0/3184240050 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d5c004a90 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6affd640 10 monclient: dump: 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:36.414 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 4 set_mon_vals no callback set 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: shutdown 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- 10.20.193.102:0/3184240050 >> v1:10.20.193.102:6789/0 conn(0x7f5d6c08b7f0 legacy=0x7f5d6c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- 10.20.193.102:0/3184240050 shutdown_connections 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- 10.20.193.102:0/3184240050 >> 10.20.193.102:0/3184240050 conn(0x7f5d6c091a50 msgr2=0x7f5d6c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- 10.20.193.102:0/3184240050 shutdown_connections 2026-02-21T22:21:36.415 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- 10.20.193.102:0/3184240050 wait complete. 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: monmap: 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 Processor -- start 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- start start 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: init 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-c con 0x7f5d6c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-b con 0x7f5d6c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient(hunting): picked mon.noname-a con 0x7f5d6c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c081560 con 0x7f5d6c08d8e0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c058a40 con 0x7f5d6c08ce00 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6c05e150 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d72523640 10 monclient(hunting): authenticate will time out at 1505.651523s 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08b7f0 0x7f5d6c0849b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44728/0 (socket says 10.20.193.102:44728) 2026-02-21T22:21:36.416 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.409+0000 7f5d6bfff640 1 -- 10.20.193.102:0/843846786 learned_addr learned my addr 10.20.193.102:0/843846786 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d6bfff640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08b7f0 legacy=0x7f5d6c0849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d70a99640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.102:6789/0 conn(0x7f5d6c08d8e0 legacy=0x7f5d6c193d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d6b7fe640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.178:6789/0 conn(0x7f5d6c08ce00 legacy=0x7f5d6c192630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485818943 0 0) 0x7f5d6c05e150 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): my global_id is 4640 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d3c003620 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3432821179 0 0) 0x7f5d3c003620 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5d6c05e150 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d58003330 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:21:36.417 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314862299 0 0) 0x7f5d6c081560 con 0x7f5d6c08d8e0 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient(hunting): my global_id is 34281 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d3c003620 con 0x7f5d6c08d8e0 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2062771239 0 0) 0x7f5d6c05e150 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.178:6789/0 conn(0x7f5d6c08ce00 legacy=0x7f5d6c192630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.102:6789/0 conn(0x7f5d6c08d8e0 legacy=0x7f5d6c193d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 monclient: found mon.c 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c080fa0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 5 monclient: authenticate success, global_id 4640 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 10 monclient: _renew_subs 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c081280 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 10 monclient: _renew_subs 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5d6c0850c0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5d58004b90 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d58004ef0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d68ff9640 10 monclient: dump: 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.413+0000 7f5d6a7fc640 4 set_mon_vals no callback set 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d72523640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d72523640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d72523640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c0008e0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5d580051c0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(300..300 src has 112..300) ==== 3927+0+0 (unknown 594352288 0 0) 0x7f5d5804c770 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.417+0000 7f5d6b7fe640 10 monclient: get_auth_request con 0x7f5d3c03d590 auth_method 0 2026-02-21T22:21:36.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.421+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5d580176b0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.421+0000 7f5d68ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.421+0000 7f5d68ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.533+0000 7f5d72523640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.533+0000 7f5d72523640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.533+0000 7f5d72523640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.533+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5d6c090810 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d68ff9640 1 -- 10.20.193.102:0/843846786 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 595850632) 0x7f5d5801c9d0 con 0x7f5d6c08b7f0 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d68ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d68ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:36.539 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5d3c03d590 legacy=0x7f5d3c05da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 10 monclient: shutdown 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 >> v1:10.20.193.102:6790/0 conn(0x7f5d6c08b7f0 legacy=0x7f5d6c0849b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 shutdown_connections 2026-02-21T22:21:36.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 >> 10.20.193.102:0/843846786 conn(0x7f5d6c091a50 msgr2=0x7f5d6c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 shutdown_connections 2026-02-21T22:21:36.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.537+0000 7f5d72523640 1 -- 10.20.193.102:0/843846786 wait complete. 2026-02-21T22:21:36.550 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:21:36.550 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:21:36.550 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:21:36.571 INFO:tasks.rados.rados.0.trial178.stdout:1413: done (7 left) 2026-02-21T22:21:36.571 INFO:tasks.rados.rados.0.trial178.stdout:1415: done (6 left) 2026-02-21T22:21:36.571 INFO:tasks.rados.rados.0.trial178.stdout:1416: done (5 left) 2026-02-21T22:21:36.571 INFO:tasks.rados.rados.0.trial178.stdout:1417: done (4 left) 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1418: done (3 left) 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1419: done (2 left) 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1420: done (1 left) 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1422: done (0 left) 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1423: copy_from oid 32 from oid 14 current snap is 144 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1424: rollback oid 27 current snap is 144 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 138 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1425: write oid 15 current snap is 144 2026-02-21T22:21:36.572 INFO:tasks.rados.rados.0.trial178.stdout:1425: seq_num 507 ranges {521149=630541,1701900=628502} 2026-02-21T22:21:36.578 INFO:tasks.rados.rados.0.trial178.stdout:1425: writing trial17818531-15 from 521149 to 1151690 tid 1 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1425: writing trial17818531-15 from 1701900 to 2330402 tid 2 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1426: read oid 44 snap -1 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1426: expect (ObjNum 485 snap 134 seq_num 485) 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1427: delete oid 7 current snap is 144 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1424: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1424: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:1424: finishing rollback tid 2 to trial17818531-27 2026-02-21T22:21:36.587 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 478 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:21:36.588 INFO:tasks.rados.rados.0.trial178.stdout:1425: finishing write tid 1 to trial17818531-15 2026-02-21T22:21:36.588 INFO:tasks.rados.rados.0.trial178.stdout:1423: finishing copy_from racing read to trial17818531-32 2026-02-21T22:21:36.589 INFO:tasks.rados.rados.0.trial178.stdout:1423: finishing copy_from to trial17818531-32 2026-02-21T22:21:36.589 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 466 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:21:36.589 INFO:tasks.rados.rados.0.trial178.stdout:1423: done (4 left) 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1424: done (3 left) 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1427: done (2 left) 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1428: copy_from oid 2 from oid 11 current snap is 144 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1429: read oid 14 snap 141 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1429: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:36.590 INFO:tasks.rados.rados.0.trial178.stdout:1430: snap_remove snap 139 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: get_monmap_and_config 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: monmap: 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:36.614807+0000 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:36.614807+0000 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:36.613 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 1 Processor -- start 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 1 -- start start 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: init 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-c con 0x7f1c6013fb80 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-a con 0x7f1c6013ff50 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-b con 0x7f1c601855d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c60093e30 con 0x7f1c6013ff50 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c60092470 con 0x7f1c6013fb80 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c60056470 con 0x7f1c601855d0 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c65226640 10 monclient(hunting): authenticate will time out at 1505.851186s 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c5e575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1c6013ff50 0x7f1c60183eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54918/0 (socket says 10.20.193.102:54918) 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c5e575640 1 -- 10.20.193.102:0/1004632071 learned_addr learned my addr 10.20.193.102:0/1004632071 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.609+0000 7f1c5f577640 1 --1- 10.20.193.102:0/1004632071 >> v1:10.20.193.102:6790/0 conn(0x7f1c601855d0 0x7f1c601a59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44744/0 (socket says 10.20.193.102:44744) 2026-02-21T22:21:36.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5f577640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.102:6790/0 conn(0x7f1c601855d0 legacy=0x7f1c601a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5e575640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.102:6789/0 conn(0x7f1c6013ff50 legacy=0x7f1c60183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870422190 0 0) 0x7f1c60056470 con 0x7f1c601855d0 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): my global_id is 4643 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c4c003620 con 0x7f1c601855d0 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196863398 0 0) 0x7f1c60093e30 con 0x7f1c6013ff50 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient(hunting): my global_id is 34284 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c60056470 con 0x7f1c6013ff50 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5ed76640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.178:6789/0 conn(0x7f1c6013fb80 legacy=0x7f1c60058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2066762663 0 0) 0x7f1c4c003620 con 0x7f1c601855d0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.178:6789/0 conn(0x7f1c6013fb80 legacy=0x7f1c60058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.102:6789/0 conn(0x7f1c6013ff50 legacy=0x7f1c60183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 monclient: found mon.noname-b 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c60057510 con 0x7f1c601855d0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 5 monclient: authenticate success, global_id 4643 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c500046a0 con 0x7f1c601855d0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: dump: 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c500048b0 con 0x7f1c601855d0 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 1 -- 10.20.193.102:0/1004632071 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c50005fa0 con 0x7f1c601855d0 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5dd74640 10 monclient: dump: 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 4 set_mon_vals no callback set 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: shutdown 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- 10.20.193.102:0/1004632071 >> v1:10.20.193.102:6790/0 conn(0x7f1c601855d0 legacy=0x7f1c601a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- 10.20.193.102:0/1004632071 shutdown_connections 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- 10.20.193.102:0/1004632071 >> 10.20.193.102:0/1004632071 conn(0x7f1c60096bc0 msgr2=0x7f1c60092a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- 10.20.193.102:0/1004632071 shutdown_connections 2026-02-21T22:21:36.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- 10.20.193.102:0/1004632071 wait complete. 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: monmap: 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 Processor -- start 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- start start 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: init 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-a con 0x7f1c6013fb80 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-b con 0x7f1c6013ff50 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient(hunting): picked mon.noname-c con 0x7f1c601855d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c60128b00 con 0x7f1c6013fb80 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c60059240 con 0x7f1c6013ff50 2026-02-21T22:21:36.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c6008e080 con 0x7f1c601855d0 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c65226640 10 monclient(hunting): authenticate will time out at 1505.855471s 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5ed76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1c6013fb80 0x7f1c60127ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54924/0 (socket says 10.20.193.102:54924) 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.613+0000 7f1c5ed76640 1 -- 10.20.193.102:0/2446768311 learned_addr learned my addr 10.20.193.102:0/2446768311 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c5ed76640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6013fb80 legacy=0x7f1c60127ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c5f577640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6790/0 conn(0x7f1c601855d0 legacy=0x7f1c60184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1987428462 0 0) 0x7f1c60128b00 con 0x7f1c6013fb80 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): my global_id is 34287 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c2c003620 con 0x7f1c6013fb80 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:21:36.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c5e575640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.178:6789/0 conn(0x7f1c6013ff50 legacy=0x7f1c601283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 38435116 0 0) 0x7f1c6008e080 con 0x7f1c601855d0 2026-02-21T22:21:36.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 10 monclient(hunting): my global_id is 4646 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c60128b00 con 0x7f1c601855d0 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.617+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2923035638 0 0) 0x7f1c2c003620 con 0x7f1c6013fb80 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1c6008e080 con 0x7f1c6013fb80 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684190549 0 0) 0x7f1c60059240 con 0x7f1c6013ff50 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): my global_id is 54100 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c2c003620 con 0x7f1c6013ff50 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c54003420 con 0x7f1c6013fb80 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): dump: 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1581977159 0 0) 0x7f1c6008e080 con 0x7f1c6013fb80 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6790/0 conn(0x7f1c601855d0 legacy=0x7f1c60184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.178:6789/0 conn(0x7f1c6013ff50 legacy=0x7f1c601283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 monclient: found mon.a 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c601853a0 con 0x7f1c6013fb80 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 5 monclient: authenticate success, global_id 34287 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 10 monclient: _renew_subs 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6010cb40 con 0x7f1c6013fb80 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 10 monclient: _renew_subs 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1c6008fb40 con 0x7f1c6013fb80 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1c540039c0 con 0x7f1c6013fb80 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c54003d50 con 0x7f1c6013fb80 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: dump: 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c5d573640 4 set_mon_vals no callback set 2026-02-21T22:21:36.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1c6008f7b0 con 0x7f1c6008ebe0 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1c54003fe0 con 0x7f1c6013fb80 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(301..301 src has 112..301) ==== 3899+0+0 (unknown 3142247521 0 0) 0x7f1c5404c680 con 0x7f1c6013fb80 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c5f577640 10 monclient: get_auth_request con 0x7f1c2c03d6f0 auth_method 0 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c5e575640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6008ebe0 legacy=0x7f1c60088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084101745 0 0) 0x7f1c6008f7b0 con 0x7f1c6008ebe0 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient(hunting): my global_id is 34290 2026-02-21T22:21:36.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c6008e080 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 329357742 0 0) 0x7f1c6008e080 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1c6008f7b0 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c48003310 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: dump: 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 130929780 0 0) 0x7f1c6008f7b0 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c60095010 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f1c48002c60 con 0x7f1c6008ebe0 2026-02-21T22:21:36.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:36.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6008ebe0 legacy=0x7f1c60088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.621+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6008ebe0 legacy=0x7f1c60088de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.625+0000 7f1c65226640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:36.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.625+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1c60051070 con 0x7f1c6012b260 2026-02-21T22:21:36.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c5e575640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6012b260 legacy=0x7f1c600894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196504966 0 0) 0x7f1c60051070 con 0x7f1c6012b260 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient(hunting): my global_id is 34293 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c6008f7b0 con 0x7f1c6012b260 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1339740563 0 0) 0x7f1c6008f7b0 con 0x7f1c6012b260 2026-02-21T22:21:36.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1c60051070 con 0x7f1c6012b260 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1c48004d10 con 0x7f1c6012b260 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient: dump: 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.634 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.635 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.635 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3157201453 0 0) 0x7f1c60051070 con 0x7f1c6012b260 2026-02-21T22:21:36.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1c601403b0 con 0x7f1c6012b260 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 1876336618) 0x7f1c48007050 con 0x7f1c6012b260 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6012b260 legacy=0x7f1c600894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c3f7fe640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6012b260 legacy=0x7f1c600894f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:36.636 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 245499, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.637 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.638 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.639 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:36.640 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.641 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.642 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.643 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.644 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:36.645 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:36.646 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.647 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:36.648 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:36.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.178:6800/2549949439 conn(0x7f1c2c03d6f0 legacy=0x7f1c2c05dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 10 monclient: shutdown 2026-02-21T22:21:36.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 >> v1:10.20.193.102:6789/0 conn(0x7f1c6013fb80 legacy=0x7f1c60127ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c5f577640 1 -- 10.20.193.102:0/2446768311 reap_dead start 2026-02-21T22:21:36.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c5f577640 1 -- 10.20.193.102:0/2446768311 reap_dead start 2026-02-21T22:21:36.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 shutdown_connections 2026-02-21T22:21:36.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 >> 10.20.193.102:0/2446768311 conn(0x7f1c60096bc0 msgr2=0x7f1c60141c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 shutdown_connections 2026-02-21T22:21:36.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.629+0000 7f1c65226640 1 -- 10.20.193.102:0/2446768311 wait complete. 2026-02-21T22:21:36.650 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:36.650 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:21:36.650 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:21:36.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: get_monmap_and_config 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: monmap: 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:36.714342+0000 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:36.714342+0000 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:36.713 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 1 Processor -- start 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 1 -- start start 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: init 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-b con 0x7fbeb808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-a con 0x7fbeb808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-c con 0x7fbeb808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb8086a10 con 0x7fbeb808b7f0 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb808b130 con 0x7fbeb808d8e0 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb808bd60 con 0x7fbeb808ce00 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbebef46640 10 monclient(hunting): authenticate will time out at 1505.950214s 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbeaffff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbeb808b7f0 0x7fbeb8181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54956/0 (socket says 10.20.193.102:54956) 2026-02-21T22:21:36.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.709+0000 7fbeaffff640 1 -- 10.20.193.102:0/1094387364 learned_addr learned my addr 10.20.193.102:0/1094387364 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaffff640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.102:6789/0 conn(0x7fbeb808b7f0 legacy=0x7fbeb8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebccbb640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.102:6790/0 conn(0x7fbeb808ce00 legacy=0x7fbeb808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669019942 0 0) 0x7fbeb8086a10 con 0x7fbeb808b7f0 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): my global_id is 34296 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbea4003620 con 0x7fbeb808b7f0 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2864099239 0 0) 0x7fbeb808bd60 con 0x7fbeb808ce00 2026-02-21T22:21:36.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient(hunting): my global_id is 4649 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb8086a10 con 0x7fbeb808ce00 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebd4bc640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.178:6789/0 conn(0x7fbeb808d8e0 legacy=0x7fbeb81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4150383014 0 0) 0x7fbeb8086a10 con 0x7fbeb808ce00 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.178:6789/0 conn(0x7fbeb808d8e0 legacy=0x7fbeb81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.102:6789/0 conn(0x7fbeb808b7f0 legacy=0x7fbeb8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 monclient: found mon.noname-b 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb808b370 con 0x7fbeb808ce00 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea8003160 con 0x7fbeb808ce00 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: dump: 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.717 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 5 monclient: authenticate success, global_id 4649 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbea8003430 con 0x7fbeb808ce00 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 1 -- 10.20.193.102:0/1094387364 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea80038c0 con 0x7fbeb808ce00 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbeaf7fe640 10 monclient: dump: 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 4 set_mon_vals no callback set 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: shutdown 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- 10.20.193.102:0/1094387364 >> v1:10.20.193.102:6790/0 conn(0x7fbeb808ce00 legacy=0x7fbeb808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- 10.20.193.102:0/1094387364 shutdown_connections 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- 10.20.193.102:0/1094387364 >> 10.20.193.102:0/1094387364 conn(0x7fbeb8091a50 msgr2=0x7fbeb8091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- 10.20.193.102:0/1094387364 shutdown_connections 2026-02-21T22:21:36.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- 10.20.193.102:0/1094387364 wait complete. 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: monmap: 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 Processor -- start 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- start start 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: init 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-c con 0x7fbeb808b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-a con 0x7fbeb808ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient(hunting): picked mon.noname-b con 0x7fbeb808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb807ed50 con 0x7fbeb808ce00 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb8088a70 con 0x7fbeb808d8e0 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeb8080810 con 0x7fbeb808b7f0 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebef46640 10 monclient(hunting): authenticate will time out at 1505.954099s 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebccbb640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fbeb808b7f0 0x7fbeb8151170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44774/0 (socket says 10.20.193.102:44774) 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.713+0000 7fbebccbb640 1 -- 10.20.193.102:0/532063079 learned_addr learned my addr 10.20.193.102:0/532063079 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebccbb640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb808b7f0 legacy=0x7fbeb8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbeaffff640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6789/0 conn(0x7fbeb808ce00 legacy=0x7fbeb81924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20909355 0 0) 0x7fbeb8080810 con 0x7fbeb808b7f0 2026-02-21T22:21:36.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): my global_id is 4652 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe88003620 con 0x7fbeb808b7f0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939058275 0 0) 0x7fbeb807ed50 con 0x7fbeb808ce00 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): my global_id is 34299 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb8080810 con 0x7fbeb808ce00 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2426107528 0 0) 0x7fbe88003620 con 0x7fbeb808b7f0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbeb807ed50 con 0x7fbeb808b7f0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea8003710 con 0x7fbeb808b7f0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): dump: 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:36.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebd4bc640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.178:6789/0 conn(0x7fbeb808d8e0 legacy=0x7fbeb8193bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3850841301 0 0) 0x7fbeb8080810 con 0x7fbeb808ce00 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe88003620 con 0x7fbeb808ce00 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea00032e0 con 0x7fbeb808ce00 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): dump: 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1810003805 0 0) 0x7fbeb807ed50 con 0x7fbeb808b7f0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.178:6789/0 conn(0x7fbeb808d8e0 legacy=0x7fbeb8193bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6789/0 conn(0x7fbeb808ce00 legacy=0x7fbeb81924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 monclient: found mon.c 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb8080af0 con 0x7fbeb808b7f0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 5 monclient: authenticate success, global_id 4652 2026-02-21T22:21:36.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 10 monclient: _renew_subs 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb807f910 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 10 monclient: _renew_subs 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbeb8081c60 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbea80033d0 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea8004090 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: dump: 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbeaeffd640 4 set_mon_vals no callback set 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbeb80008e0 con 0x7fbeb807e450 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fbea8004330 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(301..301 src has 112..301) ==== 3899+0+0 (unknown 3142247521 0 0) 0x7fbea804c730 con 0x7fbeb808b7f0 2026-02-21T22:21:36.724 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbebd4bc640 10 monclient: get_auth_request con 0x7fbe8803d6a0 auth_method 0 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbeaffff640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb807e450 legacy=0x7fbeb807a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976550507 0 0) 0x7fbeb80008e0 con 0x7fbeb807e450 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient(hunting): my global_id is 4655 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb807ed50 con 0x7fbeb807e450 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2010695503 0 0) 0x7fbeb807ed50 con 0x7fbeb807e450 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbeb80008e0 con 0x7fbeb807e450 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea0004a90 con 0x7fbeb807e450 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.717+0000 7fbead7fa640 10 monclient: dump: 2026-02-21T22:21:36.725 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1990311348 0 0) 0x7fbeb80008e0 con 0x7fbeb807e450 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbeb8090810 con 0x7fbeb807e450 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fbea00059d0 con 0x7fbeb807e450 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb807e450 legacy=0x7fbeb807a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb807e450 legacy=0x7fbeb807a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbebef46640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:36.726 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.721+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbeb80909c0 con 0x7fbeb8151880 2026-02-21T22:21:36.728 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbeaffff640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb8151880 legacy=0x7fbeb807aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285692670 0 0) 0x7fbeb80909c0 con 0x7fbeb8151880 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient(hunting): my global_id is 4658 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb80008e0 con 0x7fbeb8151880 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 534345059 0 0) 0x7fbeb80008e0 con 0x7fbeb8151880 2026-02-21T22:21:36.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbeb80909c0 con 0x7fbeb8151880 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbea00051a0 con 0x7fbeb8151880 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient: dump: 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 615322039 0 0) 0x7fbeb80909c0 con 0x7fbeb8151880 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbeb8080f60 con 0x7fbeb8151880 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 1929474188) 0x7fbea0005510 con 0x7fbeb8151880 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb8151880 legacy=0x7fbeb807aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbead7fa640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb8151880 legacy=0x7fbeb807aa80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:36.730 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 379355, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:36.731 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:36.732 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.733 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:36.734 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.735 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:21:36.736 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:36.737 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.178:6800/2549949439 conn(0x7fbe8803d6a0 legacy=0x7fbe8805db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 10 monclient: shutdown 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 >> v1:10.20.193.102:6790/0 conn(0x7fbeb808b7f0 legacy=0x7fbeb8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebd4bc640 1 -- 10.20.193.102:0/532063079 reap_dead start 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 shutdown_connections 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 >> 10.20.193.102:0/532063079 conn(0x7fbeb8091a50 msgr2=0x7fbeb805a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 shutdown_connections 2026-02-21T22:21:36.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.725+0000 7fbebef46640 1 -- 10.20.193.102:0/532063079 wait complete. 2026-02-21T22:21:36.739 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:36.740 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:21:36.740 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:21:36.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 10 monclient: get_monmap_and_config 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 10 monclient: monmap: 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:36.808185+0000 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:36.808185+0000 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:36.807 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 1 Processor -- start 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 1 -- start start 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.801+0000 7f19922ab640 10 monclient: init 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-b con 0x7f198c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-c con 0x7f198c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-a con 0x7f198c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c086a10 con 0x7f198c08d8e0 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c08b130 con 0x7f198c08b7f0 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c08bd60 con 0x7f198c08ce00 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient(hunting): authenticate will time out at 1506.044438s 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f198c08ce00 0x7f198c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44810/0 (socket says 10.20.193.102:44810) 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198bfff640 1 -- 10.20.193.102:0/1867565563 learned_addr learned my addr 10.20.193.102:0/1867565563 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f1990821640 1 --1- 10.20.193.102:0/1867565563 >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 0x7f198c1a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54982/0 (socket says 10.20.193.102:54982) 2026-02-21T22:21:36.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198bfff640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.102:6790/0 conn(0x7f198c08ce00 legacy=0x7f198c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f1990821640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 legacy=0x7f198c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472984421 0 0) 0x7f198c086a10 con 0x7f198c08d8e0 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): my global_id is 34302 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1970003620 con 0x7f198c08d8e0 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646203035 0 0) 0x7f198c08bd60 con 0x7f198c08ce00 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198b7fe640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.178:6789/0 conn(0x7f198c08b7f0 legacy=0x7f198c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient(hunting): my global_id is 4661 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c086a10 con 0x7f198c08ce00 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1609346872 0 0) 0x7f1970003620 con 0x7f198c08d8e0 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.102:6790/0 conn(0x7f198c08ce00 legacy=0x7f198c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.178:6789/0 conn(0x7f198c08b7f0 legacy=0x7f198c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 monclient: found mon.noname-a 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c08b370 con 0x7f198c08d8e0 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1980004550 con 0x7f198c08d8e0 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: dump: 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 5 monclient: authenticate success, global_id 34302 2026-02-21T22:21:36.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1980004820 con 0x7f198c08d8e0 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 1 -- 10.20.193.102:0/1867565563 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1980005df0 con 0x7f198c08d8e0 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f198affd640 10 monclient: dump: 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 4 set_mon_vals no callback set 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: shutdown 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- 10.20.193.102:0/1867565563 >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 legacy=0x7f198c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- 10.20.193.102:0/1867565563 shutdown_connections 2026-02-21T22:21:36.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- 10.20.193.102:0/1867565563 >> 10.20.193.102:0/1867565563 conn(0x7f198c091a50 msgr2=0x7f198c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- 10.20.193.102:0/1867565563 shutdown_connections 2026-02-21T22:21:36.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 1 -- 10.20.193.102:0/1867565563 wait complete. 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.805+0000 7f19922ab640 10 monclient: monmap: 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:36.816 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 Processor -- start 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- start start 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: init 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-c con 0x7f198c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-b con 0x7f198c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient(hunting): picked mon.noname-a con 0x7f198c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c083380 con 0x7f198c08d8e0 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c088a70 con 0x7f198c08ce00 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198c081020 con 0x7f198c08b7f0 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient(hunting): authenticate will time out at 1506.048440s 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1990821640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 0x7f198c195b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:54998/0 (socket says 10.20.193.102:54998) 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1990821640 1 -- 10.20.193.102:0/3472231508 learned_addr learned my addr 10.20.193.102:0/3472231508 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f198bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f198c08b7f0 0x7f198c0848b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44816/0 (socket says 10.20.193.102:44816) 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f198bfff640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.102:6790/0 conn(0x7f198c08b7f0 legacy=0x7f198c0848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1990821640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 legacy=0x7f198c195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982063971 0 0) 0x7f198c083380 con 0x7f198c08d8e0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): my global_id is 34305 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f195c003620 con 0x7f198c08d8e0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622476415 0 0) 0x7f198c081020 con 0x7f198c08b7f0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): my global_id is 4664 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c083380 con 0x7f198c08b7f0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f198b7fe640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c08ce00 legacy=0x7f198c194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 246667100 0 0) 0x7f195c003620 con 0x7f198c08d8e0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f198c081020 con 0x7f198c08d8e0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1980005df0 con 0x7f198c08d8e0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.818 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1174029774 0 0) 0x7f198c083380 con 0x7f198c08b7f0 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f195c003620 con 0x7f198c08b7f0 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f19780033a0 con 0x7f198c08b7f0 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.819 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 60212798 0 0) 0x7f198c081020 con 0x7f198c08d8e0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.102:6790/0 conn(0x7f198c08b7f0 legacy=0x7f198c0848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c08ce00 legacy=0x7f198c194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 monclient: found mon.a 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c0830a0 con 0x7f198c08d8e0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 5 monclient: authenticate success, global_id 34305 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _renew_subs 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f198c084fc0 con 0x7f198c08d8e0 2026-02-21T22:21:36.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _renew_subs 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f198c153270 con 0x7f198c08d8e0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f19800047c0 con 0x7f198c08d8e0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1980003fb0 con 0x7f198c08d8e0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 10 monclient: dump: 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f198a7fc640 4 set_mon_vals no callback set 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f198c05e1c0 con 0x7f198c05a3c0 2026-02-21T22:21:36.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.809+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1980006720 con 0x7f198c08d8e0 2026-02-21T22:21:36.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(301..301 src has 112..301) ==== 3899+0+0 (unknown 3142247521 0 0) 0x7f198004da60 con 0x7f198c08d8e0 2026-02-21T22:21:36.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f198b7fe640 10 monclient: get_auth_request con 0x7f195c03d6a0 auth_method 0 2026-02-21T22:21:36.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f198bfff640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05a3c0 legacy=0x7f198c05a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851070730 0 0) 0x7f198c05e1c0 con 0x7f198c05a3c0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient(hunting): my global_id is 54115 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c081020 con 0x7f198c05a3c0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 407446261 0 0) 0x7f198c081020 con 0x7f198c05a3c0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f198c05e1c0 con 0x7f198c05a3c0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1978004d10 con 0x7f198c05a3c0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient: dump: 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.823 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3984143079 0 0) 0x7f198c05e1c0 con 0x7f198c05a3c0 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f198c153560 con 0x7f198c05a3c0 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f19780059d0 con 0x7f198c05a3c0 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05a3c0 legacy=0x7f198c05a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.813+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05a3c0 legacy=0x7f198c05a790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.817+0000 7f19922ab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.817+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f198c062ce0 con 0x7f198c05aea0 2026-02-21T22:21:36.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f198bfff640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05aea0 legacy=0x7f198c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767558797 0 0) 0x7f198c062ce0 con 0x7f198c05aea0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient(hunting): my global_id is 54118 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f198c05e1c0 con 0x7f198c05aea0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 70058746 0 0) 0x7f198c05e1c0 con 0x7f198c05aea0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f198c062ce0 con 0x7f198c05aea0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f19780053b0 con 0x7f198c05aea0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient: dump: 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.825 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2109701354 0 0) 0x7f198c062ce0 con 0x7f198c05aea0 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f198c05b270 con 0x7f198c05aea0 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4344 (unknown 0 0 655661402) 0x7f1978005700 con 0x7f198c05aea0 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:36.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05aea0 legacy=0x7f198c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1988ff9640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6789/0 conn(0x7f198c05aea0 legacy=0x7f198c198250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 471, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:36.827 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.828 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.829 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:36.830 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.831 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.832 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:36.833 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:36.834 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.178:6800/2549949439 conn(0x7f195c03d6a0 legacy=0x7f195c05db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 10 monclient: shutdown 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 >> v1:10.20.193.102:6789/0 conn(0x7f198c08d8e0 legacy=0x7f198c195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1990821640 1 -- 10.20.193.102:0/3472231508 reap_dead start 2026-02-21T22:21:36.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f1990821640 1 -- 10.20.193.102:0/3472231508 reap_dead start 2026-02-21T22:21:36.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 shutdown_connections 2026-02-21T22:21:36.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 >> 10.20.193.102:0/3472231508 conn(0x7f198c091a50 msgr2=0x7f198c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 shutdown_connections 2026-02-21T22:21:36.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.821+0000 7f19922ab640 1 -- 10.20.193.102:0/3472231508 wait complete. 2026-02-21T22:21:36.836 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:36.836 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:21:36.836 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: get_monmap_and_config 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: monmap: 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:36.903778+0000 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:36.903778+0000 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.902 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.903 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.903 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:36.903 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:36.903 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 1 Processor -- start 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 1 -- start start 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: init 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-a con 0x7ff4b808ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-b con 0x7ff4b808b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-c con 0x7ff4b808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b8086a10 con 0x7ff4b808ce00 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b808b130 con 0x7ff4b808d8e0 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b808bd60 con 0x7ff4b808b7f0 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4be839640 10 monclient(hunting): authenticate will time out at 1506.139633s 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4b7fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff4b808ce00 0x7ff4b808d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55014/0 (socket says 10.20.193.102:55014) 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4b7fff640 1 -- 10.20.193.102:0/3942153148 learned_addr learned my addr 10.20.193.102:0/3942153148 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.897+0000 7ff4b77fe640 1 --1- 10.20.193.102:0/3942153148 >> v1:10.20.193.102:6790/0 conn(0x7ff4b808b7f0 0x7ff4b8181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44822/0 (socket says 10.20.193.102:44822) 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b7fff640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.102:6789/0 conn(0x7ff4b808ce00 legacy=0x7ff4b808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b77fe640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.102:6790/0 conn(0x7ff4b808b7f0 legacy=0x7ff4b8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4bcdaf640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.178:6789/0 conn(0x7ff4b808d8e0 legacy=0x7ff4b81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797699544 0 0) 0x7ff4b8086a10 con 0x7ff4b808ce00 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): my global_id is 34308 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff49c003620 con 0x7ff4b808ce00 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3628711817 0 0) 0x7ff4b808bd60 con 0x7ff4b808b7f0 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): my global_id is 4667 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b8086a10 con 0x7ff4b808b7f0 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189474002 0 0) 0x7ff4b808b130 con 0x7ff4b808d8e0 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient(hunting): my global_id is 54121 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b808bd60 con 0x7ff4b808d8e0 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432320358 0 0) 0x7ff49c003620 con 0x7ff4b808ce00 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.102:6790/0 conn(0x7ff4b808b7f0 legacy=0x7ff4b8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.178:6789/0 conn(0x7ff4b808d8e0 legacy=0x7ff4b81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 monclient: found mon.noname-a 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b808b370 con 0x7ff4b808ce00 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4a4003110 con 0x7ff4b808ce00 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: dump: 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.909 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 5 monclient: authenticate success, global_id 34308 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff4a40033e0 con 0x7ff4b808ce00 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 1 -- 10.20.193.102:0/3942153148 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4a4004b50 con 0x7ff4b808ce00 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4b6ffd640 10 monclient: dump: 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.910 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 4 set_mon_vals no callback set 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 10 monclient: shutdown 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- 10.20.193.102:0/3942153148 >> v1:10.20.193.102:6789/0 conn(0x7ff4b808ce00 legacy=0x7ff4b808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- 10.20.193.102:0/3942153148 shutdown_connections 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- 10.20.193.102:0/3942153148 >> 10.20.193.102:0/3942153148 conn(0x7ff4b8091a50 msgr2=0x7ff4b8091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- 10.20.193.102:0/3942153148 shutdown_connections 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- 10.20.193.102:0/3942153148 wait complete. 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 10 monclient: build_initial_monmap 2026-02-21T22:21:36.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 10 monclient: monmap: 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 Processor -- start 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 1 -- start start 2026-02-21T22:21:36.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.901+0000 7ff4be839640 10 monclient: init 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-b con 0x7ff4b808b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-c con 0x7ff4b808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient(hunting): picked mon.noname-a con 0x7ff4b808d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b8082d10 con 0x7ff4b808d8e0 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b8082ff0 con 0x7ff4b808b7f0 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4b8088a70 con 0x7ff4b808ce00 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient(hunting): authenticate will time out at 1506.144288s 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4bcdaf640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff4b808d8e0 0x7ff4b8195380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55020/0 (socket says 10.20.193.102:55020) 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4bcdaf640 1 -- 10.20.193.102:0/3417442071 learned_addr learned my addr 10.20.193.102:0/3417442071 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4bcdaf640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.102:6789/0 conn(0x7ff4b808d8e0 legacy=0x7ff4b8195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b77fe640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.102:6790/0 conn(0x7ff4b808ce00 legacy=0x7ff4b80847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b7fff640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.178:6789/0 conn(0x7ff4b808b7f0 legacy=0x7ff4b8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 51171968 0 0) 0x7ff4b8082d10 con 0x7ff4b808d8e0 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): my global_id is 34311 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff488003620 con 0x7ff4b808d8e0 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393966774 0 0) 0x7ff4b8088a70 con 0x7ff4b808ce00 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): my global_id is 4670 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b8082d10 con 0x7ff4b808ce00 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113796461 0 0) 0x7ff4b8082ff0 con 0x7ff4b808b7f0 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:36.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): my global_id is 54124 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b8088a70 con 0x7ff4b808b7f0 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 10843282 0 0) 0x7ff488003620 con 0x7ff4b808d8e0 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4b8082ff0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4ac0049a0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.918 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2802660051 0 0) 0x7ff4b8082d10 con 0x7ff4b808ce00 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff488003620 con 0x7ff4b808ce00 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4a80048e0 con 0x7ff4b808ce00 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.919 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2049284218 0 0) 0x7ff4b8088a70 con 0x7ff4b808b7f0 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff4b8082d10 con 0x7ff4b808b7f0 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4a40048e0 con 0x7ff4b808b7f0 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient(hunting): dump: 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.920 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4158474738 0 0) 0x7ff4b8082ff0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.102:6790/0 conn(0x7ff4b808ce00 legacy=0x7ff4b80847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.178:6789/0 conn(0x7ff4b808b7f0 legacy=0x7ff4b8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 monclient: found mon.a 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b8084ef0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: _finish_auth 0 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 5 monclient: authenticate success, global_id 34311 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _renew_subs 2026-02-21T22:21:36.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b8196aa0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _renew_subs 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff4b8196fb0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff4ac005080 con 0x7ff4b808d8e0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff4ac0053e0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 10 monclient: dump: 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:36.922 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b67fc640 4 set_mon_vals no callback set 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff4ac005580 con 0x7ff4b808d8e0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b80008e0 con 0x7ff4b808d8e0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.905+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(301..301 src has 112..301) ==== 3899+0+0 (unknown 3142247521 0 0) 0x7ff4ac04db20 con 0x7ff4b808d8e0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.909+0000 7ff4b7fff640 10 monclient: get_auth_request con 0x7ff48803d9c0 auth_method 0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.909+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ff4ac018b40 con 0x7ff4b808d8e0 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.909+0000 7ff4b4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:36.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:36.909+0000 7ff4b4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:37.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4be839640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:21:37.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4be839640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:21:37.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4be839640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:37.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ff4b8083230 con 0x7ff4b808d8e0 2026-02-21T22:21:37.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4b4ff9640 1 -- 10.20.193.102:0/3417442071 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7ff4ac018420 con 0x7ff4b808d8e0 2026-02-21T22:21:37.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4b4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:21:37.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.025+0000 7ff4b4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=2,health=13,kv=11,logm=61} crc {auth=2866216926,config=1618278320,health=1181117202,kv=1034924659,logm=3537502261}) 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=11,config=1,health=13,kv=11,logm=64} crc {auth=371605824,config=1228700967,health=128393527,kv=1148141255,logm=570865339}) 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=2,health=13,kv=11,logm=61} crc {auth=2866216926,config=1618278320,health=1181117202,kv=1034924659,logm=3537502261}) 2026-02-21T22:21:37.032 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.025+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=12,config=2,health=13,kv=11,logm=62} crc {auth=910165032,config=1618278320,health=1181117202,kv=2407103666,logm=3665387399}) 2026-02-21T22:21:37.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff48803d9c0 legacy=0x7ff48805de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:37.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 10 monclient: shutdown 2026-02-21T22:21:37.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:37.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 >> v1:10.20.193.102:6789/0 conn(0x7ff4b808d8e0 legacy=0x7ff4b8195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868303856}) 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3470869164}) 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868303856}) 2026-02-21T22:21:37.034 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1282882338}) 2026-02-21T22:21:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 shutdown_connections 2026-02-21T22:21:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 >> 10.20.193.102:0/3417442071 conn(0x7ff4b8091a50 msgr2=0x7ff4b805a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 shutdown_connections 2026-02-21T22:21:37.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:37.029+0000 7ff4be839640 1 -- 10.20.193.102:0/3417442071 wait complete. 2026-02-21T22:21:37.038 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.038 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2103126713}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=534621556}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2103126713}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=94624732}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3985988554}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2575735194}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3985988554}) 2026-02-21T22:21:37.039 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1821889420}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2442480048}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=376565077}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2442480048}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3456535695}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2613701086}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1788599658}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2613701086}) 2026-02-21T22:21:37.040 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.029+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321241278}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2472284469}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3656826060}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2472284469}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3944423654}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410874804}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2415385660}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1410874804}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2402989390}) 2026-02-21T22:21:37.041 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1123772617}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4166590008}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1123772617}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2271347788}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868124316}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3076871924}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2868124316}) 2026-02-21T22:21:37.042 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.033+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2778088083}) 2026-02-21T22:21:37.044 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:21:37.050 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.050 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=837489451}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338256699}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=837489451}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1572649982}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794131898}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1045200466}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2794131898}) 2026-02-21T22:21:37.051 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.045+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2267527318}) 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=83,mdsmap=2,mgr=8,mgr_command_descs=1,mgrstat=6} crc {logm=2383200505,mdsmap=4097168772,mgr=369547122,mgr_command_descs=2141687499,mgrstat=3371624109}) 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=6} crc {logm=1581805655,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2452676759}) 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=83,mdsmap=2,mgr=8,mgr_command_descs=1,mgrstat=6} crc {logm=2383200505,mdsmap=4097168772,mgr=369547122,mgr_command_descs=2141687499,mgrstat=3371624109}) 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=82,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=5} crc {logm=1621176343,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=303490852}) 2026-02-21T22:21:37.053 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=6,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=56,osdmap=17} crc {mgrstat=1234543901,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=1493522314,osd_pg_creating=623970771,osd_snap=3684467940,osdmap=1598530724}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=6,mon_config_key=7,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=55,osdmap=20} crc {mgrstat=1234543901,mon_config_key=662138479,monmap=3140514009,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=623970771,osd_snap=444591563,osdmap=1824346742}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=6,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=56,osdmap=17} crc {mgrstat=1234543901,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=1493522314,osd_pg_creating=623970771,osd_snap=3684467940,osdmap=1598530724}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=6,mon_config_key=9,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=54,osdmap=19} crc {mgrstat=1234543901,mon_config_key=2427692996,monmap=2662863928,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=623970771,osd_snap=2667447097,osdmap=3746677061}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1528744917}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2822542595}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1528744917}) 2026-02-21T22:21:37.054 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4261506986}) 2026-02-21T22:21:37.055 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1049013823}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3211408135}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1049013823}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=844672379}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=72} crc {osdmap=987336661}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=66} crc {osdmap=2467903268}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=72} crc {osdmap=987336661}) 2026-02-21T22:21:37.056 INFO:tasks.ceph.mon.c.trial102.stderr:2026-02-21T22:21:37.049+0000 7f93c091b640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=68} crc {osdmap=1478026766}) 2026-02-21T22:21:37.574 INFO:tasks.rados.rados.0.trial178.stdout:1430: done (4 left) 2026-02-21T22:21:37.574 INFO:tasks.rados.rados.0.trial178.stdout:1431: write oid 5 current snap is 144 2026-02-21T22:21:37.574 INFO:tasks.rados.rados.0.trial178.stdout:1431: seq_num 509 ranges {554763=781039,2102511=452252} 2026-02-21T22:21:37.583 INFO:tasks.rados.rados.0.trial178.stdout:1431: writing trial17818531-5 from 554763 to 1335802 tid 1 2026-02-21T22:21:37.591 INFO:tasks.rados.rados.0.trial178.stdout:1431: writing trial17818531-5 from 2102511 to 2554763 tid 2 2026-02-21T22:21:37.591 INFO:tasks.rados.rados.0.trial178.stdout:1432: delete oid 43 current snap is 144 2026-02-21T22:21:37.592 INFO:tasks.rados.rados.0.trial178.stdout:1428: finishing copy_from to trial17818531-2 2026-02-21T22:21:37.592 INFO:tasks.rados.rados.0.trial178.stdout:1428: got expected ENOENT (src dne) 2026-02-21T22:21:37.592 INFO:tasks.rados.rados.0.trial178.stdout:1428: finishing copy_from racing read to trial17818531-2 2026-02-21T22:21:37.592 INFO:tasks.rados.rados.0.trial178.stdout:1426: expect (ObjNum 485 snap 134 seq_num 485) 2026-02-21T22:21:37.592 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:37.723 INFO:tasks.rados.rados.0.trial178.stdout:1425: finishing write tid 2 to trial17818531-15 2026-02-21T22:21:37.723 INFO:tasks.rados.rados.0.trial178.stdout:1425: finishing write tid 3 to trial17818531-15 2026-02-21T22:21:37.723 INFO:tasks.rados.rados.0.trial178.stdout:1425: finishing write tid 4 to trial17818531-15 2026-02-21T22:21:37.723 INFO:tasks.rados.rados.0.trial178.stdout:1425: oid 15 updating version 0 to 510 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:1425: oid 15 updating version 510 to 512 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:1425: oid 15 version 512 is already newer than 511 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 512 (ObjNum 507 snap 144 seq_num 507) dirty exists 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:1425: left oid 15 (ObjNum 507 snap 144 seq_num 507) 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:1429: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:37.724 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:37.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1965111 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: finishing write tid 1 to trial17818531-5 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: finishing write tid 2 to trial17818531-5 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: finishing write tid 3 to trial17818531-5 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: finishing write tid 4 to trial17818531-5 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: oid 5 updating version 0 to 733 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: oid 5 updating version 733 to 734 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: oid 5 version 734 is already newer than 732 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 734 (ObjNum 509 snap 144 seq_num 509) dirty exists 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: left oid 5 (ObjNum 509 snap 144 seq_num 509) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1425: done (5 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1426: done (4 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1428: done (3 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1429: done (2 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1431: done (1 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1432: done (0 left) 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1433: write oid 30 current snap is 144 2026-02-21T22:21:37.845 INFO:tasks.rados.rados.0.trial178.stdout:1433: seq_num 511 ranges {588377=647890,1935828=635840} 2026-02-21T22:21:37.853 INFO:tasks.rados.rados.0.trial178.stdout:1433: writing trial17818531-30 from 588377 to 1236267 tid 1 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1433: writing trial17818531-30 from 1935828 to 2571668 tid 2 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1434: read oid 27 snap 140 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1434: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1435: read oid 26 snap -1 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1435: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:37.863 INFO:tasks.rados.rados.0.trial178.stdout:1436: snap_create 2026-02-21T22:21:37.864 INFO:tasks.rados.rados.0.trial178.stdout:1433: finishing write tid 1 to trial17818531-30 2026-02-21T22:21:37.871 INFO:tasks.rados.rados.0.trial178.stdout:1433: finishing write tid 2 to trial17818531-30 2026-02-21T22:21:37.877 INFO:tasks.rados.rados.0.trial178.stdout:1435: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:21:37.916 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:38.002 INFO:tasks.rados.rados.0.trial178.stdout:1433: finishing write tid 3 to trial17818531-30 2026-02-21T22:21:38.002 INFO:tasks.rados.rados.0.trial178.stdout:1433: finishing write tid 4 to trial17818531-30 2026-02-21T22:21:38.002 INFO:tasks.rados.rados.0.trial178.stdout:1433: oid 30 updating version 0 to 736 2026-02-21T22:21:38.002 INFO:tasks.rados.rados.0.trial178.stdout:1433: oid 30 updating version 736 to 737 2026-02-21T22:21:38.003 INFO:tasks.rados.rados.0.trial178.stdout:1433: oid 30 version 737 is already newer than 735 2026-02-21T22:21:38.003 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 737 (ObjNum 511 snap 144 seq_num 511) dirty exists 2026-02-21T22:21:38.003 INFO:tasks.rados.rados.0.trial178.stdout:1433: left oid 30 (ObjNum 511 snap 144 seq_num 511) 2026-02-21T22:21:38.003 INFO:tasks.rados.rados.0.trial178.stdout:1434: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:21:38.015 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1822147 2026-02-21T22:21:38.045 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:21:38.045 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:21:38.094 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 850177 2026-02-21T22:21:38.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: get_monmap_and_config 2026-02-21T22:21:38.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: monmap: 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:38.109149+0000 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:38.109149+0000 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:38.109 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 1 Processor -- start 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 1 -- start start 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: init 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-a con 0x7ff23008c1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-c con 0x7ff23008cc80 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-b con 0x7ff230183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23008d780 con 0x7ff23008c1a0 2026-02-21T22:21:38.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff230085da0 con 0x7ff23008cc80 2026-02-21T22:21:38.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff230085420 con 0x7ff230183480 2026-02-21T22:21:38.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient(hunting): authenticate will time out at 1507.345039s 2026-02-21T22:21:38.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22effd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 0x7ff23008c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55038/0 (socket says 10.20.193.102:55038) 2026-02-21T22:21:38.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22effd640 1 -- 10.20.193.102:0/3533334035 learned_addr learned my addr 10.20.193.102:0/3533334035 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22effd640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 legacy=0x7ff23008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22f7fe640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.102:6790/0 conn(0x7ff230183480 legacy=0x7ff2301a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142678918 0 0) 0x7ff224002c60 con 0x7ff23008c1a0 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22e7fc640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.178:6789/0 conn(0x7ff23008cc80 legacy=0x7ff230181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): my global_id is 34314 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff21c003620 con 0x7ff23008c1a0 2026-02-21T22:21:38.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111810505 0 0) 0x7ff230085420 con 0x7ff230183480 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient(hunting): my global_id is 4673 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff224002c60 con 0x7ff230183480 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5438849 0 0) 0x7ff21c003620 con 0x7ff23008c1a0 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.102:6790/0 conn(0x7ff230183480 legacy=0x7ff2301a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.178:6789/0 conn(0x7ff23008cc80 legacy=0x7ff230181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 monclient: found mon.noname-a 2026-02-21T22:21:38.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff230086640 con 0x7ff23008c1a0 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 5 monclient: authenticate success, global_id 34314 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff224003360 con 0x7ff23008c1a0 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: dump: 2026-02-21T22:21:38.122 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2240049a0 con 0x7ff23008c1a0 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 1 -- 10.20.193.102:0/3533334035 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff224004d70 con 0x7ff23008c1a0 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22dffb640 10 monclient: dump: 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.124 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.105+0000 7ff22ffff640 4 set_mon_vals no callback set 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: shutdown 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/3533334035 >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 legacy=0x7ff23008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/3533334035 shutdown_connections 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/3533334035 >> 10.20.193.102:0/3533334035 conn(0x7ff23008f6a0 msgr2=0x7ff23008faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/3533334035 shutdown_connections 2026-02-21T22:21:38.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/3533334035 wait complete. 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: monmap: 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:38.126 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 Processor -- start 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- start start 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: init 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-a con 0x7ff23008c1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-b con 0x7ff23008cc80 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient(hunting): picked mon.noname-c con 0x7ff230183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff230083530 con 0x7ff23008c1a0 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff230058bf0 con 0x7ff23008cc80 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff230088990 con 0x7ff230183480 2026-02-21T22:21:38.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient(hunting): authenticate will time out at 1507.349003s 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22effd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 0x7ff230150080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55052/0 (socket says 10.20.193.102:55052) 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22effd640 1 -- 10.20.193.102:0/1346887290 learned_addr learned my addr 10.20.193.102:0/1346887290 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22effd640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 legacy=0x7ff230150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22f7fe640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.102:6790/0 conn(0x7ff230183480 legacy=0x7ff230175e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2904861853 0 0) 0x7ff230083530 con 0x7ff23008c1a0 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22e7fc640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.178:6789/0 conn(0x7ff23008cc80 legacy=0x7ff230174700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): my global_id is 34317 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1fc003620 con 0x7ff23008c1a0 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498745974 0 0) 0x7ff230088990 con 0x7ff230183480 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): my global_id is 4676 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff230083530 con 0x7ff230183480 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1107740962 0 0) 0x7ff1fc003620 con 0x7ff23008c1a0 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff230088990 con 0x7ff23008c1a0 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff224003360 con 0x7ff23008c1a0 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): dump: 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:38.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590022591 0 0) 0x7ff230058bf0 con 0x7ff23008cc80 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): my global_id is 54130 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1fc003620 con 0x7ff23008cc80 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1548972822 0 0) 0x7ff230083530 con 0x7ff230183480 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff230058bf0 con 0x7ff230183480 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff220004900 con 0x7ff230183480 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient(hunting): dump: 2026-02-21T22:21:38.131 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3921206427 0 0) 0x7ff230088990 con 0x7ff23008c1a0 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.102:6790/0 conn(0x7ff230183480 legacy=0x7ff230175e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.178:6789/0 conn(0x7ff23008cc80 legacy=0x7ff230174700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 monclient: found mon.a 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2300811d0 con 0x7ff23008c1a0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff20f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 5 monclient: authenticate success, global_id 34317 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _renew_subs 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff230083250 con 0x7ff23008c1a0 2026-02-21T22:21:38.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _renew_subs 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.109+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff230080770 con 0x7ff23008c1a0 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff2240049a0 con 0x7ff23008c1a0 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff224005540 con 0x7ff23008c1a0 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: dump: 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.134 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22d7fa640 4 set_mon_vals no callback set 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22ffff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22ffff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8005130 con 0x7ff23008c1a0 2026-02-21T22:21:38.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff224011bd0 con 0x7ff23008c1a0 2026-02-21T22:21:38.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7ff22404c9b0 con 0x7ff23008c1a0 2026-02-21T22:21:38.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff22e7fc640 10 monclient: get_auth_request con 0x7ff1fc03d880 auth_method 0 2026-02-21T22:21:38.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ff23008d780 con 0x7ff23008c1a0 2026-02-21T22:21:38.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.113+0000 7ff20f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:38.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff22ffff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:21:38.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff22ffff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:38.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff22ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff1f8005420 con 0x7ff23008c1a0 2026-02-21T22:21:38.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff20f7fe640 1 -- 10.20.193.102:0/1346887290 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 874614248) 0x7ff2240179a0 con 0x7ff23008c1a0 2026-02-21T22:21:38.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff20f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:38.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.229+0000 7ff20f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:38.236 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:21:38.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff1fc03d880 legacy=0x7ff1fc05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 10 monclient: shutdown 2026-02-21T22:21:38.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 >> v1:10.20.193.102:6789/0 conn(0x7ff23008c1a0 legacy=0x7ff230150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 shutdown_connections 2026-02-21T22:21:38.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 >> 10.20.193.102:0/1346887290 conn(0x7ff23008f6a0 msgr2=0x7ff230088ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 shutdown_connections 2026-02-21T22:21:38.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.233+0000 7ff22ffff640 1 -- 10.20.193.102:0/1346887290 wait complete. 2026-02-21T22:21:38.245 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:21:38.246 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:21:38.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:21:38.246 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:21:38.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: get_monmap_and_config 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: monmap: 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:38.307701+0000 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:38.307701+0000 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.307 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:38.308 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:38.308 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 1 Processor -- start 2026-02-21T22:21:38.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 1 -- start start 2026-02-21T22:21:38.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: init 2026-02-21T22:21:38.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-b con 0x7f73b408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-a con 0x7f73b408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-c con 0x7f73b408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4086a10 con 0x7f73b408b7f0 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b408b130 con 0x7f73b408d8e0 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b408bd60 con 0x7f73b408ce00 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b9fe5640 10 monclient(hunting): authenticate will time out at 1507.543796s 2026-02-21T22:21:38.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b2ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 0x7f73b4181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55060/0 (socket says 10.20.193.102:55060) 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.301+0000 7f73b2ffd640 1 -- 10.20.193.102:0/3141281572 learned_addr learned my addr 10.20.193.102:0/3141281572 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b2ffd640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 legacy=0x7f73b4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b37fe640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.102:6790/0 conn(0x7f73b408ce00 legacy=0x7f73b408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 286722765 0 0) 0x7f73b4086a10 con 0x7f73b408b7f0 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): my global_id is 34320 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7398003620 con 0x7f73b408b7f0 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b3fff640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.178:6789/0 conn(0x7f73b408d8e0 legacy=0x7f73b41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317098984 0 0) 0x7f73b408bd60 con 0x7f73b408ce00 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient(hunting): my global_id is 4679 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b4086a10 con 0x7f73b408ce00 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3423209928 0 0) 0x7f7398003620 con 0x7f73b408b7f0 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.102:6790/0 conn(0x7f73b408ce00 legacy=0x7f73b408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.178:6789/0 conn(0x7f73b408d8e0 legacy=0x7f73b41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 monclient: found mon.noname-a 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b408b370 con 0x7f73b408b7f0 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a4003110 con 0x7f73b408b7f0 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 5 monclient: authenticate success, global_id 34320 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: dump: 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.316 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73a40033e0 con 0x7f73b408b7f0 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 1 -- 10.20.193.102:0/3141281572 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a4004b50 con 0x7f73b408b7f0 2026-02-21T22:21:38.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b27fc640 10 monclient: dump: 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.318 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 4 set_mon_vals no callback set 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 10 monclient: shutdown 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3141281572 >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 legacy=0x7f73b4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3141281572 shutdown_connections 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3141281572 >> 10.20.193.102:0/3141281572 conn(0x7f73b4091a50 msgr2=0x7f73b4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3141281572 shutdown_connections 2026-02-21T22:21:38.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3141281572 wait complete. 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 10 monclient: monmap: 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 Processor -- start 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 1 -- start start 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.305+0000 7f73b9fe5640 10 monclient: init 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-a con 0x7f73b408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-c con 0x7f73b408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient(hunting): picked mon.noname-b con 0x7f73b408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4058da0 con 0x7f73b408b7f0 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4080f70 con 0x7f73b408d8e0 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b4080c90 con 0x7f73b408ce00 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.321 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient(hunting): authenticate will time out at 1507.548365s 2026-02-21T22:21:38.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b37fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 0x7f73b41515c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55070/0 (socket says 10.20.193.102:55070) 2026-02-21T22:21:38.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b2ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f73b408ce00 0x7f73b4192150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44872/0 (socket says 10.20.193.102:44872) 2026-02-21T22:21:38.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b37fe640 1 -- 10.20.193.102:0/3732124632 learned_addr learned my addr 10.20.193.102:0/3732124632 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b37fe640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 legacy=0x7f73b41515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b2ffd640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6790/0 conn(0x7f73b408ce00 legacy=0x7f73b4192150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407872149 0 0) 0x7f73b4058da0 con 0x7f73b408b7f0 2026-02-21T22:21:38.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): my global_id is 34323 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7384003620 con 0x7f73b408b7f0 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239880768 0 0) 0x7f73b4080c90 con 0x7f73b408ce00 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): my global_id is 4682 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b4058da0 con 0x7f73b408ce00 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b3fff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.178:6789/0 conn(0x7f73b408d8e0 legacy=0x7f73b4193870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2050055769 0 0) 0x7f7384003620 con 0x7f73b408b7f0 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b4080c90 con 0x7f73b408b7f0 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a0003490 con 0x7f73b408b7f0 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): dump: 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.330 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3982244866 0 0) 0x7f73b4058da0 con 0x7f73b408ce00 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7384003620 con 0x7f73b408ce00 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a40048e0 con 0x7f73b408ce00 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): dump: 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.331 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518899343 0 0) 0x7f73b4080f70 con 0x7f73b408d8e0 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient(hunting): my global_id is 54136 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b4058da0 con 0x7f73b408d8e0 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 711792936 0 0) 0x7f73b4080c90 con 0x7f73b408b7f0 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6790/0 conn(0x7f73b408ce00 legacy=0x7f73b4192150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.178:6789/0 conn(0x7f73b408d8e0 legacy=0x7f73b4193870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 monclient: found mon.a 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b4151cd0 con 0x7f73b408b7f0 2026-02-21T22:21:38.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 5 monclient: authenticate success, global_id 34323 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _renew_subs 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73b407f770 con 0x7f73b408b7f0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _renew_subs 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f73b407e380 con 0x7f73b408b7f0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73a0003790 con 0x7f73b408b7f0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a0003de0 con 0x7f73b408b7f0 2026-02-21T22:21:38.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 10 monclient: dump: 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.335 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b1ffb640 4 set_mon_vals no callback set 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f73a00040a0 con 0x7f73b408b7f0 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f73b40806d0 con 0x7f73b407a370 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.309+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7f73a004c7a0 con 0x7f73b408b7f0 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f73b3fff640 10 monclient: get_auth_request con 0x7f738403d850 auth_method 0 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f73b2ffd640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407a370 legacy=0x7f73b407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969535467 0 0) 0x7f73b40806d0 con 0x7f73b407a370 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient(hunting): my global_id is 34326 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b4080c90 con 0x7f73b407a370 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2493908833 0 0) 0x7f73b4080c90 con 0x7f73b407a370 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b40806d0 con 0x7f73b407a370 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a4004ec0 con 0x7f73b407a370 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient: dump: 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2516864889 0 0) 0x7f73b40806d0 con 0x7f73b407a370 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73b4088a70 con 0x7f73b407a370 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f73a4007a20 con 0x7f73b407a370 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:38.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407a370 legacy=0x7f73b407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.313+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407a370 legacy=0x7f73b407a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.317+0000 7f73b9fe5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.317+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f73b407b280 con 0x7f73b407ae50 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b2ffd640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407ae50 legacy=0x7f73b4195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1124940340 0 0) 0x7f73b407b280 con 0x7f73b407ae50 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient(hunting): my global_id is 34329 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b40806d0 con 0x7f73b407ae50 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1567887178 0 0) 0x7f73b40806d0 con 0x7f73b407ae50 2026-02-21T22:21:38.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b407b280 con 0x7f73b407ae50 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f73a4005470 con 0x7f73b407ae50 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient: dump: 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.340 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1173741048 0 0) 0x7f73b407b280 con 0x7f73b407ae50 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f73b4062ce0 con 0x7f73b407ae50 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 2533078987) 0x7f73a4005650 con 0x7f73b407ae50 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407ae50 legacy=0x7f73b4195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f738ffff640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b407ae50 legacy=0x7f73b4195fa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:38.342 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 247190, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.343 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.344 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:38.345 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.346 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:38.347 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.348 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.349 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.350 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.351 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.352 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.353 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.354 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:38.355 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:38.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.178:6800/2549949439 conn(0x7f738403d850 legacy=0x7f738405dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 10 monclient: shutdown 2026-02-21T22:21:38.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 >> v1:10.20.193.102:6789/0 conn(0x7f73b408b7f0 legacy=0x7f73b41515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b3fff640 1 -- 10.20.193.102:0/3732124632 reap_dead start 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b3fff640 1 -- 10.20.193.102:0/3732124632 reap_dead start 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 shutdown_connections 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 >> 10.20.193.102:0/3732124632 conn(0x7f73b4091a50 msgr2=0x7f73b405a120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 shutdown_connections 2026-02-21T22:21:38.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.321+0000 7f73b9fe5640 1 -- 10.20.193.102:0/3732124632 wait complete. 2026-02-21T22:21:38.357 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:38.357 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:21:38.358 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:21:38.418 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 10 monclient: get_monmap_and_config 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 10 monclient: monmap: 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:38.420050+0000 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:38.420050+0000 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.419 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:38.420 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:38.420 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 1 Processor -- start 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 1 -- start start 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.413+0000 7fc993409640 10 monclient: init 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient(hunting): picked mon.noname-b con 0x7fc98c085970 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient(hunting): picked mon.noname-a con 0x7fc98c085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient(hunting): picked mon.noname-c con 0x7fc98c1854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c08df30 con 0x7fc98c085d40 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c08d900 con 0x7fc98c1854c0 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c056dd0 con 0x7fc98c085970 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient(hunting): authenticate will time out at 1507.656221s 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99117e640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 0x7fc98c08e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44886/0 (socket says 10.20.193.102:44886) 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99117e640 1 -- 10.20.193.102:0/1103128307 learned_addr learned my addr 10.20.193.102:0/1103128307 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99097d640 1 --1- 10.20.193.102:0/1103128307 >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 0x7fc98c183da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55098/0 (socket says 10.20.193.102:55098) 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99117e640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 legacy=0x7fc98c08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99097d640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 legacy=0x7fc98c183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325074015 0 0) 0x7fc98c056dd0 con 0x7fc98c085970 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient(hunting): my global_id is 4685 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc974003620 con 0x7fc98c085970 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc99197f640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.178:6789/0 conn(0x7fc98c1854c0 legacy=0x7fc98c1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4095084617 0 0) 0x7fc974003620 con 0x7fc98c085970 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.178:6789/0 conn(0x7fc98c1854c0 legacy=0x7fc98c1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 legacy=0x7fc98c183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 monclient: found mon.noname-b 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c08d620 con 0x7fc98c085970 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc97c003110 con 0x7fc98c085970 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: dump: 2026-02-21T22:21:38.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 5 monclient: authenticate success, global_id 4685 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc97c0033e0 con 0x7fc98c085970 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 1 -- 10.20.193.102:0/1103128307 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc97c004b50 con 0x7fc98c085970 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc97bfff640 10 monclient: dump: 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.430 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 4 set_mon_vals no callback set 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: shutdown 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- 10.20.193.102:0/1103128307 >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 legacy=0x7fc98c08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- 10.20.193.102:0/1103128307 shutdown_connections 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- 10.20.193.102:0/1103128307 >> 10.20.193.102:0/1103128307 conn(0x7fc98c054e40 msgr2=0x7fc98c092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- 10.20.193.102:0/1103128307 shutdown_connections 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- 10.20.193.102:0/1103128307 wait complete. 2026-02-21T22:21:38.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: monmap: 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 Processor -- start 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 1 -- start start 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.417+0000 7fc993409640 10 monclient: init 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient(hunting): picked mon.noname-c con 0x7fc98c085970 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient(hunting): picked mon.noname-a con 0x7fc98c085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient(hunting): picked mon.noname-b con 0x7fc98c1854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c07fcd0 con 0x7fc98c085d40 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c089a80 con 0x7fc98c1854c0 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc98c089370 con 0x7fc98c085970 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient(hunting): authenticate will time out at 1507.660264s 2026-02-21T22:21:38.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99117e640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 0x7fc98c131d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44890/0 (socket says 10.20.193.102:44890) 2026-02-21T22:21:38.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99097d640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 0x7fc98c183b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55110/0 (socket says 10.20.193.102:55110) 2026-02-21T22:21:38.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99117e640 1 -- 10.20.193.102:0/2742644371 learned_addr learned my addr 10.20.193.102:0/2742644371 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99117e640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 legacy=0x7fc98c131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99097d640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 legacy=0x7fc98c183b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545736657 0 0) 0x7fc98c07fcd0 con 0x7fc98c085d40 2026-02-21T22:21:38.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): my global_id is 34332 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc95c003620 con 0x7fc98c085d40 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071198356 0 0) 0x7fc98c089370 con 0x7fc98c085970 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): my global_id is 4688 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c07fcd0 con 0x7fc98c085970 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc99197f640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.178:6789/0 conn(0x7fc98c1854c0 legacy=0x7fc98c132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4117016122 0 0) 0x7fc95c003620 con 0x7fc98c085d40 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc98c089370 con 0x7fc98c085d40 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc980003330 con 0x7fc98c085d40 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): dump: 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.443 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 331431001 0 0) 0x7fc98c07fcd0 con 0x7fc98c085970 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc95c003620 con 0x7fc98c085970 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc97c0048e0 con 0x7fc98c085970 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): dump: 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.445 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031797595 0 0) 0x7fc98c089a80 con 0x7fc98c1854c0 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient(hunting): my global_id is 54139 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c07fcd0 con 0x7fc98c1854c0 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3805593869 0 0) 0x7fc98c089370 con 0x7fc98c085d40 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c085970 legacy=0x7fc98c131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.178:6789/0 conn(0x7fc98c1854c0 legacy=0x7fc98c132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 monclient: found mon.a 2026-02-21T22:21:38.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98c1852b0 con 0x7fc98c085d40 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 5 monclient: authenticate success, global_id 34332 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _renew_subs 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc98c07d970 con 0x7fc98c085d40 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _renew_subs 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc98c07d030 con 0x7fc98c085d40 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc980004b90 con 0x7fc98c085d40 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc980004ef0 con 0x7fc98c085d40 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc97b7fe640 4 set_mon_vals no callback set 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 10 monclient: dump: 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.448 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc98c08f100 con 0x7fc98c132460 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fc9800051c0 con 0x7fc98c085d40 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.421+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7fc98004c700 con 0x7fc98c085d40 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc99197f640 10 monclient: get_auth_request con 0x7fc95c03d880 auth_method 0 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc99117e640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c132460 legacy=0x7fc98c089c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459706167 0 0) 0x7fc98c08f100 con 0x7fc98c132460 2026-02-21T22:21:38.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient(hunting): my global_id is 4691 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c089370 con 0x7fc98c132460 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1229209912 0 0) 0x7fc98c089370 con 0x7fc98c132460 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc98c08f100 con 0x7fc98c132460 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc97c005070 con 0x7fc98c132460 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient: dump: 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.450 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 938073785 0 0) 0x7fc98c08f100 con 0x7fc98c132460 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc98c07faa0 con 0x7fc98c132460 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fc97c005070 con 0x7fc98c132460 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c132460 legacy=0x7fc98c089c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.425+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c132460 legacy=0x7fc98c089c50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.429+0000 7fc993409640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.429+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc98c07d370 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc99117e640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c08a360 legacy=0x7fc98c1354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712526955 0 0) 0x7fc98c07d370 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient(hunting): my global_id is 4694 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc98c08f100 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1235899576 0 0) 0x7fc98c08f100 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc98c07d370 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc97c005640 con 0x7fc98c08a360 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient: dump: 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.453 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 180821106 0 0) 0x7fc98c07d370 con 0x7fc98c08a360 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc98c062ce0 con 0x7fc98c08a360 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 1210351419) 0x7fc97c008ac0 con 0x7fc98c08a360 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c08a360 legacy=0x7fc98c1354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc979ffb640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6790/0 conn(0x7fc98c08a360 legacy=0x7fc98c1354f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 381063, 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:38.455 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.456 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.457 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:38.458 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.459 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.460 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.461 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:38.462 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:38.463 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:38.464 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:38.465 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.466 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:38.467 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:38.468 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:38.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.178:6800/2549949439 conn(0x7fc95c03d880 legacy=0x7fc95c05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 10 monclient: shutdown 2026-02-21T22:21:38.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 >> v1:10.20.193.102:6789/0 conn(0x7fc98c085d40 legacy=0x7fc98c183b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc99197f640 1 -- 10.20.193.102:0/2742644371 reap_dead start 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc99197f640 1 -- 10.20.193.102:0/2742644371 reap_dead start 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 shutdown_connections 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 >> 10.20.193.102:0/2742644371 conn(0x7fc98c054e40 msgr2=0x7fc98c087790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 shutdown_connections 2026-02-21T22:21:38.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.433+0000 7fc993409640 1 -- 10.20.193.102:0/2742644371 wait complete. 2026-02-21T22:21:38.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:38.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:21:38.470 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:21:38.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.525+0000 7f8740297640 10 monclient: get_monmap_and_config 2026-02-21T22:21:38.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.525+0000 7f8740297640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.525+0000 7f8740297640 10 monclient: monmap: 2026-02-21T22:21:38.532 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.532 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:38.532681+0000 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:38.532681+0000 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 Processor -- start 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- start start 2026-02-21T22:21:38.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient: init 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient(hunting): picked mon.noname-a con 0x7f873808ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient(hunting): picked mon.noname-b con 0x7f873808b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient(hunting): picked mon.noname-c con 0x7f873808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8738086a10 con 0x7f873808ce00 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873808b130 con 0x7f873808d8e0 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873808bd60 con 0x7f873808b7f0 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient(hunting): authenticate will time out at 1507.768558s 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d80b640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f873808b7f0 0x7f8738181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44902/0 (socket says 10.20.193.102:44902) 2026-02-21T22:21:38.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d80b640 1 -- 10.20.193.102:0/3106726377 learned_addr learned my addr 10.20.193.102:0/3106726377 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d80b640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.102:6790/0 conn(0x7f873808b7f0 legacy=0x7f8738181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873e00c640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.102:6789/0 conn(0x7f873808ce00 legacy=0x7f873808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186529033 0 0) 0x7f873808bd60 con 0x7f873808b7f0 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): my global_id is 4697 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8720003620 con 0x7f873808b7f0 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216835282 0 0) 0x7f8738086a10 con 0x7f873808ce00 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient(hunting): my global_id is 34335 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f873808bd60 con 0x7f873808ce00 2026-02-21T22:21:38.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303537784 0 0) 0x7f8720003620 con 0x7f873808b7f0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873e80d640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.178:6789/0 conn(0x7f873808d8e0 legacy=0x7f87381a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.178:6789/0 conn(0x7f873808d8e0 legacy=0x7f87381a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.102:6789/0 conn(0x7f873808ce00 legacy=0x7f873808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 monclient: found mon.noname-b 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873808b370 con 0x7f873808b7f0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f872c003110 con 0x7f873808b7f0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 5 monclient: authenticate success, global_id 4697 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: dump: 2026-02-21T22:21:38.538 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f872c003460 con 0x7f873808b7f0 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 1 -- 10.20.193.102:0/3106726377 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f872c004b50 con 0x7f873808b7f0 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f873d00a640 10 monclient: dump: 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.540 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 4 set_mon_vals no callback set 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 10 monclient: shutdown 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- 10.20.193.102:0/3106726377 >> v1:10.20.193.102:6790/0 conn(0x7f873808b7f0 legacy=0x7f8738181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- 10.20.193.102:0/3106726377 shutdown_connections 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- 10.20.193.102:0/3106726377 >> 10.20.193.102:0/3106726377 conn(0x7f8738091a50 msgr2=0x7f8738091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- 10.20.193.102:0/3106726377 shutdown_connections 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.529+0000 7f8740297640 1 -- 10.20.193.102:0/3106726377 wait complete. 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: monmap: 2026-02-21T22:21:38.541 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 Processor -- start 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- start start 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: init 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient(hunting): picked mon.noname-a con 0x7f873808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient(hunting): picked mon.noname-c con 0x7f873808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient(hunting): picked mon.noname-b con 0x7f873808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873808a7e0 con 0x7f873808b7f0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8738083470 con 0x7f873808d8e0 2026-02-21T22:21:38.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8738088a70 con 0x7f873808ce00 2026-02-21T22:21:38.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient(hunting): authenticate will time out at 1507.773127s 2026-02-21T22:21:38.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873d80b640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f873808ce00 0x7f87381944e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:44910/0 (socket says 10.20.193.102:44910) 2026-02-21T22:21:38.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873d80b640 1 -- 10.20.193.102:0/3765042897 learned_addr learned my addr 10.20.193.102:0/3765042897 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873e00c640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f873808b7f0 0x7f87381505b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55134/0 (socket says 10.20.193.102:55134) 2026-02-21T22:21:38.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873d80b640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.102:6790/0 conn(0x7f873808ce00 legacy=0x7f87381944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873e00c640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.102:6789/0 conn(0x7f873808b7f0 legacy=0x7f87381505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897205658 0 0) 0x7f8738088a70 con 0x7f873808ce00 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): my global_id is 4700 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8708003620 con 0x7f873808ce00 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037595455 0 0) 0x7f873808a7e0 con 0x7f873808b7f0 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): my global_id is 34338 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8738088a70 con 0x7f873808b7f0 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f873e80d640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f873808d8e0 legacy=0x7f8738195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3972002579 0 0) 0x7f8708003620 con 0x7f873808ce00 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f873808a7e0 con 0x7f873808ce00 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1233961071 0 0) 0x7f8738088a70 con 0x7f873808b7f0 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8708003620 con 0x7f873808b7f0 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f872c002b70 con 0x7f873808ce00 2026-02-21T22:21:38.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): dump: 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126599109 0 0) 0x7f8738083470 con 0x7f873808d8e0 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): my global_id is 54145 2026-02-21T22:21:38.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8738088a70 con 0x7f873808d8e0 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f87280033f0 con 0x7f873808b7f0 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient(hunting): dump: 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 741151717 0 0) 0x7f873808a7e0 con 0x7f873808ce00 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f873808d8e0 legacy=0x7f8738195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.102:6789/0 conn(0x7f873808b7f0 legacy=0x7f87381505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 monclient: found mon.c 2026-02-21T22:21:38.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8738083190 con 0x7f873808ce00 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f871effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 5 monclient: authenticate success, global_id 4700 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _renew_subs 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8738150cc0 con 0x7f873808ce00 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _renew_subs 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.533+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f8738084020 con 0x7f873808ce00 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f872c003460 con 0x7f873808ce00 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f872c005190 con 0x7f873808ce00 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f873c809640 4 set_mon_vals no callback set 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: dump: 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.551 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f8740297640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f873808a190 con 0x7f8738089dc0 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f872c005430 con 0x7f873808ce00 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7f872c04c740 con 0x7f873808ce00 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f873e80d640 10 monclient: get_auth_request con 0x7f870803d820 auth_method 0 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f873e00c640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f8738089dc0 legacy=0x7f873805b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696910592 0 0) 0x7f873808a190 con 0x7f8738089dc0 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient(hunting): my global_id is 54148 2026-02-21T22:21:38.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f873808a7e0 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2329745203 0 0) 0x7f873808a7e0 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f873808a190 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8728003de0 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: dump: 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1461932669 0 0) 0x7f873808a190 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87380008e0 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f87280040b0 con 0x7f8738089dc0 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:38.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f8738089dc0 legacy=0x7f873805b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.537+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f8738089dc0 legacy=0x7f873805b8e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.541+0000 7f8740297640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:38.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.541+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8738090810 con 0x7f873804d280 2026-02-21T22:21:38.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:38.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:38.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f873e00c640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f873804d280 legacy=0x7f8738198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109741262 0 0) 0x7f8738090810 con 0x7f873804d280 2026-02-21T22:21:38.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient(hunting): my global_id is 54151 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f873808a190 con 0x7f873804d280 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3152846904 0 0) 0x7f873808a190 con 0x7f873804d280 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8738090810 con 0x7f873804d280 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8728005a70 con 0x7f873804d280 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient: dump: 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.556 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2060680832 0 0) 0x7f8738090810 con 0x7f873804d280 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87380909e0 con 0x7f873804d280 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 2361845181) 0x7f8728007110 con 0x7f873804d280 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f873804d280 legacy=0x7f8738198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f871effd640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6789/0 conn(0x7f873804d280 legacy=0x7f8738198330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 26, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 2195, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:38.558 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.559 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:38.560 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.561 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.562 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:38.563 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:38.564 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:38.565 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.178:6800/2549949439 conn(0x7f870803d820 legacy=0x7f870805dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 10 monclient: shutdown 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 >> v1:10.20.193.102:6790/0 conn(0x7f873808ce00 legacy=0x7f87381944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f873e80d640 1 -- 10.20.193.102:0/3765042897 reap_dead start 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f873e80d640 1 -- 10.20.193.102:0/3765042897 reap_dead start 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 shutdown_connections 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 >> 10.20.193.102:0/3765042897 conn(0x7f8738091a50 msgr2=0x7f873805d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 shutdown_connections 2026-02-21T22:21:38.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.545+0000 7f8740297640 1 -- 10.20.193.102:0/3765042897 wait complete. 2026-02-21T22:21:38.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:21:38.567 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-21T22:21:38.567 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:21:38.567 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-21T22:21:38.567 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-21T22:21:38.567 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-21T22:21:38.567 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:21:38.581 INFO:tasks.ceph.mon.c:Stopped 2026-02-21T22:21:38.581 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:21:38.581 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:21:38.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:21:38.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:21:38.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:21:38.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:21:38.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:21:38.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:38.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:21:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:38.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: get_monmap_and_config 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: monmap: 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:38.682991+0000 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:38.682991+0000 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.683 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 1 Processor -- start 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 1 -- start start 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: init 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-c con 0x7fcae8142a40 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-a con 0x7fcae80900c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-b con 0x7fcae8143520 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae8057bf0 con 0x7fcae80900c0 2026-02-21T22:21:38.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae80563d0 con 0x7fcae8142a40 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae8057350 con 0x7fcae8143520 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcaede7a640 10 monclient(hunting): authenticate will time out at 1507.918925s 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcae7fff640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fcae8143520 legacy=0x7fcae81a5740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcae6ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcae80900c0 0x7fcae8184010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55154/0 (socket says 10.20.193.102:55154) 2026-02-21T22:21:38.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.677+0000 7fcae6ffd640 1 -- 10.20.193.102:0/2506458811 learned_addr learned my addr 10.20.193.102:0/2506458811 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae6ffd640 1 -- 10.20.193.102:0/2506458811 >> v1:10.20.193.102:6789/0 conn(0x7fcae80900c0 legacy=0x7fcae8184010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202712563 0 0) 0x7fcae8057bf0 con 0x7fcae80900c0 2026-02-21T22:21:38.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae77fe640 1 -- 10.20.193.102:0/2506458811 >> v1:10.20.193.178:6789/0 conn(0x7fcae8142a40 legacy=0x7fcae8142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient(hunting): my global_id is 34341 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcacc003620 con 0x7fcae80900c0 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 958693185 0 0) 0x7fcacc003620 con 0x7fcae80900c0 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 >> v1:10.20.193.102:6790/0 conn(0x7fcae8143520 legacy=0x7fcae81a5740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 >> v1:10.20.193.178:6789/0 conn(0x7fcae8142a40 legacy=0x7fcae8142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 monclient: found mon.noname-a 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae8057630 con 0x7fcae80900c0 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 5 monclient: authenticate success, global_id 34341 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcad8003110 con 0x7fcae80900c0 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: dump: 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.696 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcad80033e0 con 0x7fcae80900c0 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 1 -- 10.20.193.102:0/2506458811 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcad8004b50 con 0x7fcae80900c0 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 4 set_mon_vals no callback set 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae67fc640 10 monclient: dump: 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.698 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: shutdown 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- 10.20.193.102:0/2506458811 >> v1:10.20.193.102:6789/0 conn(0x7fcae80900c0 legacy=0x7fcae8184010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- 10.20.193.102:0/2506458811 shutdown_connections 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- 10.20.193.102:0/2506458811 >> 10.20.193.102:0/2506458811 conn(0x7fcae805e4c0 msgr2=0x7fcae8095e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- 10.20.193.102:0/2506458811 shutdown_connections 2026-02-21T22:21:38.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- 10.20.193.102:0/2506458811 wait complete. 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: build_initial_monmap 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: monmap: 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:38.700 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 Processor -- start 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- start start 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: init 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-b con 0x7fcae80900c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-c con 0x7fcae8142a40 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient(hunting): picked mon.noname-a con 0x7fcae8143520 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae8059190 con 0x7fcae8143520 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae81321e0 con 0x7fcae80900c0 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae81324c0 con 0x7fcae8142a40 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcaede7a640 10 monclient(hunting): authenticate will time out at 1507.922909s 2026-02-21T22:21:38.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae6ffd640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fcae8142a40 legacy=0x7fcae8140ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae7fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcae8143520 0x7fcae81956f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55162/0 (socket says 10.20.193.102:55162) 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.681+0000 7fcae7fff640 1 -- 10.20.193.102:0/1772873749 learned_addr learned my addr 10.20.193.102:0/1772873749 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcae7fff640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.102:6789/0 conn(0x7fcae8143520 legacy=0x7fcae81956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcae77fe640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.178:6789/0 conn(0x7fcae80900c0 legacy=0x7fcae8153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104940451 0 0) 0x7fcadc005300 con 0x7fcae8143520 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): my global_id is 34344 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcab8003620 con 0x7fcae8143520 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3342534484 0 0) 0x7fcae81321e0 con 0x7fcae80900c0 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _negotiate 2026-02-21T22:21:38.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): my global_id is 54157 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcadc005300 con 0x7fcae80900c0 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 224478384 0 0) 0x7fcab8003620 con 0x7fcae8143520 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcae81321e0 con 0x7fcae8143520 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcadc005b00 con 0x7fcae8143520 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): dump: 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.703 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3286005705 0 0) 0x7fcae81321e0 con 0x7fcae8143520 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.102:6790/0 conn(0x7fcae8142a40 legacy=0x7fcae8140ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.178:6789/0 conn(0x7fcae80900c0 legacy=0x7fcae8153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: _finish_hunting 0 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 monclient: found mon.a 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:38.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae8131180 con 0x7fcae8143520 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: _finish_auth 0 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: _check_auth_tickets 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 5 monclient: authenticate success, global_id 34344 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _renew_subs 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcae8130160 con 0x7fcae8143520 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _renew_subs 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fcae81413a0 con 0x7fcae8143520 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcadc005dd0 con 0x7fcae8143520 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:38.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcadc006130 con 0x7fcae8143520 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 10 monclient: dump: 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:38.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcae5ffb640 4 set_mon_vals no callback set 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae8131c20 con 0x7fcae8143520 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fcadc006400 con 0x7fcae8143520 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7fcadc04dcd0 con 0x7fcae8143520 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.685+0000 7fcae77fe640 10 monclient: get_auth_request con 0x7fcab803d470 auth_method 0 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.689+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fcae8059190 con 0x7fcae8143520 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.689+0000 7fcac3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:38.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.689+0000 7fcac3fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcae805eb70 con 0x7fcae8143520 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcac3fff640 1 -- 10.20.193.102:0/1772873749 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 214369608) 0x7fcadc018cc0 con 0x7fcae8143520 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcac3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:38.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcac3fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:38.811 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.178:6800/2549949439 conn(0x7fcab803d470 legacy=0x7fcab805d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 10 monclient: shutdown 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 >> v1:10.20.193.102:6789/0 conn(0x7fcae8143520 legacy=0x7fcae81956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 shutdown_connections 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 >> 10.20.193.102:0/1772873749 conn(0x7fcae805e4c0 msgr2=0x7fcae808f740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 shutdown_connections 2026-02-21T22:21:38.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:38.805+0000 7fcaede7a640 1 -- 10.20.193.102:0/1772873749 wait complete. 2026-02-21T22:21:41.822 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:21:41.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: get_monmap_and_config 2026-02-21T22:21:41.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: build_initial_monmap 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: monmap: 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:41.894933+0000 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:41.894933+0000 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:41.895 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 1 Processor -- start 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 1 -- start start 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: init 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-b con 0x7f232008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-c con 0x7f232008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-a con 0x7f232008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2320086a10 con 0x7f232008d8e0 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232008b130 con 0x7f232008b7f0 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232008bd60 con 0x7f232008ce00 2026-02-21T22:21:41.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:41.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f23284f1640 10 monclient(hunting): authenticate will time out at 1511.130812s 2026-02-21T22:21:41.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f2326266640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f232008ce00 legacy=0x7f232008d1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:41.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f2326a67640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 0x7f23201a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:53752/0 (socket says 10.20.193.102:53752) 2026-02-21T22:21:41.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.889+0000 7f2326a67640 1 -- 10.20.193.102:0/2820644100 learned_addr learned my addr 10.20.193.102:0/2820644100 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2326a67640 1 -- 10.20.193.102:0/2820644100 >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 legacy=0x7f23201a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325a65640 1 -- 10.20.193.102:0/2820644100 >> v1:10.20.193.178:6789/0 conn(0x7f232008b7f0 legacy=0x7f2320181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224846260 0 0) 0x7f2320086a10 con 0x7f232008d8e0 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient(hunting): _negotiate 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient(hunting): my global_id is 34347 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2308003620 con 0x7f232008d8e0 2026-02-21T22:21:41.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4195731867 0 0) 0x7f2308003620 con 0x7f232008d8e0 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 >> v1:10.20.193.102:6790/0 conn(0x7f232008ce00 legacy=0x7f232008d1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 >> v1:10.20.193.178:6789/0 conn(0x7f232008b7f0 legacy=0x7f2320181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: _finish_hunting 0 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 monclient: found mon.noname-a 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232008b370 con 0x7f232008d8e0 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: _finish_auth 0 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: _check_auth_tickets 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 5 monclient: authenticate success, global_id 34347 2026-02-21T22:21:41.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f231c004620 con 0x7f232008d8e0 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: dump: 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:41.905 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f231c0048f0 con 0x7f232008d8e0 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 1 -- 10.20.193.102:0/2820644100 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f231c006060 con 0x7f232008d8e0 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325264640 10 monclient: dump: 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:41.906 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 4 set_mon_vals no callback set 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: shutdown 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:41.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- 10.20.193.102:0/2820644100 >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 legacy=0x7f23201a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- 10.20.193.102:0/2820644100 shutdown_connections 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- 10.20.193.102:0/2820644100 >> 10.20.193.102:0/2820644100 conn(0x7f2320091a50 msgr2=0x7f2320091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- 10.20.193.102:0/2820644100 shutdown_connections 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- 10.20.193.102:0/2820644100 wait complete. 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: build_initial_monmap 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: monmap: 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:41.908 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 Processor -- start 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- start start 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: init 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-b con 0x7f232008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-c con 0x7f232008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient(hunting): picked mon.noname-a con 0x7f232008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:41.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2320080f30 con 0x7f232008d8e0 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2320088a70 con 0x7f232008b7f0 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f232005d8d0 con 0x7f232008ce00 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f23284f1640 10 monclient(hunting): authenticate will time out at 1511.134789s 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2325a65640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f232008ce00 legacy=0x7f2320192380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2326a67640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 0x7f2320193aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:53760/0 (socket says 10.20.193.102:53760) 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.893+0000 7f2326a67640 1 -- 10.20.193.102:0/3578482961 learned_addr learned my addr 10.20.193.102:0/3578482961 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2326a67640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 legacy=0x7f2320193aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694848071 0 0) 0x7f2320080f30 con 0x7f232008d8e0 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:41.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): my global_id is 34350 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f0003620 con 0x7f232008d8e0 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2326266640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.178:6789/0 conn(0x7f232008b7f0 legacy=0x7f2320151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4067810058 0 0) 0x7f22f0003620 con 0x7f232008d8e0 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2320080f30 con 0x7f232008d8e0 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f231c005df0 con 0x7f232008d8e0 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): dump: 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:41.911 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3430129217 0 0) 0x7f2320088a70 con 0x7f232008b7f0 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:41.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient(hunting): my global_id is 54163 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f0003620 con 0x7f232008b7f0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3198056441 0 0) 0x7f2320080f30 con 0x7f232008d8e0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.102:6790/0 conn(0x7f232008ce00 legacy=0x7f2320192380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.178:6789/0 conn(0x7f232008b7f0 legacy=0x7f2320151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 monclient: found mon.a 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2320080c50 con 0x7f232008d8e0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: _finish_auth 0 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:41.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 5 monclient: authenticate success, global_id 34350 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _renew_subs 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f232007f730 con 0x7f232008d8e0 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _renew_subs 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f232007e390 con 0x7f232008d8e0 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2326a67640 1 --1- 10.20.193.102:0/3578482961 >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 0x7f2320193aa0 :-1 s=OPENED pgs=275 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f231c006030 config(0 keys) 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f231c006030 con 0x7f232008d8e0 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2324a63640 4 set_mon_vals no callback set 2026-02-21T22:21:41.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2320080820 con 0x7f232008d8e0 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f231c004a40 con 0x7f232008d8e0 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 10 monclient: dump: 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:41.915 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f231c012b70 con 0x7f232008d8e0 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.897+0000 7f2326266640 10 monclient: get_auth_request con 0x7f22f003d470 auth_method 0 2026-02-21T22:21:41.916 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:41.901+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7f231c04dc20 con 0x7f232008d8e0 2026-02-21T22:21:42.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.729+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f231c018c20 con 0x7f232008d8e0 2026-02-21T22:21:42.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.729+0000 7f2306ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:42.729 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.729+0000 7f2306ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:42.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:21:42.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2320062ce0 con 0x7f232008d8e0 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f2306ffd640 1 -- 10.20.193.102:0/3578482961 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1380902088) 0x7f231c01de70 con 0x7f232008d8e0 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f2306ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f2306ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:42.845 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":26,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:21:42.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.178:6800/2549949439 conn(0x7f22f003d470 legacy=0x7f22f005d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:42.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 10 monclient: shutdown 2026-02-21T22:21:42.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:42.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 >> v1:10.20.193.102:6789/0 conn(0x7f232008d8e0 legacy=0x7f2320193aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:42.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 shutdown_connections 2026-02-21T22:21:42.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 >> 10.20.193.102:0/3578482961 conn(0x7f2320091a50 msgr2=0x7f2320058f70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:42.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 shutdown_connections 2026-02-21T22:21:42.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:42.841+0000 7f23284f1640 1 -- 10.20.193.102:0/3578482961 wait complete. 2026-02-21T22:21:43.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:43.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:45.856 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:21:45.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: get_monmap_and_config 2026-02-21T22:21:45.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: build_initial_monmap 2026-02-21T22:21:45.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: monmap: 2026-02-21T22:21:45.923 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:45.923 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:45.923006+0000 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:45.923006+0000 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 1 Processor -- start 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 1 -- start start 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: init 2026-02-21T22:21:45.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-c con 0x7f9afc13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-a con 0x7f9afc13fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-b con 0x7f9afc1401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc092490 con 0x7f9afc13fd90 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc0571f0 con 0x7f9afc13f9c0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc0574d0 con 0x7f9afc1401f0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afbfff640 10 monclient(hunting): authenticate will time out at 1515.159037s 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afb7fe640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:45.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afa7fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9afc13fd90 0x7f9afc183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:53762/0 (socket says 10.20.193.102:53762) 2026-02-21T22:21:45.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.917+0000 7f9afa7fc640 1 -- 10.20.193.102:0/2900819918 learned_addr learned my addr 10.20.193.102:0/2900819918 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:45.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.921+0000 7f9afa7fc640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9afc13fd90 legacy=0x7f9afc183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:45.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:45.921+0000 7f9afaffd640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9afc0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:46.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:46.121+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:46.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:46.525+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:47.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:47.329+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:48.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:48.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:48.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): tick 2026-02-21T22:21:48.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 monclient(hunting): continuing hunt 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1a5470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9afc0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9afc13fd90 legacy=0x7f9afc183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f9afc13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f9afc13fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f9afc1401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:48.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc0577b0 con 0x7f9afc13fd90 2026-02-21T22:21:48.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc0574d0 con 0x7f9afc13f9c0 2026-02-21T22:21:48.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae006a090 con 0x7f9afc1401f0 2026-02-21T22:21:48.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9af8ff9640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:48.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.917+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:48.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.921+0000 7f9afa7fc640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9afc13fd90 legacy=0x7f9ae00450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:48.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:48.921+0000 7f9afaffd640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9ae00238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:49.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:49.121+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:49.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:49.525+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:50.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:50.329+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:51.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): tick 2026-02-21T22:21:51.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:21:51.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 monclient(hunting): continuing hunt 2026-02-21T22:21:51.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:21:51.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9ae00238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9afc13fd90 legacy=0x7f9ae00450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f9ae006a410 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f9afc13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f9afc1401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae006b3f0 con 0x7f9ae006a410 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae006a090 con 0x7f9afc13f9c0 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae0025010 con 0x7f9afc1401f0 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9af8ff9640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:51.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:51.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9afaffd640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9ae006a410 legacy=0x7f9ae006a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:51.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:51.921+0000 7f9afa7fc640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9ae00238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:52.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:52.125+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:52.528 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:52.529+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.333+0000 7f9afb7fe640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792896803 0 0) 0x7f9ae006a090 con 0x7f9afc13f9c0 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient(hunting): my global_id is 54166 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ae8003620 con 0x7f9afc13f9c0 2026-02-21T22:21:53.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3400382696 0 0) 0x7f9ae8003620 con 0x7f9afc13f9c0 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9ae0066870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.102:6789/0 conn(0x7f9ae006a410 legacy=0x7f9ae006a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 monclient: found mon.noname-c 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae0026290 con 0x7f9afc13f9c0 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ae4003070 con 0x7f9afc13f9c0 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: dump: 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.452 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9afbfff640 5 monclient: authenticate success, global_id 54166 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9afbfff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9ae4003300 con 0x7f9afc13f9c0 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9afbfff640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9afbfff640 4 set_mon_vals no callback set 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 1 -- 10.20.193.102:0/2900819918 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ae40049a0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.441+0000 7f9af9ffb640 10 monclient: dump: 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.453 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: shutdown 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- 10.20.193.102:0/2900819918 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9ae00238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- 10.20.193.102:0/2900819918 shutdown_connections 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- 10.20.193.102:0/2900819918 >> 10.20.193.102:0/2900819918 conn(0x7f9afc05e3f0 msgr2=0x7f9afc095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- 10.20.193.102:0/2900819918 shutdown_connections 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- 10.20.193.102:0/2900819918 wait complete. 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: build_initial_monmap 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: monmap: 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:53.455 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 Processor -- start 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- start start 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: init 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-a con 0x7f9ae006a410 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:53.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-b con 0x7f9afc13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient(hunting): picked mon.noname-c con 0x7f9afc1401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc128750 con 0x7f9ae006a410 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc12a9f0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9afc12acd0 con 0x7f9afc1401f0 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afbfff640 10 monclient(hunting): authenticate will time out at 1522.685995s 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afb7fe640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1199c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afaffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ae006a410 0x7f9afc127930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:42238/0 (socket says 10.20.193.102:42238) 2026-02-21T22:21:53.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.445+0000 7f9afaffd640 1 -- 10.20.193.102:0/740352591 learned_addr learned my addr 10.20.193.102:0/740352591 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:53.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afaffd640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.102:6789/0 conn(0x7f9ae006a410 legacy=0x7f9afc127930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afa7fc640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9afc128040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556348217 0 0) 0x7f9afc12a9f0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): my global_id is 54169 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac8003620 con 0x7f9afc13f9c0 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 728494062 0 0) 0x7f9ac8003620 con 0x7f9afc13f9c0 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9afc12a9f0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ae40034b0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): dump: 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.461 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3444016382 0 0) 0x7f9afc12a9f0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.102:6790/0 conn(0x7f9afc1401f0 legacy=0x7f9afc1199c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:21:53.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.102:6789/0 conn(0x7f9ae006a410 legacy=0x7f9afc127930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 monclient: found mon.b 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9afc129cf0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 5 monclient: authenticate success, global_id 54169 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _renew_subs 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9afc128f70 con 0x7f9afc13f9c0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _renew_subs 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9afc12b910 con 0x7f9afc13f9c0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9ae4004e30 con 0x7f9afc13f9c0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ae4005190 con 0x7f9afc13f9c0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9af97fa640 4 set_mon_vals no callback set 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 10 monclient: dump: 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.464 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9afc0008e0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9ae4011820 con 0x7f9afc13f9c0 2026-02-21T22:21:53.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(302..302 src has 112..302) ==== 3943+0+0 (unknown 3955508734 0 0) 0x7f9ae404c420 con 0x7f9afc13f9c0 2026-02-21T22:21:53.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.449+0000 7f9afaffd640 10 monclient: get_auth_request con 0x7f9ac803d3c0 auth_method 0 2026-02-21T22:21:53.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.453+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9ae40174d0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.453+0000 7f9adb7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.453+0000 7f9adb7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:53.523 INFO:tasks.rados.rados.0.trial178.stdout:1433: done (3 left) 2026-02-21T22:21:53.524 INFO:tasks.rados.rados.0.trial178.stdout:1434: done (2 left) 2026-02-21T22:21:53.524 INFO:tasks.rados.rados.0.trial178.stdout:1435: done (1 left) 2026-02-21T22:21:53.524 INFO:tasks.rados.rados.0.trial178.stdout:1436: done (0 left) 2026-02-21T22:21:53.524 INFO:tasks.rados.rados.0.trial178.stdout:1437: snap_remove snap 142 2026-02-21T22:21:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:21:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:21:53.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:53.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:53.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.569+0000 7f9afbfff640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.569+0000 7f9afbfff640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.569+0000 7f9afbfff640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.569+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9afc093ce0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9adb7fe640 1 -- 10.20.193.102:0/740352591 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 1208828304) 0x7f9ae4016db0 con 0x7f9afc13f9c0 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9adb7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:21:53.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9adb7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:53.577 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":28,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:21:53.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9ac803d3c0 legacy=0x7f9ac805d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 10 monclient: shutdown 2026-02-21T22:21:53.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 >> v1:10.20.193.178:6789/0 conn(0x7f9afc13f9c0 legacy=0x7f9afc128040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 shutdown_connections 2026-02-21T22:21:53.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 >> 10.20.193.102:0/740352591 conn(0x7f9afc05e3f0 msgr2=0x7f9afc13eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 shutdown_connections 2026-02-21T22:21:53.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.573+0000 7f9afbfff640 1 -- 10.20.193.102:0/740352591 wait complete. 2026-02-21T22:21:53.587 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:21:53.587 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:21:53.588 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:21:53.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: get_monmap_and_config 2026-02-21T22:21:53.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: build_initial_monmap 2026-02-21T22:21:53.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: monmap: 2026-02-21T22:21:53.659 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:53.659473+0000 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:53.659473+0000 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 1 Processor -- start 2026-02-21T22:21:53.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 1 -- start start 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: init 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c086a10 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c08b130 con 0x7f2a3c08ce00 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c08bd60 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a40877640 10 monclient(hunting): authenticate will time out at 1522.895394s 2026-02-21T22:21:53.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a39d74640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a3ad76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08d8e0 0x7f2a3c1a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:42240/0 (socket says 10.20.193.102:42240) 2026-02-21T22:21:53.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.653+0000 7f2a3ad76640 1 -- 10.20.193.102:0/4253024700 learned_addr learned my addr 10.20.193.102:0/4253024700 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a3ad76640 1 -- 10.20.193.102:0/4253024700 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 474573331 0 0) 0x7f2a3c086a10 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.665 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient(hunting): my global_id is 34353 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a28003620 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a3a575640 1 -- 10.20.193.102:0/4253024700 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 648920124 0 0) 0x7f2a28003620 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 monclient: found mon.noname-a 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c08b370 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 5 monclient: authenticate success, global_id 34353 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a2c004620 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: dump: 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.667 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a2c0048f0 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 1 -- 10.20.193.102:0/4253024700 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a2c006060 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a39573640 10 monclient: dump: 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.669 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 4 set_mon_vals no callback set 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: shutdown 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- 10.20.193.102:0/4253024700 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- 10.20.193.102:0/4253024700 shutdown_connections 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- 10.20.193.102:0/4253024700 >> 10.20.193.102:0/4253024700 conn(0x7f2a3c091a50 msgr2=0x7f2a3c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- 10.20.193.102:0/4253024700 shutdown_connections 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- 10.20.193.102:0/4253024700 wait complete. 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: build_initial_monmap 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: monmap: 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:53.670 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 Processor -- start 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 1 -- start start 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: init 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.657+0000 7f2a40877640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:53.671 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c05da20 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c082fc0 con 0x7f2a3c08ce00 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c0832a0 con 0x7f2a3c08d8e0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient(hunting): authenticate will time out at 1522.900070s 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a3ad76640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c195330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.672 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a3a575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 0x7f2a3c152fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:42250/0 (socket says 10.20.193.102:42250) 2026-02-21T22:21:53.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a3a575640 1 -- 10.20.193.102:0/3149108822 learned_addr learned my addr 10.20.193.102:0/3149108822 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:53.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a3a575640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736755745 0 0) 0x7f2a3c08b130 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): my global_id is 34356 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a08003620 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a39d74640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c0847b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 325226650 0 0) 0x7f2a08003620 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a3c08b130 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a30003420 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): dump: 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.679 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 848719712 0 0) 0x7f2a3c08b130 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c195330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:21:53.680 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c0847b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 monclient: found mon.a 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c058d40 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 5 monclient: authenticate success, global_id 34356 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient: _renew_subs 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c088a70 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient: _renew_subs 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2a3c084ec0 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a300039c0 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a30003d20 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 10 monclient: dump: 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.682 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a38d72640 4 set_mon_vals no callback set 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2a3c0008e0 con 0x7f2a3c07a370 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a3a575640 1 --1- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 0x7f2a3c152fa0 :-1 s=OPENED pgs=283 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f2a30003ff0 mgrmap(e 5) 2026-02-21T22:21:53.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.661+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2a30003ff0 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a3ad76640 10 monclient: get_auth_request con 0x7f2a0803d340 auth_method 0 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a39d74640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07a370 legacy=0x7f2a3c07a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193983524 0 0) 0x7f2a3c0008e0 con 0x7f2a3c07a370 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient(hunting): my global_id is 34359 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a3c08b130 con 0x7f2a3c07a370 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 523382753 0 0) 0x7f2a3c08b130 con 0x7f2a3c07a370 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a3c0008e0 con 0x7f2a3c07a370 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a240041d0 con 0x7f2a3c07a370 2026-02-21T22:21:53.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient: dump: 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.685 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1529948586 0 0) 0x7f2a3c0008e0 con 0x7f2a3c07a370 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a3c090810 con 0x7f2a3c07a370 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2a240065b0 con 0x7f2a3c07a370 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07a370 legacy=0x7f2a3c07a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.665+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07a370 legacy=0x7f2a3c07a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.669+0000 7f2a40877640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.669+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2a3c0909f0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a39d74640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07ae50 legacy=0x7f2a3c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4085059272 0 0) 0x7f2a3c0909f0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.686 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient(hunting): my global_id is 34362 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a3c0008e0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 115556561 0 0) 0x7f2a3c0008e0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a3c0909f0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a24004970 con 0x7f2a3c07ae50 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient: dump: 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.687 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3779119588 0 0) 0x7f2a3c0909f0 con 0x7f2a3c07ae50 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2a3c07b250 con 0x7f2a3c07ae50 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 1860698604) 0x7f2a24005d90 con 0x7f2a3c07ae50 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:53.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07ae50 legacy=0x7f2a3c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c07ae50 legacy=0x7f2a3c197a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 28, 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 262542, 2026-02-21T22:21:53.689 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.690 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.691 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:53.692 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:53.693 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.694 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.695 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.696 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:53.697 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.698 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.699 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:53.700 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.178:6800/2549949439 conn(0x7f2a0803d340 legacy=0x7f2a0805d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 10 monclient: shutdown 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a3ad76640 1 -- 10.20.193.102:0/3149108822 reap_dead start 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a22ffd640 1 -- 10.20.193.102:0/3149108822 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(303..303 src has 112..303) ==== 3915+0+0 (unknown 3665463721 0 0) 0x7f2a3004c7e0 con 0x7f2a3c08b7f0 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a3ad76640 1 -- 10.20.193.102:0/3149108822 reap_dead start 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 shutdown_connections 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 >> 10.20.193.102:0/3149108822 conn(0x7f2a3c091a50 msgr2=0x7f2a3c05a110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 shutdown_connections 2026-02-21T22:21:53.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.673+0000 7f2a40877640 1 -- 10.20.193.102:0/3149108822 wait complete. 2026-02-21T22:21:53.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:21:53.703 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-21T22:21:53.703 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:21:53.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: get_monmap_and_config 2026-02-21T22:21:53.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: build_initial_monmap 2026-02-21T22:21:53.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: monmap: 2026-02-21T22:21:53.766 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:21:53.767941+0000 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:21:53.767941+0000 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:21:53.767 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 1 Processor -- start 2026-02-21T22:21:53.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 1 -- start start 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: init 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-a con 0x7f5b3408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-b con 0x7f5b3408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:53.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-c con 0x7f5b3408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b34086a10 con 0x7f5b3408ce00 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3408b130 con 0x7f5b3408d8e0 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3408bd60 con 0x7f5b3408b7f0 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b39b4d640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f5b3408b7f0 legacy=0x7f5b34181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3c5d9640 10 monclient(hunting): authenticate will time out at 1523.003918s 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.761+0000 7f5b3a34e640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 0x7f5b3408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:42264/0 (socket says 10.20.193.102:42264) 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3a34e640 1 -- 10.20.193.102:0/2476125656 learned_addr learned my addr 10.20.193.102:0/2476125656 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3a34e640 1 -- 10.20.193.102:0/2476125656 >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 legacy=0x7f5b3408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131109537 0 0) 0x7f5b34086a10 con 0x7f5b3408ce00 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient(hunting): my global_id is 34365 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b1c003620 con 0x7f5b3408ce00 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3ab4f640 1 -- 10.20.193.102:0/2476125656 >> v1:10.20.193.178:6789/0 conn(0x7f5b3408d8e0 legacy=0x7f5b341a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428895278 0 0) 0x7f5b1c003620 con 0x7f5b3408ce00 2026-02-21T22:21:53.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 >> v1:10.20.193.102:6790/0 conn(0x7f5b3408b7f0 legacy=0x7f5b34181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:21:53.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 >> v1:10.20.193.178:6789/0 conn(0x7f5b3408d8e0 legacy=0x7f5b341a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 monclient: found mon.noname-a 2026-02-21T22:21:53.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3408b370 con 0x7f5b3408ce00 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b24003110 con 0x7f5b3408ce00 2026-02-21T22:21:53.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: dump: 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.788 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.789 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 5 monclient: authenticate success, global_id 34365 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b240033e0 con 0x7f5b3408ce00 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 1 -- 10.20.193.102:0/2476125656 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b24004b50 con 0x7f5b3408ce00 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3934c640 10 monclient: dump: 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.791 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 10 monclient: get_monmap_and_config success 2026-02-21T22:21:53.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 4 set_mon_vals no callback set 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 10 monclient: shutdown 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/2476125656 >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 legacy=0x7f5b3408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/2476125656 shutdown_connections 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/2476125656 >> 10.20.193.102:0/2476125656 conn(0x7f5b34091a50 msgr2=0x7f5b34091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/2476125656 shutdown_connections 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/2476125656 wait complete. 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 10 monclient: build_initial_monmap 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 10 monclient: monmap: 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:21:53.794 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:21:53.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.765+0000 7f5b3c5d9640 1 Processor -- start 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- start start 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: init 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _reopen_session rank -1 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:21:53.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-c con 0x7f5b3408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-a con 0x7f5b3408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient(hunting): picked mon.noname-b con 0x7f5b3408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3a34e640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f5b3408b7f0 legacy=0x7f5b34151010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3407ee60 con 0x7f5b3408ce00 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b34058d10 con 0x7f5b3408d8e0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b34088a70 con 0x7f5b3408b7f0 2026-02-21T22:21:53.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient(hunting): _renew_subs 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient(hunting): authenticate will time out at 1523.008445s 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b39b4d640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 0x7f5b34084710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:42278/0 (socket says 10.20.193.102:42278) 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b39b4d640 1 -- 10.20.193.102:0/3227978436 learned_addr learned my addr 10.20.193.102:0/3227978436 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b39b4d640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 legacy=0x7f5b34084710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500944009 0 0) 0x7f5b3407ee60 con 0x7f5b3408ce00 2026-02-21T22:21:53.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): my global_id is 34368 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b04003620 con 0x7f5b3408ce00 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3ab4f640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3408d8e0 legacy=0x7f5b34193380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280751102 0 0) 0x7f5b34058d10 con 0x7f5b3408d8e0 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): my global_id is 54181 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b3407ee60 con 0x7f5b3408d8e0 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1027256228 0 0) 0x7f5b04003620 con 0x7f5b3408ce00 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b34058d10 con 0x7f5b3408ce00 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b28004540 con 0x7f5b3408ce00 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:21:53.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): dump: 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.801 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2165494851 0 0) 0x7f5b34058d10 con 0x7f5b3408ce00 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.102:6790/0 conn(0x7f5b3408b7f0 legacy=0x7f5b34151010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3408d8e0 legacy=0x7f5b34193380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: _finish_hunting 0 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 monclient: found mon.a 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b340811c0 con 0x7f5b3408ce00 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: _finish_auth 0 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: _check_auth_tickets 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:21:53.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 5 monclient: authenticate success, global_id 34368 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _renew_subs 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3407f9c0 con 0x7f5b3408ce00 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _renew_subs 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5b34084ff0 con 0x7f5b3408ce00 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5b28004810 con 0x7f5b3408ce00 2026-02-21T22:21:53.804 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b28004b70 con 0x7f5b3408ce00 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b1affd640 10 monclient: dump: 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.805 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b38b4b640 4 set_mon_vals no callback set 2026-02-21T22:21:53.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.769+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5b340008e0 con 0x7f5b3405c1f0 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5b28004e40 con 0x7f5b3408ce00 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(303..303 src has 112..303) ==== 3915+0+0 (unknown 3665463721 0 0) 0x7f5b2804c700 con 0x7f5b3408ce00 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b3ab4f640 10 monclient: get_auth_request con 0x7f5b0403d470 auth_method 0 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b3a34e640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c1f0 legacy=0x7f5b3404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 459196451 0 0) 0x7f5b340008e0 con 0x7f5b3405c1f0 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient(hunting): my global_id is 54184 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b34058d10 con 0x7f5b3405c1f0 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 927494993 0 0) 0x7f5b34058d10 con 0x7f5b3405c1f0 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b340008e0 con 0x7f5b3405c1f0 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b24004b50 con 0x7f5b3405c1f0 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient: dump: 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.808 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 890095525 0 0) 0x7f5b340008e0 con 0x7f5b3405c1f0 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b3404d990 con 0x7f5b3405c1f0 2026-02-21T22:21:53.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f5b24002f70 con 0x7f5b3405c1f0 2026-02-21T22:21:53.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:21:53.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:21:53.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c1f0 legacy=0x7f5b3404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.773+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c1f0 legacy=0x7f5b3404d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:53.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.777+0000 7f5b3c5d9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:21:53.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.777+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5b34090810 con 0x7f5b3405c5c0 2026-02-21T22:21:53.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3a34e640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c5c0 legacy=0x7f5b34195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:21:53.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1574276845 0 0) 0x7f5b34090810 con 0x7f5b3405c5c0 2026-02-21T22:21:53.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient(hunting): my global_id is 54187 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b340008e0 con 0x7f5b3405c5c0 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2987950001 0 0) 0x7f5b340008e0 con 0x7f5b3405c5c0 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b34090810 con 0x7f5b3405c5c0 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5b24005000 con 0x7f5b3405c5c0 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient: dump: 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:21:53.812 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1433006457 0 0) 0x7f5b34090810 con 0x7f5b3405c5c0 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b340909e0 con 0x7f5b3405c5c0 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4502 (unknown 0 0 2563652179) 0x7f5b24006450 con 0x7f5b3405c5c0 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c5c0 legacy=0x7f5b34195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b1affd640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6789/0 conn(0x7f5b3405c5c0 legacy=0x7f5b34195ab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 28, 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 17431, 2026-02-21T22:21:53.813 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.814 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:21:53.815 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:21:53.816 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:21:53.817 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:21:53.818 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.819 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:21:53.820 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:21:53.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5b0403d470 legacy=0x7f5b0405d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 10 monclient: shutdown 2026-02-21T22:21:53.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:21:53.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3ab4f640 1 -- 10.20.193.102:0/3227978436 reap_dead start 2026-02-21T22:21:53.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 >> v1:10.20.193.102:6789/0 conn(0x7f5b3408ce00 legacy=0x7f5b34084710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:21:53.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 shutdown_connections 2026-02-21T22:21:53.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 >> 10.20.193.102:0/3227978436 conn(0x7f5b34091a50 msgr2=0x7f5b3405dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:21:53.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 shutdown_connections 2026-02-21T22:21:53.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:21:53.781+0000 7f5b3c5d9640 1 -- 10.20.193.102:0/3227978436 wait complete. 2026-02-21T22:21:53.822 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:21:53.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:21:54.529 INFO:tasks.rados.rados.0.trial178.stdout:1437: done (0 left) 2026-02-21T22:21:54.530 INFO:tasks.rados.rados.0.trial178.stdout:1438: read oid 24 snap -1 2026-02-21T22:21:54.530 INFO:tasks.rados.rados.0.trial178.stdout:1438: expect deleted 2026-02-21T22:21:54.530 INFO:tasks.rados.rados.0.trial178.stdout:1439: write (excl) oid 12 current snap is 145 2026-02-21T22:21:54.530 INFO:tasks.rados.rados.0.trial178.stdout:1439: seq_num 512 ranges {605184=723139,1877957=727227} 2026-02-21T22:21:54.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:21:54.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:21:54.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:21:54.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:54.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:21:54.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:21:54.543 INFO:tasks.rados.rados.0.trial178.stdout:1439: writing trial17818531-12 from 605184 to 1328323 tid 1 2026-02-21T22:21:54.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:21:54.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:21:54.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:21:54.545 INFO:tasks.rados.rados.0.trial178.stdout:1439: writing trial17818531-12 from 1877957 to 2605184 tid 2 2026-02-21T22:21:54.545 INFO:tasks.rados.rados.0.trial178.stdout:1440: snap_remove snap 135 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1440: done (2 left) 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1441: copy_from oid 29 from oid 16 current snap is 145 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: finishing write tid 1 to trial17818531-12 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: finishing write tid 2 to trial17818531-12 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: finishing write tid 3 to trial17818531-12 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: finishing write tid 4 to trial17818531-12 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: oid 12 updating version 0 to 367 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: oid 12 version 367 is already newer than 365 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: oid 12 version 367 is already newer than 366 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 367 (ObjNum 512 snap 145 seq_num 512) dirty exists 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: left oid 12 (ObjNum 512 snap 145 seq_num 512) 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1438: done (2 left) 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1439: done (1 left) 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1442: write (excl) oid 2 current snap is 145 2026-02-21T22:21:55.533 INFO:tasks.rados.rados.0.trial178.stdout:1442: seq_num 513 ranges {621991=798388,1936439=639509} 2026-02-21T22:21:55.541 INFO:tasks.rados.rados.0.trial178.stdout:1442: writing trial17818531-2 from 621991 to 1420379 tid 1 2026-02-21T22:21:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1442: writing trial17818531-2 from 1936439 to 2575948 tid 2 2026-02-21T22:21:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1443: snap_create 2026-02-21T22:21:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1442: finishing write tid 1 to trial17818531-2 2026-02-21T22:21:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1441: finishing copy_from racing read to trial17818531-29 2026-02-21T22:21:55.550 INFO:tasks.rados.rados.0.trial178.stdout:1441: finishing copy_from to trial17818531-29 2026-02-21T22:21:55.550 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 358 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:21:55.551 INFO:tasks.rados.rados.0.trial178.stdout:1442: finishing write tid 2 to trial17818531-2 2026-02-21T22:21:55.551 INFO:tasks.rados.rados.0.trial178.stdout:1442: finishing write tid 3 to trial17818531-2 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:1442: finishing write tid 4 to trial17818531-2 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:1442: oid 2 updating version 0 to 502 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:1442: oid 2 version 502 is already newer than 500 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:1442: oid 2 version 502 is already newer than 501 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 502 (ObjNum 513 snap 145 seq_num 513) dirty exists 2026-02-21T22:21:55.552 INFO:tasks.rados.rados.0.trial178.stdout:1442: left oid 2 (ObjNum 513 snap 145 seq_num 513) 2026-02-21T22:21:56.159 INFO:tasks.rados.rados.0.trial178.stdout:1441: done (2 left) 2026-02-21T22:21:56.159 INFO:tasks.rados.rados.0.trial178.stdout:1442: done (1 left) 2026-02-21T22:21:56.159 INFO:tasks.rados.rados.0.trial178.stdout:1443: done (0 left) 2026-02-21T22:21:56.159 INFO:tasks.rados.rados.0.trial178.stdout:1444: read oid 49 snap 136 2026-02-21T22:21:56.160 INFO:tasks.rados.rados.0.trial178.stdout:1444: expect (ObjNum 486 snap 135 seq_num 486) 2026-02-21T22:21:56.160 INFO:tasks.rados.rados.0.trial178.stdout:1445: snap_remove snap 137 2026-02-21T22:21:57.159 INFO:tasks.rados.rados.0.trial178.stdout:1445: done (1 left) 2026-02-21T22:21:57.159 INFO:tasks.rados.rados.0.trial178.stdout:1446: snap_remove snap 144 2026-02-21T22:21:58.161 INFO:tasks.rados.rados.0.trial178.stdout:1446: done (1 left) 2026-02-21T22:21:58.161 INFO:tasks.rados.rados.0.trial178.stdout:1447: write oid 11 current snap is 146 2026-02-21T22:21:58.161 INFO:tasks.rados.rados.0.trial178.stdout:1447: seq_num 514 ranges {638798=473637,1594921=783167} 2026-02-21T22:21:58.168 INFO:tasks.rados.rados.0.trial178.stdout:1447: writing trial17818531-11 from 638798 to 1112435 tid 1 2026-02-21T22:21:58.177 INFO:tasks.rados.rados.0.trial178.stdout:1447: writing trial17818531-11 from 1594921 to 2378088 tid 2 2026-02-21T22:21:58.177 INFO:tasks.rados.rados.0.trial178.stdout:1448: write (excl) oid 14 current snap is 146 2026-02-21T22:21:58.177 INFO:tasks.rados.rados.0.trial178.stdout:1448: seq_num 515 ranges {655605=548886,1653403=643178} 2026-02-21T22:21:58.185 INFO:tasks.rados.rados.0.trial178.stdout:1448: writing trial17818531-14 from 655605 to 1204491 tid 1 2026-02-21T22:21:58.192 INFO:tasks.rados.rados.0.trial178.stdout:1448: writing trial17818531-14 from 1653403 to 2296581 tid 2 2026-02-21T22:21:58.192 INFO:tasks.rados.rados.0.trial178.stdout:1449: snap_create 2026-02-21T22:21:58.192 INFO:tasks.rados.rados.0.trial178.stdout:1444: expect (ObjNum 486 snap 135 seq_num 486) 2026-02-21T22:21:58.254 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: finishing write tid 1 to trial17818531-11 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: finishing write tid 2 to trial17818531-11 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: finishing write tid 3 to trial17818531-11 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: finishing write tid 4 to trial17818531-11 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: oid 11 updating version 0 to 659 2026-02-21T22:21:58.276 INFO:tasks.rados.rados.0.trial178.stdout:1447: oid 11 version 659 is already newer than 658 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1447: oid 11 updating version 659 to 660 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 660 (ObjNum 514 snap 146 seq_num 514) dirty exists 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1447: left oid 11 (ObjNum 514 snap 146 seq_num 514) 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: finishing write tid 1 to trial17818531-14 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: finishing write tid 2 to trial17818531-14 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: finishing write tid 3 to trial17818531-14 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: finishing write tid 4 to trial17818531-14 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: oid 14 updating version 0 to 479 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: oid 14 updating version 479 to 480 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: oid 14 updating version 480 to 481 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 481 (ObjNum 515 snap 146 seq_num 515) dirty exists 2026-02-21T22:21:58.277 INFO:tasks.rados.rados.0.trial178.stdout:1448: left oid 14 (ObjNum 515 snap 146 seq_num 515) 2026-02-21T22:21:58.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:21:58.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1444: done (3 left) 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1447: done (2 left) 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1448: done (1 left) 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1449: done (0 left) 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1450: read oid 19 snap 138 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1450: expect deleted 2026-02-21T22:21:59.167 INFO:tasks.rados.rados.0.trial178.stdout:1451: delete oid 3 current snap is 147 2026-02-21T22:21:59.171 INFO:tasks.rados.rados.0.trial178.stdout:1450: done (1 left) 2026-02-21T22:21:59.171 INFO:tasks.rados.rados.0.trial178.stdout:1451: done (0 left) 2026-02-21T22:21:59.171 INFO:tasks.rados.rados.0.trial178.stdout:1452: delete oid 38 current snap is 147 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1452: done (0 left) 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1453: rollback oid 41 current snap is 147 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 145 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1454: read oid 5 snap -1 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1454: expect (ObjNum 509 snap 144 seq_num 509) 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1455: read oid 6 snap -1 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1455: expect deleted 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1456: read oid 50 snap -1 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1456: expect (ObjNum 467 snap 133 seq_num 467) 2026-02-21T22:21:59.175 INFO:tasks.rados.rados.0.trial178.stdout:1457: snap_remove snap 134 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1457: done (4 left) 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1458: delete oid 39 current snap is 147 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1453: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1453: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1453: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 515 (ObjNum 481 snap 134 seq_num 481) dirty exists 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:1454: expect (ObjNum 509 snap 144 seq_num 509) 2026-02-21T22:22:00.171 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:00.230 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1277252 2026-02-21T22:22:00.282 INFO:tasks.rados.rados.0.trial178.stdout:1456: expect (ObjNum 467 snap 133 seq_num 467) 2026-02-21T22:22:00.299 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1453: done (4 left) 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1454: done (3 left) 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1455: done (2 left) 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1456: done (1 left) 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1458: done (0 left) 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1459: rollback oid 22 current snap is 147 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 136 2026-02-21T22:22:00.353 INFO:tasks.rados.rados.0.trial178.stdout:1460: snap_create 2026-02-21T22:22:00.356 INFO:tasks.rados.rados.0.trial178.stdout:1459: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:22:00.356 INFO:tasks.rados.rados.0.trial178.stdout:1459: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:22:00.356 INFO:tasks.rados.rados.0.trial178.stdout:1459: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:22:00.356 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 499 (ObjNum 394 snap 111 seq_num 394) dirty exists 2026-02-21T22:22:01.158 INFO:tasks.rados.rados.0.trial178.stdout:1459: done (1 left) 2026-02-21T22:22:01.158 INFO:tasks.rados.rados.0.trial178.stdout:1460: done (0 left) 2026-02-21T22:22:01.158 INFO:tasks.rados.rados.0.trial178.stdout:1461: snap_remove snap 136 2026-02-21T22:22:02.159 INFO:tasks.rados.rados.0.trial178.stdout:1461: done (0 left) 2026-02-21T22:22:02.160 INFO:tasks.rados.rados.0.trial178.stdout:1462: read oid 29 snap -1 2026-02-21T22:22:02.160 INFO:tasks.rados.rados.0.trial178.stdout:1462: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:22:02.160 INFO:tasks.rados.rados.0.trial178.stdout:1463: snap_remove snap 147 2026-02-21T22:22:03.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:03.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:03.719 INFO:tasks.rados.rados.0.trial178.stdout:1463: done (1 left) 2026-02-21T22:22:03.719 INFO:tasks.rados.rados.0.trial178.stdout:1464: delete oid 9 current snap is 148 2026-02-21T22:22:03.719 INFO:tasks.rados.rados.0.trial178.stdout:1462: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:22:03.845 INFO:tasks.rados.rados.0.trial178.stdout:1462: done (1 left) 2026-02-21T22:22:03.845 INFO:tasks.rados.rados.0.trial178.stdout:1464: done (0 left) 2026-02-21T22:22:03.845 INFO:tasks.rados.rados.0.trial178.stdout:1465: write (excl) oid 47 current snap is 148 2026-02-21T22:22:03.845 INFO:tasks.rados.rados.0.trial178.stdout:1465: seq_num 520 ranges {739640=641484,1829460=794174} 2026-02-21T22:22:03.854 INFO:tasks.rados.rados.0.trial178.stdout:1465: writing trial17818531-47 from 739640 to 1381124 tid 1 2026-02-21T22:22:03.863 INFO:tasks.rados.rados.0.trial178.stdout:1465: writing trial17818531-47 from 1829460 to 2623634 tid 2 2026-02-21T22:22:03.864 INFO:tasks.rados.rados.0.trial178.stdout:1466: delete oid 27 current snap is 148 2026-02-21T22:22:03.864 INFO:tasks.rados.rados.0.trial178.stdout:1465: finishing write tid 1 to trial17818531-47 2026-02-21T22:22:03.868 INFO:tasks.rados.rados.0.trial178.stdout:1465: finishing write tid 2 to trial17818531-47 2026-02-21T22:22:03.868 INFO:tasks.rados.rados.0.trial178.stdout:1465: finishing write tid 3 to trial17818531-47 2026-02-21T22:22:03.868 INFO:tasks.rados.rados.0.trial178.stdout:1465: finishing write tid 4 to trial17818531-47 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1465: oid 47 updating version 0 to 804 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1465: oid 47 updating version 804 to 806 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1465: oid 47 version 806 is already newer than 805 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 806 (ObjNum 520 snap 148 seq_num 520) dirty exists 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1465: left oid 47 (ObjNum 520 snap 148 seq_num 520) 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1465: done (1 left) 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1466: done (0 left) 2026-02-21T22:22:03.869 INFO:tasks.rados.rados.0.trial178.stdout:1467: snap_create 2026-02-21T22:22:04.165 INFO:tasks.rados.rados.0.trial178.stdout:1467: done (0 left) 2026-02-21T22:22:04.165 INFO:tasks.rados.rados.0.trial178.stdout:1468: snap_remove snap 140 2026-02-21T22:22:05.166 INFO:tasks.rados.rados.0.trial178.stdout:1468: done (0 left) 2026-02-21T22:22:05.166 INFO:tasks.rados.rados.0.trial178.stdout:1469: read oid 25 snap -1 2026-02-21T22:22:05.166 INFO:tasks.rados.rados.0.trial178.stdout:1469: expect (ObjNum 494 snap 137 seq_num 494) 2026-02-21T22:22:05.166 INFO:tasks.rados.rados.0.trial178.stdout:1470: write (excl) oid 47 current snap is 149 2026-02-21T22:22:05.166 INFO:tasks.rados.rados.0.trial178.stdout:1470: seq_num 522 ranges {773254=791982,2346424=426830} 2026-02-21T22:22:05.176 INFO:tasks.rados.rados.0.trial178.stdout:1470: writing trial17818531-47 from 773254 to 1565236 tid 1 2026-02-21T22:22:05.184 INFO:tasks.rados.rados.0.trial178.stdout:1470: writing trial17818531-47 from 2346424 to 2773254 tid 2 2026-02-21T22:22:05.185 INFO:tasks.rados.rados.0.trial178.stdout:1471: read oid 38 snap -1 2026-02-21T22:22:05.185 INFO:tasks.rados.rados.0.trial178.stdout:1471: expect deleted 2026-02-21T22:22:05.185 INFO:tasks.rados.rados.0.trial178.stdout:1472: delete oid 28 current snap is 149 2026-02-21T22:22:05.185 INFO:tasks.rados.rados.0.trial178.stdout:1469: expect (ObjNum 494 snap 137 seq_num 494) 2026-02-21T22:22:05.270 INFO:tasks.rados.rados.0.trial178.stdout:1470: finishing write tid 1 to trial17818531-47 2026-02-21T22:22:05.270 INFO:tasks.rados.rados.0.trial178.stdout:1470: finishing write tid 2 to trial17818531-47 2026-02-21T22:22:05.270 INFO:tasks.rados.rados.0.trial178.stdout:1470: finishing write tid 3 to trial17818531-47 2026-02-21T22:22:05.270 INFO:tasks.rados.rados.0.trial178.stdout:1470: finishing write tid 4 to trial17818531-47 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1470: oid 47 updating version 0 to 814 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1470: oid 47 version 814 is already newer than 813 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1470: oid 47 version 814 is already newer than 808 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 814 (ObjNum 522 snap 149 seq_num 522) dirty exists 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1470: left oid 47 (ObjNum 522 snap 149 seq_num 522) 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1469: done (3 left) 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1470: done (2 left) 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1471: done (1 left) 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1472: done (0 left) 2026-02-21T22:22:05.271 INFO:tasks.rados.rados.0.trial178.stdout:1473: delete oid 41 current snap is 149 2026-02-21T22:22:05.272 INFO:tasks.rados.rados.0.trial178.stdout:1473: done (0 left) 2026-02-21T22:22:05.272 INFO:tasks.rados.rados.0.trial178.stdout:1474: snap_create 2026-02-21T22:22:06.159 INFO:tasks.rados.rados.0.trial178.stdout:1474: done (0 left) 2026-02-21T22:22:06.159 INFO:tasks.rados.rados.0.trial178.stdout:1475: delete oid 31 current snap is 150 2026-02-21T22:22:06.160 INFO:tasks.rados.rados.0.trial178.stdout:1475: done (0 left) 2026-02-21T22:22:06.160 INFO:tasks.rados.rados.0.trial178.stdout:1476: read oid 41 snap -1 2026-02-21T22:22:06.160 INFO:tasks.rados.rados.0.trial178.stdout:1476: expect deleted 2026-02-21T22:22:06.161 INFO:tasks.rados.rados.0.trial178.stdout:1477: snap_remove snap 146 2026-02-21T22:22:07.160 INFO:tasks.rados.rados.0.trial178.stdout:1477: done (1 left) 2026-02-21T22:22:07.160 INFO:tasks.rados.rados.0.trial178.stdout:1478: rollback oid 43 current snap is 150 2026-02-21T22:22:07.160 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 149 2026-02-21T22:22:07.161 INFO:tasks.rados.rados.0.trial178.stdout:1476: done (1 left) 2026-02-21T22:22:07.161 INFO:tasks.rados.rados.0.trial178.stdout:1479: write oid 35 current snap is 150 2026-02-21T22:22:07.161 INFO:tasks.rados.rados.0.trial178.stdout:1479: seq_num 526 ranges {440482=409331,1380352=405181,2461553=378929} 2026-02-21T22:22:07.166 INFO:tasks.rados.rados.0.trial178.stdout:1479: writing trial17818531-35 from 440482 to 849813 tid 1 2026-02-21T22:22:07.172 INFO:tasks.rados.rados.0.trial178.stdout:1479: writing trial17818531-35 from 1380352 to 1785533 tid 2 2026-02-21T22:22:07.179 INFO:tasks.rados.rados.0.trial178.stdout:1479: writing trial17818531-35 from 2461553 to 2840482 tid 3 2026-02-21T22:22:07.179 INFO:tasks.rados.rados.0.trial178.stdout:1480: write oid 16 current snap is 150 2026-02-21T22:22:07.179 INFO:tasks.rados.rados.0.trial178.stdout:1480: seq_num 527 ranges {457289=484580,1438834=548839,2772623=84666} 2026-02-21T22:22:07.186 INFO:tasks.rados.rados.0.trial178.stdout:1480: writing trial17818531-16 from 457289 to 941869 tid 1 2026-02-21T22:22:07.193 INFO:tasks.rados.rados.0.trial178.stdout:1480: writing trial17818531-16 from 1438834 to 1987673 tid 2 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:1480: writing trial17818531-16 from 2772623 to 2857289 tid 3 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:1481: copy_from oid 36 from oid 29 current snap is 150 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:1478: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:1479: finishing write tid 1 to trial17818531-35 2026-02-21T22:22:07.200 INFO:tasks.rados.rados.0.trial178.stdout:1479: finishing write tid 2 to trial17818531-35 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: finishing write tid 3 to trial17818531-35 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: finishing write tid 4 to trial17818531-35 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: finishing write tid 5 to trial17818531-35 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: oid 35 updating version 0 to 675 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: oid 35 version 675 is already newer than 673 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: oid 35 version 675 is already newer than 670 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: oid 35 version 675 is already newer than 674 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 675 (ObjNum 526 snap 150 seq_num 526) dirty exists 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1479: left oid 35 (ObjNum 526 snap 150 seq_num 526) 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1480: finishing write tid 1 to trial17818531-16 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1480: finishing write tid 2 to trial17818531-16 2026-02-21T22:22:07.201 INFO:tasks.rados.rados.0.trial178.stdout:1478: done (3 left) 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1479: done (2 left) 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1482: read oid 10 snap 138 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1482: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1483: copy_from oid 2 from oid 21 current snap is 150 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1484: write (excl) oid 11 current snap is 150 2026-02-21T22:22:07.202 INFO:tasks.rados.rados.0.trial178.stdout:1484: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2026-02-21T22:22:07.206 INFO:tasks.rados.rados.0.trial178.stdout:1484: writing trial17818531-11 from 474096 to 1033925 tid 1 2026-02-21T22:22:07.214 INFO:tasks.rados.rados.0.trial178.stdout:1484: writing trial17818531-11 from 1780963 to 2189813 tid 2 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:1484: writing trial17818531-11 from 2800046 to 2874096 tid 3 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:1485: rollback oid 45 current snap is 150 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 143 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:1480: finishing write tid 3 to trial17818531-16 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:1486: write oid 27 current snap is 150 2026-02-21T22:22:07.218 INFO:tasks.rados.rados.0.trial178.stdout:1486: seq_num 529 ranges {490903=635078,1839445=552508} 2026-02-21T22:22:07.222 INFO:tasks.rados.rados.0.trial178.stdout:1486: writing trial17818531-27 from 490903 to 1125981 tid 1 2026-02-21T22:22:07.227 INFO:tasks.rados.rados.0.trial178.stdout:1486: writing trial17818531-27 from 1839445 to 2391953 tid 2 2026-02-21T22:22:07.227 INFO:tasks.rados.rados.0.trial178.stdout:1487: write oid 20 current snap is 150 2026-02-21T22:22:07.227 INFO:tasks.rados.rados.0.trial178.stdout:1487: seq_num 530 ranges {507710=710327,1897927=412519,2854892=52818} 2026-02-21T22:22:07.231 INFO:tasks.rados.rados.0.trial178.stdout:1487: writing trial17818531-20 from 507710 to 1218037 tid 1 2026-02-21T22:22:07.236 INFO:tasks.rados.rados.0.trial178.stdout:1487: writing trial17818531-20 from 1897927 to 2310446 tid 2 2026-02-21T22:22:07.238 INFO:tasks.rados.rados.0.trial178.stdout:1487: writing trial17818531-20 from 2854892 to 2907710 tid 3 2026-02-21T22:22:07.238 INFO:tasks.rados.rados.0.trial178.stdout:1488: snap_remove snap 148 2026-02-21T22:22:08.166 INFO:tasks.rados.rados.0.trial178.stdout:1488: done (8 left) 2026-02-21T22:22:08.166 INFO:tasks.rados.rados.0.trial178.stdout:1489: write oid 50 current snap is 150 2026-02-21T22:22:08.166 INFO:tasks.rados.rados.0.trial178.stdout:1489: seq_num 531 ranges {524517=785576,1956409=556177} 2026-02-21T22:22:08.172 INFO:tasks.rados.rados.0.trial178.stdout:1489: writing trial17818531-50 from 524517 to 1310093 tid 1 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1489: writing trial17818531-50 from 1956409 to 2512586 tid 2 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1490: copy_from oid 44 from oid 14 current snap is 150 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: finishing write tid 4 to trial17818531-16 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: finishing write tid 5 to trial17818531-16 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: oid 16 updating version 0 to 368 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: oid 16 updating version 368 to 370 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: oid 16 updating version 370 to 371 2026-02-21T22:22:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1480: oid 16 version 371 is already newer than 369 2026-02-21T22:22:08.179 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 371 (ObjNum 527 snap 150 seq_num 527) dirty exists 2026-02-21T22:22:08.179 INFO:tasks.rados.rados.0.trial178.stdout:1480: left oid 16 (ObjNum 527 snap 150 seq_num 527) 2026-02-21T22:22:08.179 INFO:tasks.rados.rados.0.trial178.stdout:1482: expect (ObjNum 324 snap 95 seq_num 324) 2026-02-21T22:22:08.227 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2571711 2026-02-21T22:22:08.282 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3368295 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:1484: finishing write tid 1 to trial17818531-11 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:1481: finishing copy_from racing read to trial17818531-36 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:1481: finishing copy_from to trial17818531-36 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 358 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:1484: finishing write tid 2 to trial17818531-11 2026-02-21T22:22:08.342 INFO:tasks.rados.rados.0.trial178.stdout:1483: finishing copy_from racing read to trial17818531-2 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1483: finishing copy_from to trial17818531-2 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 743 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1485: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 0 (ObjNum 33 snap 0 seq_num 99841168) dirty dne 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1484: finishing write tid 3 to trial17818531-11 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1484: finishing write tid 4 to trial17818531-11 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1480: done (9 left) 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1481: done (8 left) 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1482: done (7 left) 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1483: done (6 left) 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1485: done (5 left) 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1491: write (excl) oid 36 current snap is 150 2026-02-21T22:22:08.343 INFO:tasks.rados.rados.0.trial178.stdout:1491: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2026-02-21T22:22:08.346 INFO:tasks.rados.rados.0.trial178.stdout:1491: writing trial17818531-36 from 541324 to 1002149 tid 1 2026-02-21T22:22:08.352 INFO:tasks.rados.rados.0.trial178.stdout:1491: writing trial17818531-36 from 1498538 to 1914726 tid 2 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1491: writing trial17818531-36 from 2393385 to 2941324 tid 3 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1492: read oid 26 snap -1 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1492: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1493: read oid 48 snap -1 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1493: expect (ObjNum 505 snap 143 seq_num 505) 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1494: copy_from oid 31 from oid 13 current snap is 150 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1484: finishing write tid 5 to trial17818531-11 2026-02-21T22:22:08.361 INFO:tasks.rados.rados.0.trial178.stdout:1484: oid 11 updating version 0 to 678 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1484: oid 11 updating version 678 to 679 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1484: oid 11 updating version 679 to 680 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1484: oid 11 version 680 is already newer than 677 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 680 (ObjNum 528 snap 150 seq_num 528) dirty exists 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1484: left oid 11 (ObjNum 528 snap 150 seq_num 528) 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: finishing write tid 1 to trial17818531-27 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: finishing write tid 2 to trial17818531-27 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: finishing write tid 3 to trial17818531-27 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: finishing write tid 4 to trial17818531-27 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: oid 27 updating version 0 to 486 2026-02-21T22:22:08.362 INFO:tasks.rados.rados.0.trial178.stdout:1486: oid 27 version 486 is already newer than 485 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1486: oid 27 version 486 is already newer than 484 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 486 (ObjNum 529 snap 150 seq_num 529) dirty exists 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1486: left oid 27 (ObjNum 529 snap 150 seq_num 529) 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: finishing write tid 1 to trial17818531-20 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: finishing write tid 2 to trial17818531-20 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: finishing write tid 3 to trial17818531-20 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: finishing write tid 4 to trial17818531-20 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: finishing write tid 5 to trial17818531-20 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: oid 20 updating version 0 to 518 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: oid 20 version 518 is already newer than 517 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: oid 20 updating version 518 to 520 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:1487: oid 20 version 520 is already newer than 519 2026-02-21T22:22:08.363 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 520 (ObjNum 530 snap 150 seq_num 530) dirty exists 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1487: left oid 20 (ObjNum 530 snap 150 seq_num 530) 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: finishing write tid 1 to trial17818531-50 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: finishing write tid 2 to trial17818531-50 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: finishing write tid 3 to trial17818531-50 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1484: done (8 left) 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1486: done (7 left) 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1487: done (6 left) 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1495: delete oid 37 current snap is 150 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: finishing write tid 4 to trial17818531-50 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: oid 50 updating version 0 to 500 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: oid 50 updating version 500 to 501 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:1489: oid 50 updating version 501 to 502 2026-02-21T22:22:08.364 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 502 (ObjNum 531 snap 150 seq_num 531) dirty exists 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:1489: left oid 50 (ObjNum 531 snap 150 seq_num 531) 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:1490: finishing copy_from racing read to trial17818531-44 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:1490: finishing copy_from to trial17818531-44 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 481 (ObjNum 515 snap 146 seq_num 515) dirty exists 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:1491: finishing write tid 1 to trial17818531-36 2026-02-21T22:22:08.365 INFO:tasks.rados.rados.0.trial178.stdout:1491: finishing write tid 2 to trial17818531-36 2026-02-21T22:22:08.366 INFO:tasks.rados.rados.0.trial178.stdout:1493: expect (ObjNum 505 snap 143 seq_num 505) 2026-02-21T22:22:08.366 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:08.438 INFO:tasks.rados.rados.0.trial178.stdout:1492: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:22:08.448 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1500039 2026-02-21T22:22:08.480 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 803163 2026-02-21T22:22:08.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:08.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: finishing write tid 3 to trial17818531-36 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: finishing write tid 4 to trial17818531-36 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: finishing write tid 5 to trial17818531-36 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: oid 36 updating version 0 to 360 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: oid 36 updating version 360 to 362 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: oid 36 version 362 is already newer than 361 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:1491: oid 36 version 362 is already newer than 359 2026-02-21T22:22:08.559 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 362 (ObjNum 532 snap 150 seq_num 532) dirty exists 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1491: left oid 36 (ObjNum 532 snap 150 seq_num 532) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1494: finishing copy_from racing read to trial17818531-31 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1494: finishing copy_from to trial17818531-31 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 610 (ObjNum 454 snap 132 seq_num 454) dirty exists 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1489: done (6 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1490: done (5 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1491: done (4 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1492: done (3 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1493: done (2 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1494: done (1 left) 2026-02-21T22:22:08.560 INFO:tasks.rados.rados.0.trial178.stdout:1495: done (0 left) 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:1496: rollback oid 31 current snap is 150 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 143 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:1497: read oid 27 snap -1 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:1497: expect (ObjNum 529 snap 150 seq_num 529) 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:1498: rollback oid 7 current snap is 150 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 7 to 141 2026-02-21T22:22:08.561 INFO:tasks.rados.rados.0.trial178.stdout:1499: delete oid 5 current snap is 150 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:1496: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:1496: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 612 (ObjNum 33 snap 0 seq_num 4122737488) dirty dne 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:1496: done (3 left) 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:1499: done (2 left) 2026-02-21T22:22:08.562 INFO:tasks.rados.rados.0.trial178.stdout:1500: snap_create 2026-02-21T22:22:08.563 INFO:tasks.rados.rados.0.trial178.stdout:1497: expect (ObjNum 529 snap 150 seq_num 529) 2026-02-21T22:22:08.646 INFO:tasks.rados.rados.0.trial178.stdout:1498: finishing rollback tid 1 to trial17818531-7 2026-02-21T22:22:08.647 INFO:tasks.rados.rados.0.trial178.stdout:1498: finishing rollback tid 2 to trial17818531-7 2026-02-21T22:22:08.647 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 820 (ObjNum 457 snap 132 seq_num 457) dirty exists 2026-02-21T22:22:09.170 INFO:tasks.rados.rados.0.trial178.stdout:1497: done (2 left) 2026-02-21T22:22:09.170 INFO:tasks.rados.rados.0.trial178.stdout:1498: done (1 left) 2026-02-21T22:22:09.170 INFO:tasks.rados.rados.0.trial178.stdout:1500: done (0 left) 2026-02-21T22:22:09.170 INFO:tasks.rados.rados.0.trial178.stdout:1501: delete oid 42 current snap is 151 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1501: done (0 left) 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1502: read oid 47 snap 145 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1502: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1503: read oid 15 snap 143 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1503: expect (ObjNum 491 snap 136 seq_num 491) 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1504: read oid 4 snap -1 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1504: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1505: copy_from oid 17 from oid 31 current snap is 151 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1506: rollback oid 6 current snap is 151 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 143 2026-02-21T22:22:09.173 INFO:tasks.rados.rados.0.trial178.stdout:1507: delete oid 9 current snap is 151 2026-02-21T22:22:09.176 INFO:tasks.rados.rados.0.trial178.stdout:1506: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:22:09.176 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-21T22:22:09.177 INFO:tasks.rados.rados.0.trial178.stdout:1504: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:09.177 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:09.225 INFO:tasks.rados.rados.0.trial178.stdout:1503: expect (ObjNum 491 snap 136 seq_num 491) 2026-02-21T22:22:09.313 INFO:tasks.rados.rados.0.trial178.stdout:1505: finishing copy_from to trial17818531-17 2026-02-21T22:22:09.313 INFO:tasks.rados.rados.0.trial178.stdout:1505: got expected ENOENT (src dne) 2026-02-21T22:22:09.313 INFO:tasks.rados.rados.0.trial178.stdout:1505: finishing copy_from racing read to trial17818531-17 2026-02-21T22:22:09.313 INFO:tasks.rados.rados.0.trial178.stdout:1502: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:09.313 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:09.354 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1320101 2026-02-21T22:22:09.362 INFO:tasks.rados.rados.0.trial178.stdout:1502: done (5 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1503: done (4 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1504: done (3 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1505: done (2 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1506: done (1 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1507: done (0 left) 2026-02-21T22:22:09.363 INFO:tasks.rados.rados.0.trial178.stdout:1508: delete oid 17 current snap is 151 2026-02-21T22:22:09.364 INFO:tasks.rados.rados.0.trial178.stdout:1508: done (0 left) 2026-02-21T22:22:09.365 INFO:tasks.rados.rados.0.trial178.stdout:1509: read oid 1 snap -1 2026-02-21T22:22:09.365 INFO:tasks.rados.rados.0.trial178.stdout:1509: expect (ObjNum 331 snap 95 seq_num 331) 2026-02-21T22:22:09.365 INFO:tasks.rados.rados.0.trial178.stdout:1510: write oid 48 current snap is 151 2026-02-21T22:22:09.365 INFO:tasks.rados.rados.0.trial178.stdout:1510: seq_num 538 ranges {642166=628672,1849430=710842} 2026-02-21T22:22:09.374 INFO:tasks.rados.rados.0.trial178.stdout:1510: writing trial17818531-48 from 642166 to 1270838 tid 1 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:1510: writing trial17818531-48 from 1849430 to 2560272 tid 2 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:1511: read oid 3 snap -1 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:1511: expect deleted 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:1512: rollback oid 11 current snap is 151 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 129 2026-02-21T22:22:09.383 INFO:tasks.rados.rados.0.trial178.stdout:1509: expect (ObjNum 331 snap 95 seq_num 331) 2026-02-21T22:22:09.441 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:09.565 INFO:tasks.rados.rados.0.trial178.stdout:1510: finishing write tid 1 to trial17818531-48 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1512: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1512: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: finishing write tid 2 to trial17818531-48 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: finishing write tid 3 to trial17818531-48 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1512: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 688 (ObjNum 396 snap 111 seq_num 396) dirty exists 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: finishing write tid 4 to trial17818531-48 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: oid 48 updating version 0 to 511 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: oid 48 version 511 is already newer than 509 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: oid 48 version 511 is already newer than 510 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 511 (ObjNum 538 snap 151 seq_num 538) dirty exists 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: left oid 48 (ObjNum 538 snap 151 seq_num 538) 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1509: done (3 left) 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1510: done (2 left) 2026-02-21T22:22:09.566 INFO:tasks.rados.rados.0.trial178.stdout:1511: done (1 left) 2026-02-21T22:22:09.567 INFO:tasks.rados.rados.0.trial178.stdout:1512: done (0 left) 2026-02-21T22:22:09.567 INFO:tasks.rados.rados.0.trial178.stdout:1513: snap_remove snap 129 2026-02-21T22:22:10.171 INFO:tasks.rados.rados.0.trial178.stdout:1513: done (0 left) 2026-02-21T22:22:10.172 INFO:tasks.rados.rados.0.trial178.stdout:1514: write oid 7 current snap is 151 2026-02-21T22:22:10.172 INFO:tasks.rados.rados.0.trial178.stdout:1514: seq_num 539 ranges {658973=703921,1907912=570853,2985346=73627} 2026-02-21T22:22:10.180 INFO:tasks.rados.rados.0.trial178.stdout:1514: writing trial17818531-7 from 658973 to 1362894 tid 1 2026-02-21T22:22:10.188 INFO:tasks.rados.rados.0.trial178.stdout:1514: writing trial17818531-7 from 1907912 to 2478765 tid 2 2026-02-21T22:22:10.192 INFO:tasks.rados.rados.0.trial178.stdout:1514: writing trial17818531-7 from 2985346 to 3058973 tid 3 2026-02-21T22:22:10.192 INFO:tasks.rados.rados.0.trial178.stdout:1515: read oid 18 snap -1 2026-02-21T22:22:10.192 INFO:tasks.rados.rados.0.trial178.stdout:1515: expect deleted 2026-02-21T22:22:10.192 INFO:tasks.rados.rados.0.trial178.stdout:1516: read oid 5 snap -1 2026-02-21T22:22:10.193 INFO:tasks.rados.rados.0.trial178.stdout:1516: expect deleted 2026-02-21T22:22:10.193 INFO:tasks.rados.rados.0.trial178.stdout:1517: snap_create 2026-02-21T22:22:10.193 INFO:tasks.rados.rados.0.trial178.stdout:1514: finishing write tid 1 to trial17818531-7 2026-02-21T22:22:10.193 INFO:tasks.rados.rados.0.trial178.stdout:1514: finishing write tid 2 to trial17818531-7 2026-02-21T22:22:10.194 INFO:tasks.rados.rados.0.trial178.stdout:1514: finishing write tid 3 to trial17818531-7 2026-02-21T22:22:10.194 INFO:tasks.rados.rados.0.trial178.stdout:1514: finishing write tid 4 to trial17818531-7 2026-02-21T22:22:10.194 INFO:tasks.rados.rados.0.trial178.stdout:1514: finishing write tid 5 to trial17818531-7 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:1514: oid 7 updating version 0 to 833 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:1514: oid 7 version 833 is already newer than 832 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:1514: oid 7 updating version 833 to 834 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:1514: oid 7 updating version 834 to 835 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 835 (ObjNum 539 snap 151 seq_num 539) dirty exists 2026-02-21T22:22:10.195 INFO:tasks.rados.rados.0.trial178.stdout:1514: left oid 7 (ObjNum 539 snap 151 seq_num 539) 2026-02-21T22:22:11.160 INFO:tasks.rados.rados.0.trial178.stdout:1514: done (3 left) 2026-02-21T22:22:11.161 INFO:tasks.rados.rados.0.trial178.stdout:1515: done (2 left) 2026-02-21T22:22:11.161 INFO:tasks.rados.rados.0.trial178.stdout:1516: done (1 left) 2026-02-21T22:22:11.161 INFO:tasks.rados.rados.0.trial178.stdout:1517: done (0 left) 2026-02-21T22:22:11.161 INFO:tasks.rados.rados.0.trial178.stdout:1518: write (excl) oid 34 current snap is 152 2026-02-21T22:22:11.161 INFO:tasks.rados.rados.0.trial178.stdout:1518: seq_num 540 ranges {675780=495523,1682747=714511,3012769=63011} 2026-02-21T22:22:11.168 INFO:tasks.rados.rados.0.trial178.stdout:1518: writing trial17818531-34 from 675780 to 1171303 tid 1 2026-02-21T22:22:11.176 INFO:tasks.rados.rados.0.trial178.stdout:1518: writing trial17818531-34 from 1682747 to 2397258 tid 2 2026-02-21T22:22:11.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: writing trial17818531-34 from 3012769 to 3075780 tid 3 2026-02-21T22:22:11.181 INFO:tasks.rados.rados.0.trial178.stdout:1519: snap_remove snap 151 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1519: done (1 left) 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1520: read oid 43 snap 143 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1520: expect deleted 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1521: read oid 38 snap -1 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1521: expect deleted 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1522: read oid 8 snap -1 2026-02-21T22:22:12.180 INFO:tasks.rados.rados.0.trial178.stdout:1522: expect (ObjNum 497 snap 139 seq_num 497) 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1523: write oid 40 current snap is 152 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: finishing write tid 1 to trial17818531-34 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: finishing write tid 2 to trial17818531-34 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: finishing write tid 3 to trial17818531-34 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: finishing write tid 4 to trial17818531-34 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: finishing write tid 5 to trial17818531-34 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: oid 34 updating version 0 to 391 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: oid 34 updating version 391 to 392 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: oid 34 updating version 392 to 393 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: oid 34 version 393 is already newer than 390 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 393 (ObjNum 540 snap 152 seq_num 540) dirty exists 2026-02-21T22:22:12.181 INFO:tasks.rados.rados.0.trial178.stdout:1518: left oid 34 (ObjNum 540 snap 152 seq_num 540) 2026-02-21T22:22:12.182 INFO:tasks.rados.rados.0.trial178.stdout:1523: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2026-02-21T22:22:12.188 INFO:tasks.rados.rados.0.trial178.stdout:1523: writing trial17818531-40 from 692587 to 1263359 tid 1 2026-02-21T22:22:12.197 INFO:tasks.rados.rados.0.trial178.stdout:1523: writing trial17818531-40 from 2024876 to 2599398 tid 2 2026-02-21T22:22:12.200 INFO:tasks.rados.rados.0.trial178.stdout:1523: writing trial17818531-40 from 3040192 to 3092587 tid 3 2026-02-21T22:22:12.201 INFO:tasks.rados.rados.0.trial178.stdout:1518: done (4 left) 2026-02-21T22:22:12.201 INFO:tasks.rados.rados.0.trial178.stdout:1524: write oid 1 current snap is 152 2026-02-21T22:22:12.201 INFO:tasks.rados.rados.0.trial178.stdout:1524: seq_num 542 ranges {709394=646021,2083358=718180} 2026-02-21T22:22:12.209 INFO:tasks.rados.rados.0.trial178.stdout:1524: writing trial17818531-1 from 709394 to 1355415 tid 1 2026-02-21T22:22:12.219 INFO:tasks.rados.rados.0.trial178.stdout:1524: writing trial17818531-1 from 2083358 to 2801538 tid 2 2026-02-21T22:22:12.220 INFO:tasks.rados.rados.0.trial178.stdout:1525: read oid 9 snap -1 2026-02-21T22:22:12.220 INFO:tasks.rados.rados.0.trial178.stdout:1525: expect deleted 2026-02-21T22:22:12.220 INFO:tasks.rados.rados.0.trial178.stdout:1526: write (excl) oid 44 current snap is 152 2026-02-21T22:22:12.220 INFO:tasks.rados.rados.0.trial178.stdout:1526: seq_num 543 ranges {726201=721270,2141840=578191} 2026-02-21T22:22:12.227 INFO:tasks.rados.rados.0.trial178.stdout:1526: writing trial17818531-44 from 726201 to 1447471 tid 1 2026-02-21T22:22:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1526: writing trial17818531-44 from 2141840 to 2720031 tid 2 2026-02-21T22:22:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1527: read oid 34 snap -1 2026-02-21T22:22:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1527: expect (ObjNum 540 snap 152 seq_num 540) 2026-02-21T22:22:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1528: write oid 16 current snap is 152 2026-02-21T22:22:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1528: seq_num 544 ranges {743008=796519,2200322=721849} 2026-02-21T22:22:12.238 INFO:tasks.rados.rados.0.trial178.stdout:1528: writing trial17818531-16 from 743008 to 1539527 tid 1 2026-02-21T22:22:12.244 INFO:tasks.rados.rados.0.trial178.stdout:1528: writing trial17818531-16 from 2200322 to 2922171 tid 2 2026-02-21T22:22:12.244 INFO:tasks.rados.rados.0.trial178.stdout:1529: snap_create 2026-02-21T22:22:12.244 INFO:tasks.rados.rados.0.trial178.stdout:1522: expect (ObjNum 497 snap 139 seq_num 497) 2026-02-21T22:22:12.244 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:12.269 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 330145 2026-02-21T22:22:12.318 INFO:tasks.rados.rados.0.trial178.stdout:1523: finishing write tid 1 to trial17818531-40 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: finishing write tid 2 to trial17818531-40 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: finishing write tid 3 to trial17818531-40 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: finishing write tid 4 to trial17818531-40 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: finishing write tid 5 to trial17818531-40 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: oid 40 updating version 0 to 504 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: oid 40 updating version 504 to 505 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: oid 40 version 505 is already newer than 503 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: oid 40 updating version 505 to 506 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 506 (ObjNum 541 snap 152 seq_num 541) dirty exists 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1523: left oid 40 (ObjNum 541 snap 152 seq_num 541) 2026-02-21T22:22:12.319 INFO:tasks.rados.rados.0.trial178.stdout:1524: finishing write tid 1 to trial17818531-1 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: finishing write tid 2 to trial17818531-1 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: finishing write tid 3 to trial17818531-1 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: finishing write tid 4 to trial17818531-1 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: oid 1 updating version 0 to 513 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: oid 1 version 513 is already newer than 512 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: oid 1 updating version 513 to 514 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 514 (ObjNum 542 snap 152 seq_num 542) dirty exists 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1524: left oid 1 (ObjNum 542 snap 152 seq_num 542) 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: finishing write tid 1 to trial17818531-44 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: finishing write tid 2 to trial17818531-44 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: finishing write tid 3 to trial17818531-44 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: finishing write tid 4 to trial17818531-44 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: oid 44 updating version 0 to 482 2026-02-21T22:22:12.320 INFO:tasks.rados.rados.0.trial178.stdout:1526: oid 44 updating version 482 to 484 2026-02-21T22:22:12.321 INFO:tasks.rados.rados.0.trial178.stdout:1526: oid 44 version 484 is already newer than 483 2026-02-21T22:22:12.321 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 484 (ObjNum 543 snap 152 seq_num 543) dirty exists 2026-02-21T22:22:12.321 INFO:tasks.rados.rados.0.trial178.stdout:1526: left oid 44 (ObjNum 543 snap 152 seq_num 543) 2026-02-21T22:22:12.321 INFO:tasks.rados.rados.0.trial178.stdout:1528: finishing write tid 1 to trial17818531-16 2026-02-21T22:22:12.321 INFO:tasks.rados.rados.0.trial178.stdout:1527: expect (ObjNum 540 snap 152 seq_num 540) 2026-02-21T22:22:12.371 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:12.423 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:12.476 INFO:tasks.rados.rados.0.trial178.stdout:1528: finishing write tid 2 to trial17818531-16 2026-02-21T22:22:12.477 INFO:tasks.rados.rados.0.trial178.stdout:1528: finishing write tid 3 to trial17818531-16 2026-02-21T22:22:12.477 INFO:tasks.rados.rados.0.trial178.stdout:1528: finishing write tid 4 to trial17818531-16 2026-02-21T22:22:12.477 INFO:tasks.rados.rados.0.trial178.stdout:1528: oid 16 updating version 0 to 485 2026-02-21T22:22:12.477 INFO:tasks.rados.rados.0.trial178.stdout:1528: oid 16 updating version 485 to 486 2026-02-21T22:22:12.478 INFO:tasks.rados.rados.0.trial178.stdout:1528: oid 16 updating version 486 to 487 2026-02-21T22:22:12.478 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 487 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:12.478 INFO:tasks.rados.rados.0.trial178.stdout:1528: left oid 16 (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:13.187 INFO:tasks.rados.rados.0.trial178.stdout:1520: done (9 left) 2026-02-21T22:22:13.187 INFO:tasks.rados.rados.0.trial178.stdout:1521: done (8 left) 2026-02-21T22:22:13.187 INFO:tasks.rados.rados.0.trial178.stdout:1522: done (7 left) 2026-02-21T22:22:13.187 INFO:tasks.rados.rados.0.trial178.stdout:1523: done (6 left) 2026-02-21T22:22:13.187 INFO:tasks.rados.rados.0.trial178.stdout:1524: done (5 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1525: done (4 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1526: done (3 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1527: done (2 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1528: done (1 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1529: done (0 left) 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1530: rollback oid 19 current snap is 153 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 138 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1531: write oid 1 current snap is 153 2026-02-21T22:22:13.188 INFO:tasks.rados.rados.0.trial178.stdout:1531: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2026-02-21T22:22:13.192 INFO:tasks.rados.rados.0.trial178.stdout:1531: writing trial17818531-1 from 759815 to 1231583 tid 1 2026-02-21T22:22:13.196 INFO:tasks.rados.rados.0.trial178.stdout:1531: writing trial17818531-1 from 1742451 to 2207958 tid 2 2026-02-21T22:22:13.200 INFO:tasks.rados.rados.0.trial178.stdout:1531: writing trial17818531-1 from 2917178 to 3159815 tid 3 2026-02-21T22:22:13.200 INFO:tasks.rados.rados.0.trial178.stdout:1532: read oid 20 snap -1 2026-02-21T22:22:13.200 INFO:tasks.rados.rados.0.trial178.stdout:1532: expect (ObjNum 530 snap 150 seq_num 530) 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1533: copy_from oid 23 from oid 17 current snap is 153 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1530: finishing rollback tid 0 to trial17818531-19 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1530: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 838 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1531: finishing write tid 1 to trial17818531-1 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1531: finishing write tid 2 to trial17818531-1 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1530: done (3 left) 2026-02-21T22:22:13.201 INFO:tasks.rados.rados.0.trial178.stdout:1534: snap_create 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: finishing write tid 3 to trial17818531-1 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: finishing write tid 4 to trial17818531-1 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: finishing write tid 5 to trial17818531-1 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: oid 1 updating version 0 to 516 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: oid 1 updating version 516 to 518 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: oid 1 version 518 is already newer than 515 2026-02-21T22:22:13.202 INFO:tasks.rados.rados.0.trial178.stdout:1531: oid 1 version 518 is already newer than 517 2026-02-21T22:22:13.203 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 518 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:22:13.203 INFO:tasks.rados.rados.0.trial178.stdout:1531: left oid 1 (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:22:13.208 INFO:tasks.rados.rados.0.trial178.stdout:1533: finishing copy_from to trial17818531-23 2026-02-21T22:22:13.208 INFO:tasks.rados.rados.0.trial178.stdout:1533: got expected ENOENT (src dne) 2026-02-21T22:22:13.208 INFO:tasks.rados.rados.0.trial178.stdout:1533: finishing copy_from racing read to trial17818531-23 2026-02-21T22:22:13.208 INFO:tasks.rados.rados.0.trial178.stdout:1532: expect (ObjNum 530 snap 150 seq_num 530) 2026-02-21T22:22:13.306 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:13.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:13.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:14.190 INFO:tasks.rados.rados.0.trial178.stdout:1531: done (3 left) 2026-02-21T22:22:14.190 INFO:tasks.rados.rados.0.trial178.stdout:1532: done (2 left) 2026-02-21T22:22:14.190 INFO:tasks.rados.rados.0.trial178.stdout:1533: done (1 left) 2026-02-21T22:22:14.190 INFO:tasks.rados.rados.0.trial178.stdout:1534: done (0 left) 2026-02-21T22:22:14.190 INFO:tasks.rados.rados.0.trial178.stdout:1535: snap_remove snap 143 2026-02-21T22:22:15.192 INFO:tasks.rados.rados.0.trial178.stdout:1535: done (0 left) 2026-02-21T22:22:15.192 INFO:tasks.rados.rados.0.trial178.stdout:1536: delete oid 13 current snap is 154 2026-02-21T22:22:15.196 INFO:tasks.rados.rados.0.trial178.stdout:1536: done (0 left) 2026-02-21T22:22:15.196 INFO:tasks.rados.rados.0.trial178.stdout:1537: write oid 16 current snap is 154 2026-02-21T22:22:15.196 INFO:tasks.rados.rados.0.trial178.stdout:1537: seq_num 547 ranges {793429=622266,1859415=469176,2972024=221405} 2026-02-21T22:22:15.206 INFO:tasks.rados.rados.0.trial178.stdout:1537: writing trial17818531-16 from 793429 to 1415695 tid 1 2026-02-21T22:22:15.212 INFO:tasks.rados.rados.0.trial178.stdout:1537: writing trial17818531-16 from 1859415 to 2328591 tid 2 2026-02-21T22:22:15.218 INFO:tasks.rados.rados.0.trial178.stdout:1537: writing trial17818531-16 from 2972024 to 3193429 tid 3 2026-02-21T22:22:15.218 INFO:tasks.rados.rados.0.trial178.stdout:1538: read oid 20 snap -1 2026-02-21T22:22:15.218 INFO:tasks.rados.rados.0.trial178.stdout:1538: expect (ObjNum 530 snap 150 seq_num 530) 2026-02-21T22:22:15.218 INFO:tasks.rados.rados.0.trial178.stdout:1539: write oid 25 current snap is 154 2026-02-21T22:22:15.218 INFO:tasks.rados.rados.0.trial178.stdout:1539: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2026-02-21T22:22:15.223 INFO:tasks.rados.rados.0.trial178.stdout:1539: writing trial17818531-25 from 410236 to 824104 tid 1 2026-02-21T22:22:15.231 INFO:tasks.rados.rados.0.trial178.stdout:1539: writing trial17818531-25 from 1234250 to 1963437 tid 2 2026-02-21T22:22:15.239 INFO:tasks.rados.rados.0.trial178.stdout:1539: writing trial17818531-25 from 2715800 to 3138863 tid 3 2026-02-21T22:22:15.239 INFO:tasks.rados.rados.0.trial178.stdout:1540: snap_remove snap 149 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1540: done (3 left) 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1541: read oid 4 snap -1 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1541: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1542: snap_create 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: finishing write tid 1 to trial17818531-16 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: finishing write tid 2 to trial17818531-16 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: finishing write tid 3 to trial17818531-16 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: finishing write tid 4 to trial17818531-16 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: finishing write tid 5 to trial17818531-16 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: oid 16 updating version 0 to 491 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: oid 16 version 491 is already newer than 489 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: oid 16 version 491 is already newer than 488 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: oid 16 version 491 is already newer than 490 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 491 (ObjNum 547 snap 154 seq_num 547) dirty exists 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1537: left oid 16 (ObjNum 547 snap 154 seq_num 547) 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:1538: expect (ObjNum 530 snap 150 seq_num 530) 2026-02-21T22:22:16.161 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:22:16.166 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:16.167 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:16.168 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.169 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:16.170 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:16.245 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2353152 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: finishing write tid 1 to trial17818531-25 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: finishing write tid 2 to trial17818531-25 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: finishing write tid 3 to trial17818531-25 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: finishing write tid 4 to trial17818531-25 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: finishing write tid 5 to trial17818531-25 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: oid 25 updating version 0 to 613 2026-02-21T22:22:16.292 INFO:tasks.rados.rados.0.trial178.stdout:1539: oid 25 updating version 613 to 614 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:1539: oid 25 updating version 614 to 616 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:1539: oid 25 version 616 is already newer than 615 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 616 (ObjNum 548 snap 154 seq_num 548) dirty exists 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:1539: left oid 25 (ObjNum 548 snap 154 seq_num 548) 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:1541: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:16.293 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1537: done (4 left) 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1538: done (3 left) 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1539: done (2 left) 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1541: done (1 left) 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1542: done (0 left) 2026-02-21T22:22:17.162 INFO:tasks.rados.rados.0.trial178.stdout:1543: snap_remove snap 138 2026-02-21T22:22:18.165 INFO:tasks.rados.rados.0.trial178.stdout:1543: done (0 left) 2026-02-21T22:22:18.165 INFO:tasks.rados.rados.0.trial178.stdout:1544: snap_remove snap 150 2026-02-21T22:22:18.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:18.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:19.168 INFO:tasks.rados.rados.0.trial178.stdout:1544: done (0 left) 2026-02-21T22:22:19.168 INFO:tasks.rados.rados.0.trial178.stdout:1545: read oid 13 snap -1 2026-02-21T22:22:19.169 INFO:tasks.rados.rados.0.trial178.stdout:1545: expect deleted 2026-02-21T22:22:19.169 INFO:tasks.rados.rados.0.trial178.stdout:1546: delete oid 48 current snap is 155 2026-02-21T22:22:19.171 INFO:tasks.rados.rados.0.trial178.stdout:1545: done (1 left) 2026-02-21T22:22:19.171 INFO:tasks.rados.rados.0.trial178.stdout:1546: done (0 left) 2026-02-21T22:22:19.172 INFO:tasks.rados.rados.0.trial178.stdout:1547: read oid 38 snap -1 2026-02-21T22:22:19.172 INFO:tasks.rados.rados.0.trial178.stdout:1547: expect deleted 2026-02-21T22:22:19.172 INFO:tasks.rados.rados.0.trial178.stdout:1548: copy_from oid 6 from oid 48 current snap is 155 2026-02-21T22:22:19.172 INFO:tasks.rados.rados.0.trial178.stdout:1549: write oid 12 current snap is 155 2026-02-21T22:22:19.172 INFO:tasks.rados.rados.0.trial178.stdout:1549: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2026-02-21T22:22:19.178 INFO:tasks.rados.rados.0.trial178.stdout:1549: writing trial17818531-12 from 443850 to 1008216 tid 1 2026-02-21T22:22:19.187 INFO:tasks.rados.rados.0.trial178.stdout:1549: writing trial17818531-12 from 1634861 to 2367717 tid 2 2026-02-21T22:22:19.193 INFO:tasks.rados.rados.0.trial178.stdout:1549: writing trial17818531-12 from 2770646 to 3243850 tid 3 2026-02-21T22:22:19.193 INFO:tasks.rados.rados.0.trial178.stdout:1550: read oid 1 snap -1 2026-02-21T22:22:19.193 INFO:tasks.rados.rados.0.trial178.stdout:1550: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:22:19.193 INFO:tasks.rados.rados.0.trial178.stdout:1551: snap_remove snap 141 2026-02-21T22:22:20.374 INFO:tasks.rados.rados.0.trial178.stdout:1551: done (4 left) 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1552: read oid 23 snap 145 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1552: expect (ObjNum 495 snap 137 seq_num 495) 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1553: rollback oid 16 current snap is 155 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 153 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1548: finishing copy_from to trial17818531-6 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1548: got expected ENOENT (src dne) 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1548: finishing copy_from racing read to trial17818531-6 2026-02-21T22:22:20.375 INFO:tasks.rados.rados.0.trial178.stdout:1549: finishing write tid 1 to trial17818531-12 2026-02-21T22:22:20.376 INFO:tasks.rados.rados.0.trial178.stdout:1549: finishing write tid 2 to trial17818531-12 2026-02-21T22:22:20.376 INFO:tasks.rados.rados.0.trial178.stdout:1549: finishing write tid 3 to trial17818531-12 2026-02-21T22:22:20.376 INFO:tasks.rados.rados.0.trial178.stdout:1550: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:22:20.397 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1093776 2026-02-21T22:22:20.523 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: finishing write tid 4 to trial17818531-12 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: finishing write tid 5 to trial17818531-12 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: oid 12 updating version 0 to 494 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: oid 12 updating version 494 to 495 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: oid 12 version 495 is already newer than 492 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:1549: oid 12 version 495 is already newer than 493 2026-02-21T22:22:20.592 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 495 (ObjNum 550 snap 155 seq_num 550) dirty exists 2026-02-21T22:22:20.593 INFO:tasks.rados.rados.0.trial178.stdout:1549: left oid 12 (ObjNum 550 snap 155 seq_num 550) 2026-02-21T22:22:20.593 INFO:tasks.rados.rados.0.trial178.stdout:1552: expect (ObjNum 495 snap 137 seq_num 495) 2026-02-21T22:22:20.626 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 783224 2026-02-21T22:22:20.661 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1576799 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:1553: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:1553: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:1553: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 498 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:1547: done (5 left) 2026-02-21T22:22:20.670 INFO:tasks.rados.rados.0.trial178.stdout:1548: done (4 left) 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1549: done (3 left) 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1550: done (2 left) 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1552: done (1 left) 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1553: done (0 left) 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1554: write oid 38 current snap is 155 2026-02-21T22:22:20.671 INFO:tasks.rados.rados.0.trial178.stdout:1554: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2026-02-21T22:22:20.678 INFO:tasks.rados.rados.0.trial178.stdout:1554: writing trial17818531-38 from 460657 to 1100272 tid 1 2026-02-21T22:22:20.685 INFO:tasks.rados.rados.0.trial178.stdout:1554: writing trial17818531-38 from 1693343 to 2169857 tid 2 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1554: writing trial17818531-38 from 2681716 to 3260657 tid 3 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1555: read oid 37 snap -1 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1555: expect deleted 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1556: snap_create 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1554: finishing write tid 1 to trial17818531-38 2026-02-21T22:22:20.693 INFO:tasks.rados.rados.0.trial178.stdout:1554: finishing write tid 2 to trial17818531-38 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: finishing write tid 3 to trial17818531-38 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: finishing write tid 4 to trial17818531-38 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: finishing write tid 5 to trial17818531-38 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: oid 38 updating version 0 to 392 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: oid 38 updating version 392 to 394 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: oid 38 version 394 is already newer than 393 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: oid 38 version 394 is already newer than 391 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 394 (ObjNum 551 snap 155 seq_num 551) dirty exists 2026-02-21T22:22:20.695 INFO:tasks.rados.rados.0.trial178.stdout:1554: left oid 38 (ObjNum 551 snap 155 seq_num 551) 2026-02-21T22:22:21.162 INFO:tasks.rados.rados.0.trial178.stdout:1554: done (2 left) 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1555: done (1 left) 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1556: done (0 left) 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1557: copy_from oid 49 from oid 13 current snap is 156 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1558: rollback oid 9 current snap is 156 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 155 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1559: read oid 15 snap -1 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1559: expect (ObjNum 507 snap 144 seq_num 507) 2026-02-21T22:22:21.163 INFO:tasks.rados.rados.0.trial178.stdout:1560: read oid 7 snap 154 2026-02-21T22:22:21.164 INFO:tasks.rados.rados.0.trial178.stdout:1560: expect (ObjNum 539 snap 151 seq_num 539) 2026-02-21T22:22:21.164 INFO:tasks.rados.rados.0.trial178.stdout:1561: delete oid 34 current snap is 156 2026-02-21T22:22:21.170 INFO:tasks.rados.rados.0.trial178.stdout:1557: finishing copy_from to trial17818531-49 2026-02-21T22:22:21.170 INFO:tasks.rados.rados.0.trial178.stdout:1557: got expected ENOENT (src dne) 2026-02-21T22:22:21.170 INFO:tasks.rados.rados.0.trial178.stdout:1557: finishing copy_from racing read to trial17818531-49 2026-02-21T22:22:21.171 INFO:tasks.rados.rados.0.trial178.stdout:1559: expect (ObjNum 507 snap 144 seq_num 507) 2026-02-21T22:22:21.171 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:21.225 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:21.282 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2403785 2026-02-21T22:22:21.286 INFO:tasks.rados.rados.0.trial178.stdout:1558: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:22:21.286 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 0 (ObjNum 129 snap 0 seq_num 101000528) dirty dne 2026-02-21T22:22:21.286 INFO:tasks.rados.rados.0.trial178.stdout:1560: expect (ObjNum 539 snap 151 seq_num 539) 2026-02-21T22:22:21.328 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 390656 2026-02-21T22:22:21.369 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1557: done (4 left) 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1558: done (3 left) 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1559: done (2 left) 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1560: done (1 left) 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1561: done (0 left) 2026-02-21T22:22:21.412 INFO:tasks.rados.rados.0.trial178.stdout:1562: read oid 7 snap 152 2026-02-21T22:22:21.413 INFO:tasks.rados.rados.0.trial178.stdout:1562: expect (ObjNum 539 snap 151 seq_num 539) 2026-02-21T22:22:21.413 INFO:tasks.rados.rados.0.trial178.stdout:1563: write (excl) oid 46 current snap is 156 2026-02-21T22:22:21.413 INFO:tasks.rados.rados.0.trial178.stdout:1563: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2026-02-21T22:22:21.421 INFO:tasks.rados.rados.0.trial178.stdout:1563: writing trial17818531-46 from 494271 to 1284384 tid 1 2026-02-21T22:22:21.427 INFO:tasks.rados.rados.0.trial178.stdout:1563: writing trial17818531-46 from 1693954 to 2174137 tid 2 2026-02-21T22:22:21.433 INFO:tasks.rados.rados.0.trial178.stdout:1563: writing trial17818531-46 from 2620209 to 3132338 tid 3 2026-02-21T22:22:21.434 INFO:tasks.rados.rados.0.trial178.stdout:1564: snap_remove snap 155 2026-02-21T22:22:22.378 INFO:tasks.rados.rados.0.trial178.stdout:1564: done (2 left) 2026-02-21T22:22:22.378 INFO:tasks.rados.rados.0.trial178.stdout:1565: copy_from oid 34 from oid 8 current snap is 156 2026-02-21T22:22:22.378 INFO:tasks.rados.rados.0.trial178.stdout:1562: expect (ObjNum 539 snap 151 seq_num 539) 2026-02-21T22:22:22.427 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2803571 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: finishing write tid 1 to trial17818531-46 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: finishing write tid 2 to trial17818531-46 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: finishing write tid 3 to trial17818531-46 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: finishing write tid 4 to trial17818531-46 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: finishing write tid 5 to trial17818531-46 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: oid 46 updating version 0 to 440 2026-02-21T22:22:22.515 INFO:tasks.rados.rados.0.trial178.stdout:1563: oid 46 version 440 is already newer than 438 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1563: oid 46 version 440 is already newer than 439 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1563: oid 46 version 440 is already newer than 437 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 440 (ObjNum 553 snap 156 seq_num 553) dirty exists 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1563: left oid 46 (ObjNum 553 snap 156 seq_num 553) 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1565: finishing copy_from racing read to trial17818531-34 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1565: finishing copy_from to trial17818531-34 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 739 (ObjNum 497 snap 139 seq_num 497) dirty exists 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1562: done (2 left) 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1563: done (1 left) 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1565: done (0 left) 2026-02-21T22:22:22.516 INFO:tasks.rados.rados.0.trial178.stdout:1566: snap_create 2026-02-21T22:22:23.451 INFO:tasks.rados.rados.0.trial178.stdout:1566: done (0 left) 2026-02-21T22:22:23.452 INFO:tasks.rados.rados.0.trial178.stdout:1567: snap_create 2026-02-21T22:22:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:22:23] 2026-02-21T22:22:23.526 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:22:23, mode upmap, max misplaced 0.050000 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'rbd', 'unique_pool_0'] 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:22:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:22:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:22:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:22:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:22:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:22:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:22:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:22:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:22:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:22:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:22:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:22:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:22:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:22:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:22:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:22:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "333", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:22:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:22:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:22:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 333, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 162, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:22:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:22:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:22:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:22:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:22:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:22:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0003994199964735243 of space, bias 1.0, pg target 0.11982599894205728 quantized to 32 (current 16) 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:22:24.455 INFO:tasks.rados.rados.0.trial178.stdout:1567: done (0 left) 2026-02-21T22:22:24.455 INFO:tasks.rados.rados.0.trial178.stdout:1568: write oid 7 current snap is 158 2026-02-21T22:22:24.455 INFO:tasks.rados.rados.0.trial178.stdout:1568: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2026-02-21T22:22:24.461 INFO:tasks.rados.rados.0.trial178.stdout:1568: writing trial17818531-7 from 511078 to 976440 tid 1 2026-02-21T22:22:24.471 INFO:tasks.rados.rados.0.trial178.stdout:1568: writing trial17818531-7 from 1752436 to 2492630 tid 2 2026-02-21T22:22:24.477 INFO:tasks.rados.rados.0.trial178.stdout:1568: writing trial17818531-7 from 3163985 to 3311078 tid 3 2026-02-21T22:22:24.477 INFO:tasks.rados.rados.0.trial178.stdout:1569: write (excl) oid 41 current snap is 158 2026-02-21T22:22:24.477 INFO:tasks.rados.rados.0.trial178.stdout:1568: finishing write tid 1 to trial17818531-7 2026-02-21T22:22:24.478 INFO:tasks.rados.rados.0.trial178.stdout:1568: finishing write tid 2 to trial17818531-7 2026-02-21T22:22:24.478 INFO:tasks.rados.rados.0.trial178.stdout:1569: seq_num 555 ranges {527885=656964,1927271=483852,3191408=136477} 2026-02-21T22:22:24.484 INFO:tasks.rados.rados.0.trial178.stdout:1569: writing trial17818531-41 from 527885 to 1184849 tid 1 2026-02-21T22:22:24.492 INFO:tasks.rados.rados.0.trial178.stdout:1569: writing trial17818531-41 from 1927271 to 2411123 tid 2 2026-02-21T22:22:24.499 INFO:tasks.rados.rados.0.trial178.stdout:1569: writing trial17818531-41 from 3191408 to 3327885 tid 3 2026-02-21T22:22:24.499 INFO:tasks.rados.rados.0.trial178.stdout:1570: read oid 26 snap -1 2026-02-21T22:22:24.499 INFO:tasks.rados.rados.0.trial178.stdout:1570: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:22:24.499 INFO:tasks.rados.rados.0.trial178.stdout:1571: copy_from oid 8 from oid 36 current snap is 158 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: finishing write tid 3 to trial17818531-7 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: finishing write tid 4 to trial17818531-7 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: finishing write tid 5 to trial17818531-7 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: oid 7 updating version 0 to 877 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: oid 7 version 877 is already newer than 876 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: oid 7 version 877 is already newer than 874 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: oid 7 version 877 is already newer than 875 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 877 (ObjNum 554 snap 158 seq_num 554) dirty exists 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1568: left oid 7 (ObjNum 554 snap 158 seq_num 554) 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1569: finishing write tid 1 to trial17818531-41 2026-02-21T22:22:24.500 INFO:tasks.rados.rados.0.trial178.stdout:1569: finishing write tid 2 to trial17818531-41 2026-02-21T22:22:24.501 INFO:tasks.rados.rados.0.trial178.stdout:1568: done (3 left) 2026-02-21T22:22:24.501 INFO:tasks.rados.rados.0.trial178.stdout:1572: write oid 38 current snap is 158 2026-02-21T22:22:24.501 INFO:tasks.rados.rados.0.trial178.stdout:1572: seq_num 556 ranges {544692=732213,1985753=743863,3335184=9508} 2026-02-21T22:22:24.507 INFO:tasks.rados.rados.0.trial178.stdout:1572: writing trial17818531-38 from 544692 to 1276905 tid 1 2026-02-21T22:22:24.514 INFO:tasks.rados.rados.0.trial178.stdout:1572: writing trial17818531-38 from 1985753 to 2729616 tid 2 2026-02-21T22:22:24.516 INFO:tasks.rados.rados.0.trial178.stdout:1572: writing trial17818531-38 from 3335184 to 3344692 tid 3 2026-02-21T22:22:24.516 INFO:tasks.rados.rados.0.trial178.stdout:1573: snap_remove snap 157 2026-02-21T22:22:25.457 INFO:tasks.rados.rados.0.trial178.stdout:1573: done (4 left) 2026-02-21T22:22:25.457 INFO:tasks.rados.rados.0.trial178.stdout:1574: snap_remove snap 153 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1574: done (4 left) 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1575: read oid 31 snap -1 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1575: expect deleted 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1576: copy_from oid 39 from oid 44 current snap is 158 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: finishing write tid 3 to trial17818531-41 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: finishing write tid 4 to trial17818531-41 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: finishing write tid 5 to trial17818531-41 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: oid 41 updating version 0 to 565 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: oid 41 version 565 is already newer than 563 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: oid 41 version 565 is already newer than 564 2026-02-21T22:22:26.163 INFO:tasks.rados.rados.0.trial178.stdout:1569: oid 41 version 565 is already newer than 562 2026-02-21T22:22:26.164 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 565 (ObjNum 555 snap 158 seq_num 555) dirty exists 2026-02-21T22:22:26.164 INFO:tasks.rados.rados.0.trial178.stdout:1569: left oid 41 (ObjNum 555 snap 158 seq_num 555) 2026-02-21T22:22:26.164 INFO:tasks.rados.rados.0.trial178.stdout:1570: expect (ObjNum 338 snap 95 seq_num 338) 2026-02-21T22:22:26.164 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:26.241 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1742922 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1571: finishing copy_from racing read to trial17818531-8 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1571: finishing copy_from to trial17818531-8 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 362 (ObjNum 532 snap 150 seq_num 532) dirty exists 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: finishing write tid 1 to trial17818531-38 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: finishing write tid 2 to trial17818531-38 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: finishing write tid 3 to trial17818531-38 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: finishing write tid 4 to trial17818531-38 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: finishing write tid 5 to trial17818531-38 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: oid 38 updating version 0 to 395 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: oid 38 updating version 395 to 398 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: oid 38 version 398 is already newer than 397 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: oid 38 version 398 is already newer than 396 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 398 (ObjNum 556 snap 158 seq_num 556) dirty exists 2026-02-21T22:22:26.305 INFO:tasks.rados.rados.0.trial178.stdout:1572: left oid 38 (ObjNum 556 snap 158 seq_num 556) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1576: finishing copy_from racing read to trial17818531-39 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1576: finishing copy_from to trial17818531-39 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 484 (ObjNum 543 snap 152 seq_num 543) dirty exists 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1569: done (5 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1570: done (4 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1571: done (3 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1572: done (2 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1575: done (1 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1576: done (0 left) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1577: read oid 25 snap -1 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1577: expect (ObjNum 548 snap 154 seq_num 548) 2026-02-21T22:22:26.306 INFO:tasks.rados.rados.0.trial178.stdout:1578: rollback oid 38 current snap is 158 2026-02-21T22:22:26.307 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 145 2026-02-21T22:22:26.307 INFO:tasks.rados.rados.0.trial178.stdout:1579: write oid 19 current snap is 158 2026-02-21T22:22:26.307 INFO:tasks.rados.rados.0.trial178.stdout:1579: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2026-02-21T22:22:26.311 INFO:tasks.rados.rados.0.trial178.stdout:1579: writing trial17818531-19 from 561499 to 968961 tid 1 2026-02-21T22:22:26.319 INFO:tasks.rados.rados.0.trial178.stdout:1579: writing trial17818531-19 from 1527882 to 2015403 tid 2 2026-02-21T22:22:26.328 INFO:tasks.rados.rados.0.trial178.stdout:1579: writing trial17818531-19 from 2729901 to 3287118 tid 3 2026-02-21T22:22:26.328 INFO:tasks.rados.rados.0.trial178.stdout:1580: copy_from oid 7 from oid 18 current snap is 158 2026-02-21T22:22:26.328 INFO:tasks.rados.rados.0.trial178.stdout:1578: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:22:26.328 INFO:tasks.rados.rados.0.trial178.stdout:1578: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:22:26.329 INFO:tasks.rados.rados.0.trial178.stdout:1578: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:22:26.329 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 401 (ObjNum 502 snap 142 seq_num 502) dirty exists 2026-02-21T22:22:26.329 INFO:tasks.rados.rados.0.trial178.stdout:1577: expect (ObjNum 548 snap 154 seq_num 548) 2026-02-21T22:22:26.329 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:26.369 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1371979 2026-02-21T22:22:26.424 INFO:tasks.rados.rados.0.trial178.stdout:1579: finishing write tid 1 to trial17818531-19 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: finishing write tid 2 to trial17818531-19 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: finishing write tid 3 to trial17818531-19 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: finishing write tid 4 to trial17818531-19 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: finishing write tid 5 to trial17818531-19 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: oid 19 updating version 0 to 889 2026-02-21T22:22:26.425 INFO:tasks.rados.rados.0.trial178.stdout:1579: oid 19 version 889 is already newer than 888 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1579: oid 19 updating version 889 to 891 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1579: oid 19 version 891 is already newer than 890 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 891 (ObjNum 557 snap 158 seq_num 557) dirty exists 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1579: left oid 19 (ObjNum 557 snap 158 seq_num 557) 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1580: finishing copy_from to trial17818531-7 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1580: got expected ENOENT (src dne) 2026-02-21T22:22:26.426 INFO:tasks.rados.rados.0.trial178.stdout:1580: finishing copy_from racing read to trial17818531-7 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1577: done (3 left) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1578: done (2 left) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1579: done (1 left) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1580: done (0 left) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1581: read oid 36 snap 154 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1581: expect (ObjNum 532 snap 150 seq_num 532) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1582: read oid 38 snap -1 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1582: expect (ObjNum 502 snap 142 seq_num 502) 2026-02-21T22:22:26.427 INFO:tasks.rados.rados.0.trial178.stdout:1583: write oid 33 current snap is 158 2026-02-21T22:22:26.428 INFO:tasks.rados.rados.0.trial178.stdout:1583: seq_num 558 ranges {578306=482711,1586364=631179,2757324=484842} 2026-02-21T22:22:26.431 INFO:tasks.rados.rados.0.trial178.stdout:1583: writing trial17818531-33 from 578306 to 1061017 tid 1 2026-02-21T22:22:26.439 INFO:tasks.rados.rados.0.trial178.stdout:1583: writing trial17818531-33 from 1586364 to 2217543 tid 2 2026-02-21T22:22:26.446 INFO:tasks.rados.rados.0.trial178.stdout:1583: writing trial17818531-33 from 2757324 to 3242166 tid 3 2026-02-21T22:22:26.446 INFO:tasks.rados.rados.0.trial178.stdout:1584: read oid 23 snap -1 2026-02-21T22:22:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1584: expect (ObjNum 495 snap 137 seq_num 495) 2026-02-21T22:22:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1585: rollback oid 28 current snap is 158 2026-02-21T22:22:26.447 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 145 2026-02-21T22:22:26.447 INFO:tasks.rados.rados.0.trial178.stdout:1582: expect (ObjNum 502 snap 142 seq_num 502) 2026-02-21T22:22:26.542 INFO:tasks.rados.rados.0.trial178.stdout:1581: expect (ObjNum 532 snap 150 seq_num 532) 2026-02-21T22:22:26.574 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:26.648 INFO:tasks.rados.rados.0.trial178.stdout:1583: finishing write tid 1 to trial17818531-33 2026-02-21T22:22:26.649 INFO:tasks.rados.rados.0.trial178.stdout:1583: finishing write tid 2 to trial17818531-33 2026-02-21T22:22:26.649 INFO:tasks.rados.rados.0.trial178.stdout:1581: done (4 left) 2026-02-21T22:22:26.649 INFO:tasks.rados.rados.0.trial178.stdout:1582: done (3 left) 2026-02-21T22:22:26.649 INFO:tasks.rados.rados.0.trial178.stdout:1586: write (excl) oid 24 current snap is 158 2026-02-21T22:22:26.649 INFO:tasks.rados.rados.0.trial178.stdout:1586: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2026-02-21T22:22:26.656 INFO:tasks.rados.rados.0.trial178.stdout:1586: writing trial17818531-24 from 595113 to 1153073 tid 1 2026-02-21T22:22:26.663 INFO:tasks.rados.rados.0.trial178.stdout:1586: writing trial17818531-24 from 1644846 to 2136036 tid 2 2026-02-21T22:22:26.672 INFO:tasks.rados.rados.0.trial178.stdout:1586: writing trial17818531-24 from 2784747 to 3395113 tid 3 2026-02-21T22:22:26.672 INFO:tasks.rados.rados.0.trial178.stdout:1587: read oid 46 snap -1 2026-02-21T22:22:26.672 INFO:tasks.rados.rados.0.trial178.stdout:1587: expect (ObjNum 553 snap 156 seq_num 553) 2026-02-21T22:22:26.672 INFO:tasks.rados.rados.0.trial178.stdout:1588: copy_from oid 20 from oid 35 current snap is 158 2026-02-21T22:22:26.672 INFO:tasks.rados.rados.0.trial178.stdout:1584: expect (ObjNum 495 snap 137 seq_num 495) 2026-02-21T22:22:26.703 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 581416 2026-02-21T22:22:26.748 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1867746 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: finishing write tid 3 to trial17818531-33 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: finishing write tid 4 to trial17818531-33 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: finishing write tid 5 to trial17818531-33 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: oid 33 updating version 0 to 404 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: oid 33 updating version 404 to 405 2026-02-21T22:22:26.751 INFO:tasks.rados.rados.0.trial178.stdout:1583: oid 33 version 405 is already newer than 403 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1583: oid 33 version 405 is already newer than 402 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 405 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1583: left oid 33 (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1585: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1585: finishing rollback tid 2 to trial17818531-28 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 407 (ObjNum 430 snap 123 seq_num 430) dirty exists 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: finishing write tid 1 to trial17818531-24 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: finishing write tid 2 to trial17818531-24 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: finishing write tid 3 to trial17818531-24 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: finishing write tid 4 to trial17818531-24 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: finishing write tid 5 to trial17818531-24 2026-02-21T22:22:26.752 INFO:tasks.rados.rados.0.trial178.stdout:1586: oid 24 updating version 0 to 895 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:1586: oid 24 version 895 is already newer than 892 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:1586: oid 24 version 895 is already newer than 893 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:1586: oid 24 version 895 is already newer than 894 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 895 (ObjNum 559 snap 158 seq_num 559) dirty exists 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:1586: left oid 24 (ObjNum 559 snap 158 seq_num 559) 2026-02-21T22:22:26.753 INFO:tasks.rados.rados.0.trial178.stdout:1587: expect (ObjNum 553 snap 156 seq_num 553) 2026-02-21T22:22:26.773 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3229878 2026-02-21T22:22:26.810 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:26.846 INFO:tasks.rados.rados.0.trial178.stdout:1588: finishing copy_from racing read to trial17818531-20 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1588: finishing copy_from to trial17818531-20 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 675 (ObjNum 526 snap 150 seq_num 526) dirty exists 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1583: done (5 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1584: done (4 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1585: done (3 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1586: done (2 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1587: done (1 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1588: done (0 left) 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1589: read oid 21 snap 152 2026-02-21T22:22:26.847 INFO:tasks.rados.rados.0.trial178.stdout:1589: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:22:26.848 INFO:tasks.rados.rados.0.trial178.stdout:1590: snap_remove snap 156 2026-02-21T22:22:27.465 INFO:tasks.rados.rados.0.trial178.stdout:1590: done (1 left) 2026-02-21T22:22:27.465 INFO:tasks.rados.rados.0.trial178.stdout:1591: rollback oid 29 current snap is 158 2026-02-21T22:22:27.465 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 152 2026-02-21T22:22:27.465 INFO:tasks.rados.rados.0.trial178.stdout:1589: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1591: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1591: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1591: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 742 (ObjNum 469 snap 133 seq_num 469) dirty exists 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1589: done (1 left) 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1591: done (0 left) 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1592: rollback oid 41 current snap is 158 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 154 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1593: copy_from oid 13 from oid 1 current snap is 158 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1594: copy_from oid 39 from oid 21 current snap is 158 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1595: read oid 15 snap -1 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1595: expect (ObjNum 507 snap 144 seq_num 507) 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1596: write oid 25 current snap is 158 2026-02-21T22:22:27.596 INFO:tasks.rados.rados.0.trial178.stdout:1596: seq_num 560 ranges {611920=633209,1703328=634848,2812170=507386} 2026-02-21T22:22:27.605 INFO:tasks.rados.rados.0.trial178.stdout:1596: writing trial17818531-25 from 611920 to 1245129 tid 1 2026-02-21T22:22:27.612 INFO:tasks.rados.rados.0.trial178.stdout:1596: writing trial17818531-25 from 1703328 to 2338176 tid 2 2026-02-21T22:22:27.619 INFO:tasks.rados.rados.0.trial178.stdout:1596: writing trial17818531-25 from 2812170 to 3319556 tid 3 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:1597: rollback oid 32 current snap is 158 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 154 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:1592: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:1592: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 677 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:22:27.620 INFO:tasks.rados.rados.0.trial178.stdout:1595: expect (ObjNum 507 snap 144 seq_num 507) 2026-02-21T22:22:27.666 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1449201 2026-02-21T22:22:27.682 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 120028 2026-02-21T22:22:27.716 INFO:tasks.rados.rados.0.trial178.stdout:1594: finishing copy_from racing read to trial17818531-39 2026-02-21T22:22:27.716 INFO:tasks.rados.rados.0.trial178.stdout:1593: finishing copy_from racing read to trial17818531-13 2026-02-21T22:22:27.716 INFO:tasks.rados.rados.0.trial178.stdout:1593: finishing copy_from to trial17818531-13 2026-02-21T22:22:27.716 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 518 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:22:27.716 INFO:tasks.rados.rados.0.trial178.stdout:1596: finishing write tid 1 to trial17818531-25 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1594: finishing copy_from to trial17818531-39 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 743 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: finishing write tid 2 to trial17818531-25 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: finishing write tid 3 to trial17818531-25 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1597: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: finishing write tid 4 to trial17818531-25 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: finishing write tid 5 to trial17818531-25 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: oid 25 updating version 0 to 619 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: oid 25 version 619 is already newer than 618 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: oid 25 updating version 619 to 620 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: oid 25 version 620 is already newer than 617 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 620 (ObjNum 560 snap 158 seq_num 560) dirty exists 2026-02-21T22:22:27.717 INFO:tasks.rados.rados.0.trial178.stdout:1596: left oid 25 (ObjNum 560 snap 158 seq_num 560) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1597: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1597: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 733 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1592: done (5 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1593: done (4 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1594: done (3 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1595: done (2 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1596: done (1 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1597: done (0 left) 2026-02-21T22:22:27.718 INFO:tasks.rados.rados.0.trial178.stdout:1598: delete oid 20 current snap is 158 2026-02-21T22:22:27.719 INFO:tasks.rados.rados.0.trial178.stdout:1598: done (0 left) 2026-02-21T22:22:27.719 INFO:tasks.rados.rados.0.trial178.stdout:1599: read oid 37 snap -1 2026-02-21T22:22:27.719 INFO:tasks.rados.rados.0.trial178.stdout:1599: expect deleted 2026-02-21T22:22:27.720 INFO:tasks.rados.rados.0.trial178.stdout:1600: read oid 47 snap -1 2026-02-21T22:22:27.720 INFO:tasks.rados.rados.0.trial178.stdout:1600: expect (ObjNum 522 snap 149 seq_num 522) 2026-02-21T22:22:27.720 INFO:tasks.rados.rados.0.trial178.stdout:1601: write oid 41 current snap is 158 2026-02-21T22:22:27.720 INFO:tasks.rados.rados.0.trial178.stdout:1601: seq_num 562 ranges {645534=783707,2103939=638517,3150663=294871} 2026-02-21T22:22:27.726 INFO:tasks.rados.rados.0.trial178.stdout:1601: writing trial17818531-41 from 645534 to 1429241 tid 1 2026-02-21T22:22:27.736 INFO:tasks.rados.rados.0.trial178.stdout:1601: writing trial17818531-41 from 2103939 to 2742456 tid 2 2026-02-21T22:22:27.741 INFO:tasks.rados.rados.0.trial178.stdout:1601: writing trial17818531-41 from 3150663 to 3445534 tid 3 2026-02-21T22:22:27.741 INFO:tasks.rados.rados.0.trial178.stdout:1602: write (excl) oid 35 current snap is 158 2026-02-21T22:22:27.741 INFO:tasks.rados.rados.0.trial178.stdout:1602: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2026-02-21T22:22:27.749 INFO:tasks.rados.rados.0.trial178.stdout:1602: writing trial17818531-35 from 662341 to 1237650 tid 1 2026-02-21T22:22:27.757 INFO:tasks.rados.rados.0.trial178.stdout:1602: writing trial17818531-35 from 1878774 to 2377302 tid 2 2026-02-21T22:22:27.762 INFO:tasks.rados.rados.0.trial178.stdout:1602: writing trial17818531-35 from 2894439 to 3351994 tid 3 2026-02-21T22:22:27.762 INFO:tasks.rados.rados.0.trial178.stdout:1603: write oid 26 current snap is 158 2026-02-21T22:22:27.763 INFO:tasks.rados.rados.0.trial178.stdout:1603: seq_num 564 ranges {679148=650558,1937256=642186,3321862=157286} 2026-02-21T22:22:27.767 INFO:tasks.rados.rados.0.trial178.stdout:1603: writing trial17818531-26 from 679148 to 1329706 tid 1 2026-02-21T22:22:27.772 INFO:tasks.rados.rados.0.trial178.stdout:1603: writing trial17818531-26 from 1937256 to 2579442 tid 2 2026-02-21T22:22:27.776 INFO:tasks.rados.rados.0.trial178.stdout:1603: writing trial17818531-26 from 3321862 to 3479148 tid 3 2026-02-21T22:22:27.776 INFO:tasks.rados.rados.0.trial178.stdout:1604: delete oid 2 current snap is 158 2026-02-21T22:22:27.776 INFO:tasks.rados.rados.0.trial178.stdout:1600: expect (ObjNum 522 snap 149 seq_num 522) 2026-02-21T22:22:27.776 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 69202 2026-02-21T22:22:27.880 INFO:tasks.rados.rados.0.trial178.stdout:1601: finishing write tid 1 to trial17818531-41 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1601: finishing write tid 2 to trial17818531-41 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1601: finishing write tid 3 to trial17818531-41 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1601: finishing write tid 4 to trial17818531-41 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1599: done (5 left) 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1600: done (4 left) 2026-02-21T22:22:27.881 INFO:tasks.rados.rados.0.trial178.stdout:1604: done (3 left) 2026-02-21T22:22:27.882 INFO:tasks.rados.rados.0.trial178.stdout:1605: delete oid 7 current snap is 158 2026-02-21T22:22:27.882 INFO:tasks.rados.rados.0.trial178.stdout:1601: finishing write tid 5 to trial17818531-41 2026-02-21T22:22:27.882 INFO:tasks.rados.rados.0.trial178.stdout:1601: oid 41 updating version 0 to 679 2026-02-21T22:22:27.882 INFO:tasks.rados.rados.0.trial178.stdout:1601: oid 41 updating version 679 to 680 2026-02-21T22:22:27.882 INFO:tasks.rados.rados.0.trial178.stdout:1601: oid 41 updating version 680 to 681 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1601: oid 41 updating version 681 to 682 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 682 (ObjNum 562 snap 158 seq_num 562) dirty exists 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1601: left oid 41 (ObjNum 562 snap 158 seq_num 562) 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1602: finishing write tid 1 to trial17818531-35 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1602: finishing write tid 2 to trial17818531-35 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1602: finishing write tid 3 to trial17818531-35 2026-02-21T22:22:27.883 INFO:tasks.rados.rados.0.trial178.stdout:1602: finishing write tid 4 to trial17818531-35 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:1602: finishing write tid 5 to trial17818531-35 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:1602: oid 35 updating version 0 to 738 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:1602: oid 35 version 738 is already newer than 735 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:1602: oid 35 version 738 is already newer than 737 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:1602: oid 35 version 738 is already newer than 736 2026-02-21T22:22:27.884 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 738 (ObjNum 563 snap 158 seq_num 563) dirty exists 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1602: left oid 35 (ObjNum 563 snap 158 seq_num 563) 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1603: finishing write tid 1 to trial17818531-26 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1603: finishing write tid 2 to trial17818531-26 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1603: finishing write tid 3 to trial17818531-26 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1603: finishing write tid 4 to trial17818531-26 2026-02-21T22:22:27.885 INFO:tasks.rados.rados.0.trial178.stdout:1603: finishing write tid 5 to trial17818531-26 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:1603: oid 26 updating version 0 to 747 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:1603: oid 26 version 747 is already newer than 746 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:1603: oid 26 version 747 is already newer than 745 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:1603: oid 26 version 747 is already newer than 744 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 747 (ObjNum 564 snap 158 seq_num 564) dirty exists 2026-02-21T22:22:27.886 INFO:tasks.rados.rados.0.trial178.stdout:1603: left oid 26 (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1601: done (3 left) 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1602: done (2 left) 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1603: done (1 left) 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1605: done (0 left) 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1606: write (excl) oid 29 current snap is 158 2026-02-21T22:22:27.887 INFO:tasks.rados.rados.0.trial178.stdout:1606: seq_num 567 ranges {729569=476305,1996349=505866,3004131=525438} 2026-02-21T22:22:27.889 INFO:tasks.rados.rados.0.trial178.stdout:1606: writing trial17818531-29 from 729569 to 1205874 tid 1 2026-02-21T22:22:27.898 INFO:tasks.rados.rados.0.trial178.stdout:1606: writing trial17818531-29 from 1996349 to 2502215 tid 2 2026-02-21T22:22:27.905 INFO:tasks.rados.rados.0.trial178.stdout:1606: writing trial17818531-29 from 3004131 to 3529569 tid 3 2026-02-21T22:22:27.906 INFO:tasks.rados.rados.0.trial178.stdout:1607: copy_from oid 30 from oid 16 current snap is 158 2026-02-21T22:22:27.906 INFO:tasks.rados.rados.0.trial178.stdout:1606: finishing write tid 1 to trial17818531-29 2026-02-21T22:22:27.906 INFO:tasks.rados.rados.0.trial178.stdout:1606: finishing write tid 2 to trial17818531-29 2026-02-21T22:22:27.906 INFO:tasks.rados.rados.0.trial178.stdout:1608: write oid 10 current snap is 158 2026-02-21T22:22:27.906 INFO:tasks.rados.rados.0.trial178.stdout:1608: seq_num 568 ranges {746376=551554,2054831=649524,3315201=231175} 2026-02-21T22:22:27.913 INFO:tasks.rados.rados.0.trial178.stdout:1608: writing trial17818531-10 from 746376 to 1297930 tid 1 2026-02-21T22:22:27.922 INFO:tasks.rados.rados.0.trial178.stdout:1608: writing trial17818531-10 from 2054831 to 2704355 tid 2 2026-02-21T22:22:27.925 INFO:tasks.rados.rados.0.trial178.stdout:1608: writing trial17818531-10 from 3315201 to 3546376 tid 3 2026-02-21T22:22:27.926 INFO:tasks.rados.rados.0.trial178.stdout:1609: read oid 18 snap -1 2026-02-21T22:22:27.926 INFO:tasks.rados.rados.0.trial178.stdout:1609: expect deleted 2026-02-21T22:22:27.926 INFO:tasks.rados.rados.0.trial178.stdout:1610: write oid 37 current snap is 158 2026-02-21T22:22:27.926 INFO:tasks.rados.rados.0.trial178.stdout:1610: seq_num 569 ranges {763183=626803,1996960=793182,3226271=336912} 2026-02-21T22:22:27.931 INFO:tasks.rados.rados.0.trial178.stdout:1610: writing trial17818531-37 from 763183 to 1389986 tid 1 2026-02-21T22:22:27.936 INFO:tasks.rados.rados.0.trial178.stdout:1610: writing trial17818531-37 from 1996960 to 2790142 tid 2 2026-02-21T22:22:27.939 INFO:tasks.rados.rados.0.trial178.stdout:1610: writing trial17818531-37 from 3226271 to 3563183 tid 3 2026-02-21T22:22:27.939 INFO:tasks.rados.rados.0.trial178.stdout:1611: write (excl) oid 41 current snap is 158 2026-02-21T22:22:27.939 INFO:tasks.rados.rados.0.trial178.stdout:1611: seq_num 570 ranges {779990=702052,2055442=653193,3253694=326296} 2026-02-21T22:22:27.945 INFO:tasks.rados.rados.0.trial178.stdout:1611: writing trial17818531-41 from 779990 to 1482042 tid 1 2026-02-21T22:22:27.950 INFO:tasks.rados.rados.0.trial178.stdout:1611: writing trial17818531-41 from 2055442 to 2708635 tid 2 2026-02-21T22:22:27.953 INFO:tasks.rados.rados.0.trial178.stdout:1611: writing trial17818531-41 from 3253694 to 3579990 tid 3 2026-02-21T22:22:27.953 INFO:tasks.rados.rados.0.trial178.stdout:1612: copy_from oid 21 from oid 5 current snap is 158 2026-02-21T22:22:27.953 INFO:tasks.rados.rados.0.trial178.stdout:1606: finishing write tid 3 to trial17818531-29 2026-02-21T22:22:27.953 INFO:tasks.rados.rados.0.trial178.stdout:1613: rollback oid 39 current snap is 158 2026-02-21T22:22:27.953 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 145 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: finishing write tid 4 to trial17818531-29 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: finishing write tid 5 to trial17818531-29 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: oid 29 updating version 0 to 743 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: oid 29 updating version 743 to 744 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: oid 29 updating version 744 to 746 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: oid 29 version 746 is already newer than 745 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 746 (ObjNum 567 snap 158 seq_num 567) dirty exists 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: left oid 29 (ObjNum 567 snap 158 seq_num 567) 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1607: finishing copy_from racing read to trial17818531-30 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1607: finishing copy_from to trial17818531-30 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 498 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1608: finishing write tid 1 to trial17818531-10 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1608: finishing write tid 2 to trial17818531-10 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1606: done (7 left) 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1607: done (6 left) 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1614: write (excl) oid 17 current snap is 158 2026-02-21T22:22:27.954 INFO:tasks.rados.rados.0.trial178.stdout:1614: seq_num 571 ranges {796797=493654,1830277=796851,3397470=199327} 2026-02-21T22:22:27.958 INFO:tasks.rados.rados.0.trial178.stdout:1614: writing trial17818531-17 from 796797 to 1290451 tid 1 2026-02-21T22:22:27.963 INFO:tasks.rados.rados.0.trial178.stdout:1614: writing trial17818531-17 from 1830277 to 2627128 tid 2 2026-02-21T22:22:27.967 INFO:tasks.rados.rados.0.trial178.stdout:1614: writing trial17818531-17 from 3397470 to 3596797 tid 3 2026-02-21T22:22:27.967 INFO:tasks.rados.rados.0.trial178.stdout:1615: write oid 15 current snap is 158 2026-02-21T22:22:27.967 INFO:tasks.rados.rados.0.trial178.stdout:1615: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2026-02-21T22:22:27.971 INFO:tasks.rados.rados.0.trial178.stdout:1615: writing trial17818531-15 from 413604 to 982507 tid 1 2026-02-21T22:22:27.975 INFO:tasks.rados.rados.0.trial178.stdout:1615: writing trial17818531-15 from 1488759 to 2145621 tid 2 2026-02-21T22:22:27.980 INFO:tasks.rados.rados.0.trial178.stdout:1615: writing trial17818531-15 from 2624893 to 3216602 tid 3 2026-02-21T22:22:27.980 INFO:tasks.rados.rados.0.trial178.stdout:1616: read oid 29 snap 152 2026-02-21T22:22:27.980 INFO:tasks.rados.rados.0.trial178.stdout:1616: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:22:27.980 INFO:tasks.rados.rados.0.trial178.stdout:1617: snap_remove snap 154 2026-02-21T22:22:28.470 INFO:tasks.rados.rados.0.trial178.stdout:1617: done (9 left) 2026-02-21T22:22:28.470 INFO:tasks.rados.rados.0.trial178.stdout:1618: write oid 23 current snap is 158 2026-02-21T22:22:28.470 INFO:tasks.rados.rados.0.trial178.stdout:1618: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2026-02-21T22:22:28.474 INFO:tasks.rados.rados.0.trial178.stdout:1618: writing trial17818531-23 from 430411 to 1074563 tid 1 2026-02-21T22:22:28.478 INFO:tasks.rados.rados.0.trial178.stdout:1618: writing trial17818531-23 from 1830888 to 2231408 tid 2 2026-02-21T22:22:28.484 INFO:tasks.rados.rados.0.trial178.stdout:1618: writing trial17818531-23 from 2935963 to 3338944 tid 3 2026-02-21T22:22:28.484 INFO:tasks.rados.rados.0.trial178.stdout:1619: copy_from oid 42 from oid 4 current snap is 158 2026-02-21T22:22:28.484 INFO:tasks.rados.rados.0.trial178.stdout:1608: finishing write tid 3 to trial17818531-10 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: finishing write tid 4 to trial17818531-10 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: finishing write tid 5 to trial17818531-10 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: oid 10 updating version 0 to 751 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: oid 10 updating version 751 to 752 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: oid 10 version 752 is already newer than 750 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: oid 10 version 752 is already newer than 749 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 752 (ObjNum 568 snap 158 seq_num 568) dirty exists 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1608: left oid 10 (ObjNum 568 snap 158 seq_num 568) 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1610: finishing write tid 1 to trial17818531-37 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1610: finishing write tid 2 to trial17818531-37 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1610: finishing write tid 3 to trial17818531-37 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1610: finishing write tid 4 to trial17818531-37 2026-02-21T22:22:28.485 INFO:tasks.rados.rados.0.trial178.stdout:1610: finishing write tid 5 to trial17818531-37 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1610: oid 37 updating version 0 to 521 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1610: oid 37 updating version 521 to 523 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1610: oid 37 version 523 is already newer than 522 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1610: oid 37 version 523 is already newer than 520 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 523 (ObjNum 569 snap 158 seq_num 569) dirty exists 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1610: left oid 37 (ObjNum 569 snap 158 seq_num 569) 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1611: finishing write tid 1 to trial17818531-41 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1611: finishing write tid 2 to trial17818531-41 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1612: finishing copy_from to trial17818531-21 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1612: got expected ENOENT (src dne) 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1612: finishing copy_from racing read to trial17818531-21 2026-02-21T22:22:28.486 INFO:tasks.rados.rados.0.trial178.stdout:1611: finishing write tid 3 to trial17818531-41 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1613: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: finishing write tid 4 to trial17818531-41 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: finishing write tid 5 to trial17818531-41 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: oid 41 updating version 0 to 686 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: oid 41 version 686 is already newer than 684 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: oid 41 version 686 is already newer than 685 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: oid 41 version 686 is already newer than 683 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 686 (ObjNum 570 snap 158 seq_num 570) dirty exists 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1611: left oid 41 (ObjNum 570 snap 158 seq_num 570) 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1613: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1613: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 746 (ObjNum 466 snap 133 seq_num 466) dirty exists 2026-02-21T22:22:28.487 INFO:tasks.rados.rados.0.trial178.stdout:1614: finishing write tid 1 to trial17818531-17 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1614: finishing write tid 2 to trial17818531-17 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1608: done (10 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1609: done (9 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1610: done (8 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1611: done (7 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1612: done (6 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1613: done (5 left) 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1620: write (excl) oid 27 current snap is 158 2026-02-21T22:22:28.488 INFO:tasks.rados.rados.0.trial178.stdout:1620: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2026-02-21T22:22:28.489 INFO:tasks.rados.rados.0.trial178.stdout:1620: writing trial17818531-27 from 447218 to 1166619 tid 1 2026-02-21T22:22:28.492 INFO:tasks.rados.rados.0.trial178.stdout:1620: writing trial17818531-27 from 1889370 to 2549901 tid 2 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:1620: writing trial17818531-27 from 2963386 to 3577639 tid 3 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:1621: rollback oid 8 current snap is 158 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 145 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:1614: finishing write tid 3 to trial17818531-17 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:1622: write (excl) oid 38 current snap is 158 2026-02-21T22:22:28.497 INFO:tasks.rados.rados.0.trial178.stdout:1622: seq_num 575 ranges {464025=794650,1947852=404189,2990809=425525} 2026-02-21T22:22:28.501 INFO:tasks.rados.rados.0.trial178.stdout:1622: writing trial17818531-38 from 464025 to 1258675 tid 1 2026-02-21T22:22:28.505 INFO:tasks.rados.rados.0.trial178.stdout:1622: writing trial17818531-38 from 1947852 to 2352041 tid 2 2026-02-21T22:22:28.510 INFO:tasks.rados.rados.0.trial178.stdout:1622: writing trial17818531-38 from 2990809 to 3416334 tid 3 2026-02-21T22:22:28.510 INFO:tasks.rados.rados.0.trial178.stdout:1623: write oid 40 current snap is 158 2026-02-21T22:22:28.510 INFO:tasks.rados.rados.0.trial178.stdout:1623: seq_num 576 ranges {480832=469899,1606334=664200,3018232=662600} 2026-02-21T22:22:28.514 INFO:tasks.rados.rados.0.trial178.stdout:1623: writing trial17818531-40 from 480832 to 950731 tid 1 2026-02-21T22:22:28.519 INFO:tasks.rados.rados.0.trial178.stdout:1623: writing trial17818531-40 from 1606334 to 2270534 tid 2 2026-02-21T22:22:28.525 INFO:tasks.rados.rados.0.trial178.stdout:1623: writing trial17818531-40 from 3018232 to 3680832 tid 3 2026-02-21T22:22:28.526 INFO:tasks.rados.rados.0.trial178.stdout:1624: read oid 5 snap -1 2026-02-21T22:22:28.526 INFO:tasks.rados.rados.0.trial178.stdout:1624: expect deleted 2026-02-21T22:22:28.526 INFO:tasks.rados.rados.0.trial178.stdout:1625: snap_create 2026-02-21T22:22:28.526 INFO:tasks.rados.rados.0.trial178.stdout:1614: finishing write tid 4 to trial17818531-17 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:1614: finishing write tid 5 to trial17818531-17 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:1614: oid 17 updating version 0 to 690 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:1614: oid 17 version 690 is already newer than 688 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:1614: oid 17 version 690 is already newer than 689 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:1614: oid 17 version 690 is already newer than 687 2026-02-21T22:22:28.527 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 690 (ObjNum 571 snap 158 seq_num 571) dirty exists 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1614: left oid 17 (ObjNum 571 snap 158 seq_num 571) 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1615: finishing write tid 1 to trial17818531-15 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1615: finishing write tid 2 to trial17818531-15 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1615: finishing write tid 3 to trial17818531-15 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1615: finishing write tid 4 to trial17818531-15 2026-02-21T22:22:28.528 INFO:tasks.rados.rados.0.trial178.stdout:1615: finishing write tid 5 to trial17818531-15 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:1615: oid 15 updating version 0 to 691 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:1615: oid 15 updating version 691 to 693 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:1615: oid 15 version 693 is already newer than 692 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:1615: oid 15 updating version 693 to 694 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 694 (ObjNum 572 snap 158 seq_num 572) dirty exists 2026-02-21T22:22:28.529 INFO:tasks.rados.rados.0.trial178.stdout:1615: left oid 15 (ObjNum 572 snap 158 seq_num 572) 2026-02-21T22:22:28.530 INFO:tasks.rados.rados.0.trial178.stdout:1616: expect (ObjNum 469 snap 133 seq_num 469) 2026-02-21T22:22:28.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:28.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:28.564 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2869673 2026-02-21T22:22:28.662 INFO:tasks.rados.rados.0.trial178.stdout:1618: finishing write tid 1 to trial17818531-23 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: finishing write tid 2 to trial17818531-23 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: finishing write tid 3 to trial17818531-23 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: finishing write tid 4 to trial17818531-23 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: finishing write tid 5 to trial17818531-23 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: oid 23 updating version 0 to 442 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: oid 23 updating version 442 to 443 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: oid 23 updating version 443 to 444 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:1618: oid 23 version 444 is already newer than 441 2026-02-21T22:22:28.663 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 444 (ObjNum 573 snap 158 seq_num 573) dirty exists 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1618: left oid 23 (ObjNum 573 snap 158 seq_num 573) 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: finishing write tid 1 to trial17818531-27 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1619: finishing copy_from racing read to trial17818531-42 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1619: finishing copy_from to trial17818531-42 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 364 (ObjNum 498 snap 139 seq_num 498) dirty exists 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: finishing write tid 2 to trial17818531-27 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: finishing write tid 3 to trial17818531-27 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: finishing write tid 4 to trial17818531-27 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1621: finishing rollback tid 0 to trial17818531-8 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: finishing write tid 5 to trial17818531-27 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: oid 27 updating version 0 to 747 2026-02-21T22:22:28.664 INFO:tasks.rados.rados.0.trial178.stdout:1620: oid 27 updating version 747 to 749 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1620: oid 27 updating version 749 to 750 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1620: oid 27 version 750 is already newer than 748 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 750 (ObjNum 574 snap 158 seq_num 574) dirty exists 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1620: left oid 27 (ObjNum 574 snap 158 seq_num 574) 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1621: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1621: finishing rollback tid 2 to trial17818531-8 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 908 (ObjNum 497 snap 139 seq_num 497) dirty exists 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: finishing write tid 1 to trial17818531-38 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: finishing write tid 2 to trial17818531-38 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: finishing write tid 3 to trial17818531-38 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: finishing write tid 4 to trial17818531-38 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: finishing write tid 5 to trial17818531-38 2026-02-21T22:22:28.665 INFO:tasks.rados.rados.0.trial178.stdout:1622: oid 38 updating version 0 to 409 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1622: oid 38 updating version 409 to 411 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1622: oid 38 version 411 is already newer than 408 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1622: oid 38 version 411 is already newer than 410 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 411 (ObjNum 575 snap 158 seq_num 575) dirty exists 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1622: left oid 38 (ObjNum 575 snap 158 seq_num 575) 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: finishing write tid 1 to trial17818531-40 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: finishing write tid 2 to trial17818531-40 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: finishing write tid 3 to trial17818531-40 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: finishing write tid 4 to trial17818531-40 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: finishing write tid 5 to trial17818531-40 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: oid 40 updating version 0 to 507 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: oid 40 updating version 507 to 508 2026-02-21T22:22:28.666 INFO:tasks.rados.rados.0.trial178.stdout:1623: oid 40 updating version 508 to 509 2026-02-21T22:22:28.667 INFO:tasks.rados.rados.0.trial178.stdout:1623: oid 40 updating version 509 to 510 2026-02-21T22:22:28.667 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 510 (ObjNum 576 snap 158 seq_num 576) dirty exists 2026-02-21T22:22:28.667 INFO:tasks.rados.rados.0.trial178.stdout:1623: left oid 40 (ObjNum 576 snap 158 seq_num 576) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1614: done (10 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1615: done (9 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1616: done (8 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1618: done (7 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1619: done (6 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1620: done (5 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1621: done (4 left) 2026-02-21T22:22:29.473 INFO:tasks.rados.rados.0.trial178.stdout:1622: done (3 left) 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1623: done (2 left) 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1624: done (1 left) 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1625: done (0 left) 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1626: rollback oid 2 current snap is 159 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 158 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1627: read oid 15 snap -1 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1627: expect (ObjNum 572 snap 158 seq_num 572) 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1628: rollback oid 12 current snap is 159 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 152 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1629: delete oid 11 current snap is 159 2026-02-21T22:22:29.474 INFO:tasks.rados.rados.0.trial178.stdout:1626: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:22:29.475 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 748 (ObjNum 33 snap 0 seq_num 101034352) dirty dne 2026-02-21T22:22:29.475 INFO:tasks.rados.rados.0.trial178.stdout:1628: finishing rollback tid 0 to trial17818531-12 2026-02-21T22:22:29.476 INFO:tasks.rados.rados.0.trial178.stdout:1628: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:22:29.476 INFO:tasks.rados.rados.0.trial178.stdout:1628: finishing rollback tid 2 to trial17818531-12 2026-02-21T22:22:29.476 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 501 (ObjNum 512 snap 145 seq_num 512) dirty exists 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1626: done (3 left) 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1628: done (2 left) 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1629: done (1 left) 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1630: rollback oid 31 current snap is 159 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 152 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1631: copy_from oid 11 from oid 36 current snap is 159 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1632: read oid 7 snap -1 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1632: expect deleted 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1633: read oid 26 snap -1 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1633: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:22:29.481 INFO:tasks.rados.rados.0.trial178.stdout:1634: snap_create 2026-02-21T22:22:29.482 INFO:tasks.rados.rados.0.trial178.stdout:1627: expect (ObjNum 572 snap 158 seq_num 572) 2026-02-21T22:22:29.482 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:29.656 INFO:tasks.rados.rados.0.trial178.stdout:1630: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:22:29.656 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 0 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-21T22:22:29.656 INFO:tasks.rados.rados.0.trial178.stdout:1633: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:22:29.782 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2990234 2026-02-21T22:22:29.792 INFO:tasks.rados.rados.0.trial178.stdout:1631: finishing copy_from racing read to trial17818531-11 2026-02-21T22:22:29.792 INFO:tasks.rados.rados.0.trial178.stdout:1631: finishing copy_from to trial17818531-11 2026-02-21T22:22:29.792 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 362 (ObjNum 532 snap 150 seq_num 532) dirty exists 2026-02-21T22:22:30.477 INFO:tasks.rados.rados.0.trial178.stdout:1627: done (5 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1630: done (4 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1631: done (3 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1632: done (2 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1633: done (1 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1634: done (0 left) 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1635: write (excl) oid 22 current snap is 160 2026-02-21T22:22:30.478 INFO:tasks.rados.rados.0.trial178.stdout:1635: seq_num 578 ranges {514446=736750,1723298=667869,3073078=641368} 2026-02-21T22:22:30.485 INFO:tasks.rados.rados.0.trial178.stdout:1635: writing trial17818531-22 from 514446 to 1251196 tid 1 2026-02-21T22:22:30.493 INFO:tasks.rados.rados.0.trial178.stdout:1635: writing trial17818531-22 from 1723298 to 2391167 tid 2 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1635: writing trial17818531-22 from 3073078 to 3714446 tid 3 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1636: read oid 28 snap -1 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1636: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1637: read oid 16 snap -1 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1637: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1638: snap_create 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1635: finishing write tid 1 to trial17818531-22 2026-02-21T22:22:30.503 INFO:tasks.rados.rados.0.trial178.stdout:1635: finishing write tid 2 to trial17818531-22 2026-02-21T22:22:30.504 INFO:tasks.rados.rados.0.trial178.stdout:1635: finishing write tid 3 to trial17818531-22 2026-02-21T22:22:30.504 INFO:tasks.rados.rados.0.trial178.stdout:1635: finishing write tid 4 to trial17818531-22 2026-02-21T22:22:30.505 INFO:tasks.rados.rados.0.trial178.stdout:1635: finishing write tid 5 to trial17818531-22 2026-02-21T22:22:30.505 INFO:tasks.rados.rados.0.trial178.stdout:1635: oid 22 updating version 0 to 511 2026-02-21T22:22:30.505 INFO:tasks.rados.rados.0.trial178.stdout:1635: oid 22 updating version 511 to 513 2026-02-21T22:22:30.505 INFO:tasks.rados.rados.0.trial178.stdout:1635: oid 22 updating version 513 to 514 2026-02-21T22:22:30.505 INFO:tasks.rados.rados.0.trial178.stdout:1635: oid 22 version 514 is already newer than 512 2026-02-21T22:22:30.506 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 514 (ObjNum 578 snap 160 seq_num 578) dirty exists 2026-02-21T22:22:30.506 INFO:tasks.rados.rados.0.trial178.stdout:1635: left oid 22 (ObjNum 578 snap 160 seq_num 578) 2026-02-21T22:22:30.509 INFO:tasks.rados.rados.0.trial178.stdout:1636: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:22:30.509 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:30.580 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2143681 2026-02-21T22:22:30.591 INFO:tasks.rados.rados.0.trial178.stdout:1637: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:30.591 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:30.630 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:30.680 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1930223 2026-02-21T22:22:31.164 INFO:tasks.rados.rados.0.trial178.stdout:1635: done (3 left) 2026-02-21T22:22:31.164 INFO:tasks.rados.rados.0.trial178.stdout:1636: done (2 left) 2026-02-21T22:22:31.164 INFO:tasks.rados.rados.0.trial178.stdout:1637: done (1 left) 2026-02-21T22:22:31.164 INFO:tasks.rados.rados.0.trial178.stdout:1638: done (0 left) 2026-02-21T22:22:31.164 INFO:tasks.rados.rados.0.trial178.stdout:1639: snap_remove snap 158 2026-02-21T22:22:31.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:22:31.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:22:31.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:22:31.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:22:31.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:22:32.165 INFO:tasks.rados.rados.0.trial178.stdout:1639: done (0 left) 2026-02-21T22:22:32.165 INFO:tasks.rados.rados.0.trial178.stdout:1640: delete oid 46 current snap is 161 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1640: done (0 left) 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1641: copy_from oid 7 from oid 39 current snap is 161 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1642: read oid 14 snap -1 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1642: expect (ObjNum 515 snap 146 seq_num 515) 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1643: rollback oid 16 current snap is 161 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 160 2026-02-21T22:22:32.168 INFO:tasks.rados.rados.0.trial178.stdout:1644: read oid 24 snap -1 2026-02-21T22:22:32.169 INFO:tasks.rados.rados.0.trial178.stdout:1644: expect (ObjNum 559 snap 158 seq_num 559) 2026-02-21T22:22:32.169 INFO:tasks.rados.rados.0.trial178.stdout:1645: snap_remove snap 159 2026-02-21T22:22:33.167 INFO:tasks.rados.rados.0.trial178.stdout:1645: done (4 left) 2026-02-21T22:22:33.167 INFO:tasks.rados.rados.0.trial178.stdout:1646: read oid 25 snap -1 2026-02-21T22:22:33.167 INFO:tasks.rados.rados.0.trial178.stdout:1646: expect (ObjNum 560 snap 158 seq_num 560) 2026-02-21T22:22:33.167 INFO:tasks.rados.rados.0.trial178.stdout:1647: read oid 35 snap -1 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1647: expect (ObjNum 563 snap 158 seq_num 563) 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1648: rollback oid 43 current snap is 161 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 145 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1643: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1643: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1643: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 504 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:1642: expect (ObjNum 515 snap 146 seq_num 515) 2026-02-21T22:22:33.168 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:33.226 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 235005 2026-02-21T22:22:33.304 INFO:tasks.rados.rados.0.trial178.stdout:1644: expect (ObjNum 559 snap 158 seq_num 559) 2026-02-21T22:22:33.304 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 143289 2026-02-21T22:22:33.353 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1641: finishing copy_from racing read to trial17818531-7 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1641: finishing copy_from to trial17818531-7 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 746 (ObjNum 466 snap 133 seq_num 466) dirty exists 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1641: done (6 left) 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1642: done (5 left) 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1643: done (4 left) 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1644: done (3 left) 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1649: write oid 46 current snap is 161 2026-02-21T22:22:33.379 INFO:tasks.rados.rados.0.trial178.stdout:1649: seq_num 580 ranges {548060=487248,1440262=671538,2844277=798238} 2026-02-21T22:22:33.383 INFO:tasks.rados.rados.0.trial178.stdout:1649: writing trial17818531-46 from 548060 to 1035308 tid 1 2026-02-21T22:22:33.387 INFO:tasks.rados.rados.0.trial178.stdout:1649: writing trial17818531-46 from 1440262 to 2111800 tid 2 2026-02-21T22:22:33.393 INFO:tasks.rados.rados.0.trial178.stdout:1649: writing trial17818531-46 from 2844277 to 3642515 tid 3 2026-02-21T22:22:33.393 INFO:tasks.rados.rados.0.trial178.stdout:1650: write (excl) oid 41 current snap is 161 2026-02-21T22:22:33.393 INFO:tasks.rados.rados.0.trial178.stdout:1648: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:22:33.393 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 0 (ObjNum 129 snap 0 seq_num 101532928) dirty dne 2026-02-21T22:22:33.393 INFO:tasks.rados.rados.0.trial178.stdout:1647: expect (ObjNum 563 snap 158 seq_num 563) 2026-02-21T22:22:33.423 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2240298 2026-02-21T22:22:33.475 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1853898 2026-02-21T22:22:33.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:33.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:33.573 INFO:tasks.rados.rados.0.trial178.stdout:1646: expect (ObjNum 560 snap 158 seq_num 560) 2026-02-21T22:22:33.579 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1203116 2026-02-21T22:22:33.659 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:33.704 INFO:tasks.rados.rados.0.trial178.stdout:1649: finishing write tid 1 to trial17818531-46 2026-02-21T22:22:33.704 INFO:tasks.rados.rados.0.trial178.stdout:1650: seq_num 581 ranges {564867=562497,1898744=415196,2755347=725863} 2026-02-21T22:22:33.708 INFO:tasks.rados.rados.0.trial178.stdout:1650: writing trial17818531-41 from 564867 to 1127364 tid 1 2026-02-21T22:22:33.713 INFO:tasks.rados.rados.0.trial178.stdout:1650: writing trial17818531-41 from 1898744 to 2313940 tid 2 2026-02-21T22:22:33.717 INFO:tasks.rados.rados.0.trial178.stdout:1650: writing trial17818531-41 from 2755347 to 3481210 tid 3 2026-02-21T22:22:33.717 INFO:tasks.rados.rados.0.trial178.stdout:1646: done (4 left) 2026-02-21T22:22:33.718 INFO:tasks.rados.rados.0.trial178.stdout:1647: done (3 left) 2026-02-21T22:22:33.718 INFO:tasks.rados.rados.0.trial178.stdout:1648: done (2 left) 2026-02-21T22:22:33.718 INFO:tasks.rados.rados.0.trial178.stdout:1651: snap_remove snap 160 2026-02-21T22:22:34.167 INFO:tasks.rados.rados.0.trial178.stdout:1651: done (2 left) 2026-02-21T22:22:34.168 INFO:tasks.rados.rados.0.trial178.stdout:1652: write (excl) oid 45 current snap is 161 2026-02-21T22:22:34.168 INFO:tasks.rados.rados.0.trial178.stdout:1652: seq_num 582 ranges {581674=637746,1840873=558854,3066417=537135} 2026-02-21T22:22:34.173 INFO:tasks.rados.rados.0.trial178.stdout:1652: writing trial17818531-45 from 581674 to 1219420 tid 1 2026-02-21T22:22:34.181 INFO:tasks.rados.rados.0.trial178.stdout:1652: writing trial17818531-45 from 1840873 to 2399727 tid 2 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1652: writing trial17818531-45 from 3066417 to 3603552 tid 3 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1653: delete oid 29 current snap is 161 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1649: finishing write tid 2 to trial17818531-46 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1649: finishing write tid 3 to trial17818531-46 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1649: finishing write tid 4 to trial17818531-46 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1649: finishing write tid 5 to trial17818531-46 2026-02-21T22:22:34.190 INFO:tasks.rados.rados.0.trial178.stdout:1649: oid 46 updating version 0 to 447 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1649: oid 46 version 447 is already newer than 446 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1649: oid 46 updating version 447 to 448 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1649: oid 46 updating version 448 to 449 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 449 (ObjNum 580 snap 161 seq_num 580) dirty exists 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1649: left oid 46 (ObjNum 580 snap 161 seq_num 580) 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: finishing write tid 1 to trial17818531-41 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: finishing write tid 2 to trial17818531-41 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: finishing write tid 3 to trial17818531-41 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: finishing write tid 4 to trial17818531-41 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: finishing write tid 5 to trial17818531-41 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: oid 41 updating version 0 to 697 2026-02-21T22:22:34.191 INFO:tasks.rados.rados.0.trial178.stdout:1650: oid 41 version 697 is already newer than 695 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:1650: oid 41 updating version 697 to 698 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:1650: oid 41 version 698 is already newer than 696 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 698 (ObjNum 581 snap 161 seq_num 581) dirty exists 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:1650: left oid 41 (ObjNum 581 snap 161 seq_num 581) 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:1652: finishing write tid 1 to trial17818531-45 2026-02-21T22:22:34.192 INFO:tasks.rados.rados.0.trial178.stdout:1652: finishing write tid 2 to trial17818531-45 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1649: done (3 left) 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1650: done (2 left) 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1653: done (1 left) 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1654: snap_create 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1652: finishing write tid 3 to trial17818531-45 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1652: finishing write tid 4 to trial17818531-45 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1652: finishing write tid 5 to trial17818531-45 2026-02-21T22:22:34.193 INFO:tasks.rados.rados.0.trial178.stdout:1652: oid 45 updating version 0 to 524 2026-02-21T22:22:34.194 INFO:tasks.rados.rados.0.trial178.stdout:1652: oid 45 updating version 524 to 527 2026-02-21T22:22:34.194 INFO:tasks.rados.rados.0.trial178.stdout:1652: oid 45 version 527 is already newer than 526 2026-02-21T22:22:34.194 INFO:tasks.rados.rados.0.trial178.stdout:1652: oid 45 version 527 is already newer than 525 2026-02-21T22:22:34.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 527 (ObjNum 582 snap 161 seq_num 582) dirty exists 2026-02-21T22:22:34.194 INFO:tasks.rados.rados.0.trial178.stdout:1652: left oid 45 (ObjNum 582 snap 161 seq_num 582) 2026-02-21T22:22:35.174 INFO:tasks.rados.rados.0.trial178.stdout:1652: done (1 left) 2026-02-21T22:22:35.174 INFO:tasks.rados.rados.0.trial178.stdout:1654: done (0 left) 2026-02-21T22:22:35.174 INFO:tasks.rados.rados.0.trial178.stdout:1655: write (excl) oid 15 current snap is 162 2026-02-21T22:22:35.174 INFO:tasks.rados.rados.0.trial178.stdout:1655: seq_num 584 ranges {615288=788244,1957837=562523,3121263=559679} 2026-02-21T22:22:35.183 INFO:tasks.rados.rados.0.trial178.stdout:1655: writing trial17818531-15 from 615288 to 1403532 tid 1 2026-02-21T22:22:35.191 INFO:tasks.rados.rados.0.trial178.stdout:1655: writing trial17818531-15 from 1957837 to 2520360 tid 2 2026-02-21T22:22:35.199 INFO:tasks.rados.rados.0.trial178.stdout:1655: writing trial17818531-15 from 3121263 to 3680942 tid 3 2026-02-21T22:22:35.199 INFO:tasks.rados.rados.0.trial178.stdout:1656: snap_remove snap 145 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1656: done (1 left) 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1657: copy_from oid 17 from oid 48 current snap is 162 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: finishing write tid 1 to trial17818531-15 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: finishing write tid 2 to trial17818531-15 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: finishing write tid 3 to trial17818531-15 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: finishing write tid 4 to trial17818531-15 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: finishing write tid 5 to trial17818531-15 2026-02-21T22:22:36.165 INFO:tasks.rados.rados.0.trial178.stdout:1655: oid 15 updating version 0 to 699 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1655: oid 15 updating version 699 to 701 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1655: oid 15 updating version 701 to 702 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1655: oid 15 version 702 is already newer than 700 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 702 (ObjNum 584 snap 162 seq_num 584) dirty exists 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1655: left oid 15 (ObjNum 584 snap 162 seq_num 584) 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1655: done (1 left) 2026-02-21T22:22:36.166 INFO:tasks.rados.rados.0.trial178.stdout:1658: snap_remove snap 161 2026-02-21T22:22:37.166 INFO:tasks.rados.rados.0.trial178.stdout:1658: done (1 left) 2026-02-21T22:22:37.166 INFO:tasks.rados.rados.0.trial178.stdout:1659: delete oid 23 current snap is 162 2026-02-21T22:22:37.166 INFO:tasks.rados.rados.0.trial178.stdout:1657: finishing copy_from to trial17818531-17 2026-02-21T22:22:37.166 INFO:tasks.rados.rados.0.trial178.stdout:1657: got expected ENOENT (src dne) 2026-02-21T22:22:37.166 INFO:tasks.rados.rados.0.trial178.stdout:1657: finishing copy_from racing read to trial17818531-17 2026-02-21T22:22:37.168 INFO:tasks.rados.rados.0.trial178.stdout:1657: done (1 left) 2026-02-21T22:22:37.169 INFO:tasks.rados.rados.0.trial178.stdout:1659: done (0 left) 2026-02-21T22:22:37.169 INFO:tasks.rados.rados.0.trial178.stdout:1660: copy_from oid 2 from oid 30 current snap is 162 2026-02-21T22:22:37.169 INFO:tasks.rados.rados.0.trial178.stdout:1661: delete oid 17 current snap is 162 2026-02-21T22:22:37.170 INFO:tasks.rados.rados.0.trial178.stdout:1661: done (1 left) 2026-02-21T22:22:37.170 INFO:tasks.rados.rados.0.trial178.stdout:1662: copy_from oid 43 from oid 4 current snap is 162 2026-02-21T22:22:37.170 INFO:tasks.rados.rados.0.trial178.stdout:1663: read oid 18 snap -1 2026-02-21T22:22:37.170 INFO:tasks.rados.rados.0.trial178.stdout:1663: expect deleted 2026-02-21T22:22:37.170 INFO:tasks.rados.rados.0.trial178.stdout:1664: read oid 48 snap 152 2026-02-21T22:22:37.171 INFO:tasks.rados.rados.0.trial178.stdout:1664: expect (ObjNum 538 snap 151 seq_num 538) 2026-02-21T22:22:37.171 INFO:tasks.rados.rados.0.trial178.stdout:1665: read oid 7 snap -1 2026-02-21T22:22:37.171 INFO:tasks.rados.rados.0.trial178.stdout:1665: expect (ObjNum 466 snap 133 seq_num 466) 2026-02-21T22:22:37.171 INFO:tasks.rados.rados.0.trial178.stdout:1666: delete oid 16 current snap is 162 2026-02-21T22:22:37.177 INFO:tasks.rados.rados.0.trial178.stdout:1664: expect (ObjNum 538 snap 151 seq_num 538) 2026-02-21T22:22:37.249 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2084521 2026-02-21T22:22:37.295 INFO:tasks.rados.rados.0.trial178.stdout:1662: finishing copy_from racing read to trial17818531-43 2026-02-21T22:22:37.295 INFO:tasks.rados.rados.0.trial178.stdout:1665: expect (ObjNum 466 snap 133 seq_num 466) 2026-02-21T22:22:37.388 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:37.437 INFO:tasks.rados.rados.0.trial178.stdout:1662: finishing copy_from to trial17818531-43 2026-02-21T22:22:37.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 364 (ObjNum 498 snap 139 seq_num 498) dirty exists 2026-02-21T22:22:37.437 INFO:tasks.rados.rados.0.trial178.stdout:1660: finishing copy_from racing read to trial17818531-2 2026-02-21T22:22:37.437 INFO:tasks.rados.rados.0.trial178.stdout:1660: finishing copy_from to trial17818531-2 2026-02-21T22:22:37.437 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 498 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1660: done (5 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1662: done (4 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1663: done (3 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1664: done (2 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1665: done (1 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1666: done (0 left) 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1667: write (excl) oid 44 current snap is 162 2026-02-21T22:22:37.438 INFO:tasks.rados.rados.0.trial178.stdout:1667: seq_num 588 ranges {682516=405593,1791765=569861,2830955=721120} 2026-02-21T22:22:37.443 INFO:tasks.rados.rados.0.trial178.stdout:1667: writing trial17818531-44 from 682516 to 1088109 tid 1 2026-02-21T22:22:37.452 INFO:tasks.rados.rados.0.trial178.stdout:1667: writing trial17818531-44 from 1791765 to 2361626 tid 2 2026-02-21T22:22:37.460 INFO:tasks.rados.rados.0.trial178.stdout:1667: writing trial17818531-44 from 2830955 to 3552075 tid 3 2026-02-21T22:22:37.460 INFO:tasks.rados.rados.0.trial178.stdout:1668: write oid 36 current snap is 162 2026-02-21T22:22:37.460 INFO:tasks.rados.rados.0.trial178.stdout:1668: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2026-02-21T22:22:37.468 INFO:tasks.rados.rados.0.trial178.stdout:1668: writing trial17818531-36 from 699323 to 1180165 tid 1 2026-02-21T22:22:37.475 INFO:tasks.rados.rados.0.trial178.stdout:1668: writing trial17818531-36 from 1850247 to 2280119 tid 2 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1668: writing trial17818531-36 from 2858378 to 3390770 tid 3 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1669: read oid 13 snap -1 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1669: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1670: snap_create 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: finishing write tid 1 to trial17818531-44 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: finishing write tid 2 to trial17818531-44 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: finishing write tid 3 to trial17818531-44 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: finishing write tid 4 to trial17818531-44 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: finishing write tid 5 to trial17818531-44 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: oid 44 updating version 0 to 507 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: oid 44 version 507 is already newer than 506 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: oid 44 updating version 507 to 508 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: oid 44 updating version 508 to 509 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 509 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:22:37.482 INFO:tasks.rados.rados.0.trial178.stdout:1667: left oid 44 (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:22:37.483 INFO:tasks.rados.rados.0.trial178.stdout:1668: finishing write tid 1 to trial17818531-36 2026-02-21T22:22:37.483 INFO:tasks.rados.rados.0.trial178.stdout:1668: finishing write tid 2 to trial17818531-36 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: finishing write tid 3 to trial17818531-36 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: finishing write tid 4 to trial17818531-36 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: finishing write tid 5 to trial17818531-36 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: oid 36 updating version 0 to 364 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: oid 36 version 364 is already newer than 363 2026-02-21T22:22:37.484 INFO:tasks.rados.rados.0.trial178.stdout:1668: oid 36 updating version 364 to 365 2026-02-21T22:22:37.485 INFO:tasks.rados.rados.0.trial178.stdout:1668: oid 36 updating version 365 to 366 2026-02-21T22:22:37.485 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 366 (ObjNum 589 snap 162 seq_num 589) dirty exists 2026-02-21T22:22:37.485 INFO:tasks.rados.rados.0.trial178.stdout:1668: left oid 36 (ObjNum 589 snap 162 seq_num 589) 2026-02-21T22:22:37.488 INFO:tasks.rados.rados.0.trial178.stdout:1669: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:22:37.529 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1707515 2026-02-21T22:22:37.639 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 748155 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1667: done (3 left) 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1668: done (2 left) 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1669: done (1 left) 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1670: done (0 left) 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1671: write (excl) oid 27 current snap is 163 2026-02-21T22:22:38.179 INFO:tasks.rados.rados.0.trial178.stdout:1671: seq_num 590 ranges {716130=556091,1792376=573530,2769448=460017,3821956=94174} 2026-02-21T22:22:38.184 INFO:tasks.rados.rados.0.trial178.stdout:1671: writing trial17818531-27 from 716130 to 1272221 tid 1 2026-02-21T22:22:38.188 INFO:tasks.rados.rados.0.trial178.stdout:1671: writing trial17818531-27 from 1792376 to 2365906 tid 2 2026-02-21T22:22:38.191 INFO:tasks.rados.rados.0.trial178.stdout:1671: writing trial17818531-27 from 2769448 to 3229465 tid 3 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1671: writing trial17818531-27 from 3821956 to 3916130 tid 4 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1672: copy_from oid 10 from oid 33 current snap is 163 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 1 to trial17818531-27 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 2 to trial17818531-27 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 3 to trial17818531-27 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1673: write (excl) oid 24 current snap is 163 2026-02-21T22:22:38.194 INFO:tasks.rados.rados.0.trial178.stdout:1673: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2026-02-21T22:22:38.199 INFO:tasks.rados.rados.0.trial178.stdout:1673: writing trial17818531-24 from 732937 to 1364277 tid 1 2026-02-21T22:22:38.202 INFO:tasks.rados.rados.0.trial178.stdout:1673: writing trial17818531-24 from 1850858 to 2284399 tid 2 2026-02-21T22:22:38.207 INFO:tasks.rados.rados.0.trial178.stdout:1673: writing trial17818531-24 from 2796871 to 3468160 tid 3 2026-02-21T22:22:38.209 INFO:tasks.rados.rados.0.trial178.stdout:1673: writing trial17818531-24 from 3888195 to 3932937 tid 4 2026-02-21T22:22:38.209 INFO:tasks.rados.rados.0.trial178.stdout:1674: write (excl) oid 29 current snap is 163 2026-02-21T22:22:38.209 INFO:tasks.rados.rados.0.trial178.stdout:1674: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2026-02-21T22:22:38.214 INFO:tasks.rados.rados.0.trial178.stdout:1674: writing trial17818531-29 from 749744 to 1456333 tid 1 2026-02-21T22:22:38.218 INFO:tasks.rados.rados.0.trial178.stdout:1674: writing trial17818531-29 from 1909340 to 2486539 tid 2 2026-02-21T22:22:38.223 INFO:tasks.rados.rados.0.trial178.stdout:1674: writing trial17818531-29 from 3224294 to 3706855 tid 3 2026-02-21T22:22:38.223 INFO:tasks.rados.rados.0.trial178.stdout:1675: write oid 15 current snap is 163 2026-02-21T22:22:38.223 INFO:tasks.rados.rados.0.trial178.stdout:1675: seq_num 593 ranges {766551=498191,1684175=720857,2851717=693833} 2026-02-21T22:22:38.228 INFO:tasks.rados.rados.0.trial178.stdout:1675: writing trial17818531-15 from 766551 to 1264742 tid 1 2026-02-21T22:22:38.233 INFO:tasks.rados.rados.0.trial178.stdout:1675: writing trial17818531-15 from 1684175 to 2405032 tid 2 2026-02-21T22:22:38.238 INFO:tasks.rados.rados.0.trial178.stdout:1675: writing trial17818531-15 from 2851717 to 3545550 tid 3 2026-02-21T22:22:38.238 INFO:tasks.rados.rados.0.trial178.stdout:1676: copy_from oid 6 from oid 44 current snap is 163 2026-02-21T22:22:38.238 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 4 to trial17818531-27 2026-02-21T22:22:38.238 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 5 to trial17818531-27 2026-02-21T22:22:38.238 INFO:tasks.rados.rados.0.trial178.stdout:1671: finishing write tid 6 to trial17818531-27 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: oid 27 updating version 0 to 753 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: oid 27 version 753 is already newer than 752 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: oid 27 updating version 753 to 755 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: oid 27 version 755 is already newer than 751 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: oid 27 version 755 is already newer than 754 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 755 (ObjNum 590 snap 163 seq_num 590) dirty exists 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: left oid 27 (ObjNum 590 snap 163 seq_num 590) 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 1 to trial17818531-24 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1671: done (5 left) 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1677: delete oid 14 current snap is 163 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 2 to trial17818531-24 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1672: finishing copy_from racing read to trial17818531-10 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1672: finishing copy_from to trial17818531-10 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 405 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:22:38.239 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 3 to trial17818531-24 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 4 to trial17818531-24 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 5 to trial17818531-24 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: finishing write tid 6 to trial17818531-24 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: oid 24 updating version 0 to 923 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: oid 24 version 923 is already newer than 922 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: oid 24 version 923 is already newer than 919 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: oid 24 version 923 is already newer than 921 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: oid 24 version 923 is already newer than 920 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 923 (ObjNum 591 snap 163 seq_num 591) dirty exists 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1673: left oid 24 (ObjNum 591 snap 163 seq_num 591) 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: finishing write tid 1 to trial17818531-29 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: finishing write tid 2 to trial17818531-29 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: finishing write tid 3 to trial17818531-29 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: finishing write tid 4 to trial17818531-29 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: finishing write tid 5 to trial17818531-29 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: oid 29 updating version 0 to 748 2026-02-21T22:22:38.240 INFO:tasks.rados.rados.0.trial178.stdout:1674: oid 29 updating version 748 to 750 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1674: oid 29 updating version 750 to 751 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1674: oid 29 version 751 is already newer than 749 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 751 (ObjNum 592 snap 163 seq_num 592) dirty exists 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1674: left oid 29 (ObjNum 592 snap 163 seq_num 592) 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1675: finishing write tid 1 to trial17818531-15 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1675: finishing write tid 2 to trial17818531-15 2026-02-21T22:22:38.241 INFO:tasks.rados.rados.0.trial178.stdout:1675: finishing write tid 3 to trial17818531-15 2026-02-21T22:22:38.242 INFO:tasks.rados.rados.0.trial178.stdout:1672: done (5 left) 2026-02-21T22:22:38.242 INFO:tasks.rados.rados.0.trial178.stdout:1673: done (4 left) 2026-02-21T22:22:38.242 INFO:tasks.rados.rados.0.trial178.stdout:1674: done (3 left) 2026-02-21T22:22:38.242 INFO:tasks.rados.rados.0.trial178.stdout:1677: done (2 left) 2026-02-21T22:22:38.242 INFO:tasks.rados.rados.0.trial178.stdout:1678: snap_remove snap 152 2026-02-21T22:22:38.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:38.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:39.183 INFO:tasks.rados.rados.0.trial178.stdout:1678: done (2 left) 2026-02-21T22:22:39.183 INFO:tasks.rados.rados.0.trial178.stdout:1679: snap_create 2026-02-21T22:22:39.183 INFO:tasks.rados.rados.0.trial178.stdout:1675: finishing write tid 4 to trial17818531-15 2026-02-21T22:22:39.183 INFO:tasks.rados.rados.0.trial178.stdout:1675: finishing write tid 5 to trial17818531-15 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: oid 15 updating version 0 to 707 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: oid 15 version 707 is already newer than 704 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: oid 15 version 707 is already newer than 706 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: oid 15 version 707 is already newer than 705 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 707 (ObjNum 593 snap 163 seq_num 593) dirty exists 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: left oid 15 (ObjNum 593 snap 163 seq_num 593) 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1676: finishing copy_from racing read to trial17818531-6 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:1676: finishing copy_from to trial17818531-6 2026-02-21T22:22:39.184 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 509 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1675: done (2 left) 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1676: done (1 left) 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1679: done (0 left) 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1680: rollback oid 26 current snap is 164 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 162 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1681: write (excl) oid 47 current snap is 164 2026-02-21T22:22:40.184 INFO:tasks.rados.rados.0.trial178.stdout:1681: seq_num 595 ranges {400165=648689,1684786=315379} 2026-02-21T22:22:40.188 INFO:tasks.rados.rados.0.trial178.stdout:1681: writing trial17818531-47 from 400165 to 1048854 tid 1 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:1681: writing trial17818531-47 from 1684786 to 2000165 tid 2 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:1682: snap_create 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:1680: finishing rollback tid 0 to trial17818531-26 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:1680: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:1680: finishing rollback tid 2 to trial17818531-26 2026-02-21T22:22:40.192 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 755 (ObjNum 564 snap 158 seq_num 564) dirty exists 2026-02-21T22:22:40.195 INFO:tasks.rados.rados.0.trial178.stdout:1681: finishing write tid 1 to trial17818531-47 2026-02-21T22:22:40.195 INFO:tasks.rados.rados.0.trial178.stdout:1681: finishing write tid 2 to trial17818531-47 2026-02-21T22:22:40.195 INFO:tasks.rados.rados.0.trial178.stdout:1681: finishing write tid 3 to trial17818531-47 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:1681: finishing write tid 4 to trial17818531-47 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:1681: oid 47 updating version 0 to 932 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:1681: oid 47 updating version 932 to 933 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:1681: oid 47 version 933 is already newer than 931 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 933 (ObjNum 595 snap 164 seq_num 595) dirty exists 2026-02-21T22:22:40.196 INFO:tasks.rados.rados.0.trial178.stdout:1681: left oid 47 (ObjNum 595 snap 164 seq_num 595) 2026-02-21T22:22:41.165 INFO:tasks.rados.rados.0.trial178.stdout:1680: done (2 left) 2026-02-21T22:22:41.166 INFO:tasks.rados.rados.0.trial178.stdout:1681: done (1 left) 2026-02-21T22:22:41.166 INFO:tasks.rados.rados.0.trial178.stdout:1682: done (0 left) 2026-02-21T22:22:41.166 INFO:tasks.rados.rados.0.trial178.stdout:1683: write (excl) oid 25 current snap is 165 2026-02-21T22:22:41.166 INFO:tasks.rados.rados.0.trial178.stdout:1683: seq_num 596 ranges {416972=723938,1743268=273704} 2026-02-21T22:22:41.173 INFO:tasks.rados.rados.0.trial178.stdout:1683: writing trial17818531-25 from 416972 to 1140910 tid 1 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1683: writing trial17818531-25 from 1743268 to 2016972 tid 2 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1684: rollback oid 41 current snap is 165 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 162 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1683: finishing write tid 1 to trial17818531-25 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1685: read oid 30 snap 162 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1685: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1686: write oid 20 current snap is 165 2026-02-21T22:22:41.180 INFO:tasks.rados.rados.0.trial178.stdout:1686: seq_num 597 ranges {433779=799187,1801750=232029} 2026-02-21T22:22:41.187 INFO:tasks.rados.rados.0.trial178.stdout:1686: writing trial17818531-20 from 433779 to 1232966 tid 1 2026-02-21T22:22:41.193 INFO:tasks.rados.rados.0.trial178.stdout:1686: writing trial17818531-20 from 1801750 to 2033779 tid 2 2026-02-21T22:22:41.193 INFO:tasks.rados.rados.0.trial178.stdout:1687: snap_create 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1684: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1684: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: finishing write tid 2 to trial17818531-25 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: finishing write tid 3 to trial17818531-25 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: finishing write tid 4 to trial17818531-25 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: oid 25 updating version 0 to 622 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: oid 25 version 622 is already newer than 621 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: oid 25 updating version 622 to 623 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 623 (ObjNum 596 snap 165 seq_num 596) dirty exists 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: left oid 25 (ObjNum 596 snap 165 seq_num 596) 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:1684: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:22:41.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 710 (ObjNum 581 snap 161 seq_num 581) dirty exists 2026-02-21T22:22:41.195 INFO:tasks.rados.rados.0.trial178.stdout:1685: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:41.234 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2632309 2026-02-21T22:22:41.284 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:41.324 INFO:tasks.rados.rados.0.trial178.stdout:1686: finishing write tid 1 to trial17818531-20 2026-02-21T22:22:41.324 INFO:tasks.rados.rados.0.trial178.stdout:1686: finishing write tid 2 to trial17818531-20 2026-02-21T22:22:41.324 INFO:tasks.rados.rados.0.trial178.stdout:1686: finishing write tid 3 to trial17818531-20 2026-02-21T22:22:41.324 INFO:tasks.rados.rados.0.trial178.stdout:1686: finishing write tid 4 to trial17818531-20 2026-02-21T22:22:41.325 INFO:tasks.rados.rados.0.trial178.stdout:1686: oid 20 updating version 0 to 711 2026-02-21T22:22:41.325 INFO:tasks.rados.rados.0.trial178.stdout:1686: oid 20 updating version 711 to 712 2026-02-21T22:22:41.325 INFO:tasks.rados.rados.0.trial178.stdout:1686: oid 20 updating version 712 to 713 2026-02-21T22:22:41.325 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 713 (ObjNum 597 snap 165 seq_num 597) dirty exists 2026-02-21T22:22:41.325 INFO:tasks.rados.rados.0.trial178.stdout:1686: left oid 20 (ObjNum 597 snap 165 seq_num 597) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1683: done (4 left) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1684: done (3 left) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1685: done (2 left) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1686: done (1 left) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1687: done (0 left) 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1688: rollback oid 36 current snap is 166 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 162 2026-02-21T22:22:42.194 INFO:tasks.rados.rados.0.trial178.stdout:1689: copy_from oid 14 from oid 21 current snap is 166 2026-02-21T22:22:42.195 INFO:tasks.rados.rados.0.trial178.stdout:1690: snap_remove snap 163 2026-02-21T22:22:43.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:43.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:43.942 INFO:tasks.rados.rados.0.trial178.stdout:1690: done (2 left) 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1691: snap_create 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1688: finishing rollback tid 0 to trial17818531-36 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1688: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1688: finishing rollback tid 2 to trial17818531-36 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 369 (ObjNum 589 snap 162 seq_num 589) dirty exists 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1689: finishing copy_from racing read to trial17818531-14 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:1689: finishing copy_from to trial17818531-14 2026-02-21T22:22:43.943 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 743 (ObjNum 471 snap 133 seq_num 471) dirty exists 2026-02-21T22:22:44.201 INFO:tasks.rados.rados.0.trial178.stdout:1688: done (2 left) 2026-02-21T22:22:44.202 INFO:tasks.rados.rados.0.trial178.stdout:1689: done (1 left) 2026-02-21T22:22:44.202 INFO:tasks.rados.rados.0.trial178.stdout:1691: done (0 left) 2026-02-21T22:22:44.202 INFO:tasks.rados.rados.0.trial178.stdout:1692: write (excl) oid 14 current snap is 167 2026-02-21T22:22:44.202 INFO:tasks.rados.rados.0.trial178.stdout:1692: seq_num 598 ranges {450586=474436,1343879=588206} 2026-02-21T22:22:44.207 INFO:tasks.rados.rados.0.trial178.stdout:1692: writing trial17818531-14 from 450586 to 925022 tid 1 2026-02-21T22:22:44.214 INFO:tasks.rados.rados.0.trial178.stdout:1692: writing trial17818531-14 from 1343879 to 1932085 tid 2 2026-02-21T22:22:44.214 INFO:tasks.rados.rados.0.trial178.stdout:1693: read oid 26 snap -1 2026-02-21T22:22:44.215 INFO:tasks.rados.rados.0.trial178.stdout:1693: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:22:44.215 INFO:tasks.rados.rados.0.trial178.stdout:1694: write (excl) oid 35 current snap is 167 2026-02-21T22:22:44.215 INFO:tasks.rados.rados.0.trial178.stdout:1692: finishing write tid 1 to trial17818531-14 2026-02-21T22:22:44.215 INFO:tasks.rados.rados.0.trial178.stdout:1694: seq_num 599 ranges {467393=549685,1802361=265032} 2026-02-21T22:22:44.219 INFO:tasks.rados.rados.0.trial178.stdout:1694: writing trial17818531-35 from 467393 to 1017078 tid 1 2026-02-21T22:22:44.224 INFO:tasks.rados.rados.0.trial178.stdout:1694: writing trial17818531-35 from 1802361 to 2067393 tid 2 2026-02-21T22:22:44.224 INFO:tasks.rados.rados.0.trial178.stdout:1695: read oid 43 snap -1 2026-02-21T22:22:44.224 INFO:tasks.rados.rados.0.trial178.stdout:1695: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:44.224 INFO:tasks.rados.rados.0.trial178.stdout:1696: write (excl) oid 39 current snap is 167 2026-02-21T22:22:44.224 INFO:tasks.rados.rados.0.trial178.stdout:1696: seq_num 600 ranges {484200=624934,1860843=223357} 2026-02-21T22:22:44.227 INFO:tasks.rados.rados.0.trial178.stdout:1696: writing trial17818531-39 from 484200 to 1109134 tid 1 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1696: writing trial17818531-39 from 1860843 to 2084200 tid 2 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1697: read oid 41 snap -1 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1697: expect (ObjNum 581 snap 161 seq_num 581) 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1698: snap_create 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1692: finishing write tid 2 to trial17818531-14 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1692: finishing write tid 3 to trial17818531-14 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1692: finishing write tid 4 to trial17818531-14 2026-02-21T22:22:44.232 INFO:tasks.rados.rados.0.trial178.stdout:1692: oid 14 updating version 0 to 757 2026-02-21T22:22:44.233 INFO:tasks.rados.rados.0.trial178.stdout:1692: oid 14 updating version 757 to 758 2026-02-21T22:22:44.233 INFO:tasks.rados.rados.0.trial178.stdout:1692: oid 14 updating version 758 to 759 2026-02-21T22:22:44.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 759 (ObjNum 598 snap 167 seq_num 598) dirty exists 2026-02-21T22:22:44.233 INFO:tasks.rados.rados.0.trial178.stdout:1692: left oid 14 (ObjNum 598 snap 167 seq_num 598) 2026-02-21T22:22:44.233 INFO:tasks.rados.rados.0.trial178.stdout:1693: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:22:44.299 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: finishing write tid 1 to trial17818531-35 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: finishing write tid 2 to trial17818531-35 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: finishing write tid 3 to trial17818531-35 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: finishing write tid 4 to trial17818531-35 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: oid 35 updating version 0 to 767 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: oid 35 version 767 is already newer than 765 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: oid 35 version 767 is already newer than 766 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 767 (ObjNum 599 snap 167 seq_num 599) dirty exists 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1694: left oid 35 (ObjNum 599 snap 167 seq_num 599) 2026-02-21T22:22:44.390 INFO:tasks.rados.rados.0.trial178.stdout:1695: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:22:44.407 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:44.425 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: finishing write tid 1 to trial17818531-39 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: finishing write tid 2 to trial17818531-39 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: finishing write tid 3 to trial17818531-39 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: finishing write tid 4 to trial17818531-39 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: oid 39 updating version 0 to 761 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: oid 39 version 761 is already newer than 760 2026-02-21T22:22:44.443 INFO:tasks.rados.rados.0.trial178.stdout:1696: oid 39 updating version 761 to 762 2026-02-21T22:22:44.444 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 762 (ObjNum 600 snap 167 seq_num 600) dirty exists 2026-02-21T22:22:44.444 INFO:tasks.rados.rados.0.trial178.stdout:1696: left oid 39 (ObjNum 600 snap 167 seq_num 600) 2026-02-21T22:22:44.444 INFO:tasks.rados.rados.0.trial178.stdout:1697: expect (ObjNum 581 snap 161 seq_num 581) 2026-02-21T22:22:44.519 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3460973 2026-02-21T22:22:45.205 INFO:tasks.rados.rados.0.trial178.stdout:1692: done (6 left) 2026-02-21T22:22:45.205 INFO:tasks.rados.rados.0.trial178.stdout:1693: done (5 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1694: done (4 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1695: done (3 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1696: done (2 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1697: done (1 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1698: done (0 left) 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1699: write oid 12 current snap is 168 2026-02-21T22:22:45.206 INFO:tasks.rados.rados.0.trial178.stdout:1699: seq_num 601 ranges {501007=416536,1635678=465329} 2026-02-21T22:22:45.211 INFO:tasks.rados.rados.0.trial178.stdout:1699: writing trial17818531-12 from 501007 to 917543 tid 1 2026-02-21T22:22:45.219 INFO:tasks.rados.rados.0.trial178.stdout:1699: writing trial17818531-12 from 1635678 to 2101007 tid 2 2026-02-21T22:22:45.219 INFO:tasks.rados.rados.0.trial178.stdout:1700: write oid 46 current snap is 168 2026-02-21T22:22:45.219 INFO:tasks.rados.rados.0.trial178.stdout:1700: seq_num 602 ranges {517814=491785,1577807=540007} 2026-02-21T22:22:45.226 INFO:tasks.rados.rados.0.trial178.stdout:1700: writing trial17818531-46 from 517814 to 1009599 tid 1 2026-02-21T22:22:45.233 INFO:tasks.rados.rados.0.trial178.stdout:1700: writing trial17818531-46 from 1577807 to 2117814 tid 2 2026-02-21T22:22:45.233 INFO:tasks.rados.rados.0.trial178.stdout:1701: snap_remove snap 166 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1701: done (2 left) 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1702: read oid 21 snap -1 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1702: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1703: write oid 50 current snap is 168 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: finishing write tid 1 to trial17818531-12 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: finishing write tid 2 to trial17818531-12 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: finishing write tid 3 to trial17818531-12 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: finishing write tid 4 to trial17818531-12 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: oid 12 updating version 0 to 512 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: oid 12 version 512 is already newer than 511 2026-02-21T22:22:46.166 INFO:tasks.rados.rados.0.trial178.stdout:1699: oid 12 version 512 is already newer than 510 2026-02-21T22:22:46.167 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 512 (ObjNum 601 snap 168 seq_num 601) dirty exists 2026-02-21T22:22:46.167 INFO:tasks.rados.rados.0.trial178.stdout:1699: left oid 12 (ObjNum 601 snap 168 seq_num 601) 2026-02-21T22:22:46.167 INFO:tasks.rados.rados.0.trial178.stdout:1700: finishing write tid 1 to trial17818531-46 2026-02-21T22:22:46.167 INFO:tasks.rados.rados.0.trial178.stdout:1700: finishing write tid 2 to trial17818531-46 2026-02-21T22:22:46.167 INFO:tasks.rados.rados.0.trial178.stdout:1703: seq_num 603 ranges {534621=567034,1636289=498332} 2026-02-21T22:22:46.175 INFO:tasks.rados.rados.0.trial178.stdout:1703: writing trial17818531-50 from 534621 to 1101655 tid 1 2026-02-21T22:22:46.179 INFO:tasks.rados.rados.0.trial178.stdout:1703: writing trial17818531-50 from 1636289 to 2134621 tid 2 2026-02-21T22:22:46.179 INFO:tasks.rados.rados.0.trial178.stdout:1699: done (3 left) 2026-02-21T22:22:46.179 INFO:tasks.rados.rados.0.trial178.stdout:1704: delete oid 9 current snap is 168 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: finishing write tid 3 to trial17818531-46 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: finishing write tid 4 to trial17818531-46 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: oid 46 updating version 0 to 452 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: oid 46 version 452 is already newer than 451 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: oid 46 updating version 452 to 453 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 453 (ObjNum 602 snap 168 seq_num 602) dirty exists 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1700: left oid 46 (ObjNum 602 snap 168 seq_num 602) 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:1702: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:22:46.180 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:46.341 INFO:tasks.rados.rados.0.trial178.stdout:1703: finishing write tid 1 to trial17818531-50 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: finishing write tid 2 to trial17818531-50 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1700: done (3 left) 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1702: done (2 left) 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1704: done (1 left) 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1705: copy_from oid 24 from oid 6 current snap is 168 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: finishing write tid 3 to trial17818531-50 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: finishing write tid 4 to trial17818531-50 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: oid 50 updating version 0 to 517 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: oid 50 version 517 is already newer than 515 2026-02-21T22:22:46.342 INFO:tasks.rados.rados.0.trial178.stdout:1703: oid 50 version 517 is already newer than 516 2026-02-21T22:22:46.343 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 517 (ObjNum 603 snap 168 seq_num 603) dirty exists 2026-02-21T22:22:46.343 INFO:tasks.rados.rados.0.trial178.stdout:1703: left oid 50 (ObjNum 603 snap 168 seq_num 603) 2026-02-21T22:22:46.343 INFO:tasks.rados.rados.0.trial178.stdout:1703: done (1 left) 2026-02-21T22:22:46.343 INFO:tasks.rados.rados.0.trial178.stdout:1706: delete oid 7 current snap is 168 2026-02-21T22:22:46.344 INFO:tasks.rados.rados.0.trial178.stdout:1706: done (1 left) 2026-02-21T22:22:46.344 INFO:tasks.rados.rados.0.trial178.stdout:1707: read oid 37 snap 164 2026-02-21T22:22:46.344 INFO:tasks.rados.rados.0.trial178.stdout:1707: expect (ObjNum 569 snap 158 seq_num 569) 2026-02-21T22:22:46.344 INFO:tasks.rados.rados.0.trial178.stdout:1708: snap_create 2026-02-21T22:22:46.348 INFO:tasks.rados.rados.0.trial178.stdout:1707: expect (ObjNum 569 snap 158 seq_num 569) 2026-02-21T22:22:46.354 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1409419 2026-02-21T22:22:46.432 INFO:tasks.rados.rados.0.trial178.stdout:1705: finishing copy_from racing read to trial17818531-24 2026-02-21T22:22:46.432 INFO:tasks.rados.rados.0.trial178.stdout:1705: finishing copy_from to trial17818531-24 2026-02-21T22:22:46.432 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 509 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:22:47.210 INFO:tasks.rados.rados.0.trial178.stdout:1705: done (2 left) 2026-02-21T22:22:47.210 INFO:tasks.rados.rados.0.trial178.stdout:1707: done (1 left) 2026-02-21T22:22:47.210 INFO:tasks.rados.rados.0.trial178.stdout:1708: done (0 left) 2026-02-21T22:22:47.210 INFO:tasks.rados.rados.0.trial178.stdout:1709: write (excl) oid 33 current snap is 169 2026-02-21T22:22:47.210 INFO:tasks.rados.rados.0.trial178.stdout:1709: seq_num 606 ranges {585042=792781,2095382=89660} 2026-02-21T22:22:47.219 INFO:tasks.rados.rados.0.trial178.stdout:1709: writing trial17818531-33 from 585042 to 1377823 tid 1 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1709: writing trial17818531-33 from 2095382 to 2185042 tid 2 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1710: read oid 28 snap -1 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1710: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1711: rollback oid 17 current snap is 169 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 165 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1712: rollback oid 6 current snap is 169 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 167 2026-02-21T22:22:47.225 INFO:tasks.rados.rados.0.trial178.stdout:1713: delete oid 1 current snap is 169 2026-02-21T22:22:47.226 INFO:tasks.rados.rados.0.trial178.stdout:1709: finishing write tid 1 to trial17818531-33 2026-02-21T22:22:47.228 INFO:tasks.rados.rados.0.trial178.stdout:1711: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:22:47.228 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:22:47.230 INFO:tasks.rados.rados.0.trial178.stdout:1711: done (4 left) 2026-02-21T22:22:47.230 INFO:tasks.rados.rados.0.trial178.stdout:1713: done (3 left) 2026-02-21T22:22:47.230 INFO:tasks.rados.rados.0.trial178.stdout:1714: snap_create 2026-02-21T22:22:47.230 INFO:tasks.rados.rados.0.trial178.stdout:1712: finishing rollback tid 0 to trial17818531-6 2026-02-21T22:22:47.231 INFO:tasks.rados.rados.0.trial178.stdout:1710: expect (ObjNum 430 snap 123 seq_num 430) 2026-02-21T22:22:47.236 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 707118 2026-02-21T22:22:47.318 INFO:tasks.rados.rados.0.trial178.stdout:1712: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:22:47.318 INFO:tasks.rados.rados.0.trial178.stdout:1712: finishing rollback tid 2 to trial17818531-6 2026-02-21T22:22:47.318 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 758 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:22:47.318 INFO:tasks.rados.rados.0.trial178.stdout:1709: finishing write tid 2 to trial17818531-33 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: finishing write tid 3 to trial17818531-33 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: finishing write tid 4 to trial17818531-33 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: oid 33 updating version 0 to 413 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: oid 33 updating version 413 to 414 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: oid 33 version 414 is already newer than 412 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 414 (ObjNum 606 snap 169 seq_num 606) dirty exists 2026-02-21T22:22:47.319 INFO:tasks.rados.rados.0.trial178.stdout:1709: left oid 33 (ObjNum 606 snap 169 seq_num 606) 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1709: done (3 left) 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1710: done (2 left) 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1712: done (1 left) 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1714: done (0 left) 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1715: rollback oid 10 current snap is 170 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 169 2026-02-21T22:22:48.213 INFO:tasks.rados.rados.0.trial178.stdout:1716: snap_remove snap 164 2026-02-21T22:22:48.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:48.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:49.214 INFO:tasks.rados.rados.0.trial178.stdout:1716: done (1 left) 2026-02-21T22:22:49.214 INFO:tasks.rados.rados.0.trial178.stdout:1717: write (excl) oid 31 current snap is 170 2026-02-21T22:22:49.214 INFO:tasks.rados.rados.0.trial178.stdout:1717: seq_num 608 ranges {618656=543279,1812346=406310} 2026-02-21T22:22:49.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: writing trial17818531-31 from 618656 to 1161935 tid 1 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1717: writing trial17818531-31 from 1812346 to 2218656 tid 2 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1718: copy_from oid 27 from oid 22 current snap is 170 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1715: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1715: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1715: finishing rollback tid 2 to trial17818531-10 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 761 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1717: finishing write tid 1 to trial17818531-31 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1715: done (2 left) 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1719: read oid 30 snap -1 2026-02-21T22:22:49.230 INFO:tasks.rados.rados.0.trial178.stdout:1719: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:49.231 INFO:tasks.rados.rados.0.trial178.stdout:1720: copy_from oid 1 from oid 41 current snap is 170 2026-02-21T22:22:49.231 INFO:tasks.rados.rados.0.trial178.stdout:1721: snap_remove snap 165 2026-02-21T22:22:50.221 INFO:tasks.rados.rados.0.trial178.stdout:1721: done (4 left) 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1722: copy_from oid 43 from oid 7 current snap is 170 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: finishing write tid 2 to trial17818531-31 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: finishing write tid 3 to trial17818531-31 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: finishing write tid 4 to trial17818531-31 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: oid 31 updating version 0 to 624 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: oid 31 updating version 624 to 626 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: oid 31 version 626 is already newer than 625 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 626 (ObjNum 608 snap 170 seq_num 608) dirty exists 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1717: left oid 31 (ObjNum 608 snap 170 seq_num 608) 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:1719: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:22:50.222 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:50.281 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:50.320 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1720: finishing copy_from racing read to trial17818531-1 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1720: finishing copy_from to trial17818531-1 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 710 (ObjNum 581 snap 161 seq_num 581) dirty exists 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1718: finishing copy_from racing read to trial17818531-27 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1718: finishing copy_from to trial17818531-27 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 514 (ObjNum 578 snap 160 seq_num 578) dirty exists 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1722: finishing copy_from to trial17818531-43 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1722: got expected ENOENT (src dne) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1722: finishing copy_from racing read to trial17818531-43 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1717: done (4 left) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1718: done (3 left) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1719: done (2 left) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1720: done (1 left) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1722: done (0 left) 2026-02-21T22:22:50.361 INFO:tasks.rados.rados.0.trial178.stdout:1723: delete oid 17 current snap is 170 2026-02-21T22:22:50.362 INFO:tasks.rados.rados.0.trial178.stdout:1723: done (0 left) 2026-02-21T22:22:50.362 INFO:tasks.rados.rados.0.trial178.stdout:1724: snap_create 2026-02-21T22:22:51.167 INFO:tasks.rados.rados.0.trial178.stdout:1724: done (0 left) 2026-02-21T22:22:51.167 INFO:tasks.rados.rados.0.trial178.stdout:1725: delete oid 31 current snap is 171 2026-02-21T22:22:51.171 INFO:tasks.rados.rados.0.trial178.stdout:1725: done (0 left) 2026-02-21T22:22:51.171 INFO:tasks.rados.rados.0.trial178.stdout:1726: read oid 9 snap 169 2026-02-21T22:22:51.172 INFO:tasks.rados.rados.0.trial178.stdout:1726: expect deleted 2026-02-21T22:22:51.172 INFO:tasks.rados.rados.0.trial178.stdout:1727: write oid 11 current snap is 171 2026-02-21T22:22:51.172 INFO:tasks.rados.rados.0.trial178.stdout:1727: seq_num 611 ranges {669077=485379,1587792=681285} 2026-02-21T22:22:51.179 INFO:tasks.rados.rados.0.trial178.stdout:1727: writing trial17818531-11 from 669077 to 1154456 tid 1 2026-02-21T22:22:51.187 INFO:tasks.rados.rados.0.trial178.stdout:1727: writing trial17818531-11 from 1587792 to 2269077 tid 2 2026-02-21T22:22:51.187 INFO:tasks.rados.rados.0.trial178.stdout:1728: snap_create 2026-02-21T22:22:51.187 INFO:tasks.rados.rados.0.trial178.stdout:1727: finishing write tid 1 to trial17818531-11 2026-02-21T22:22:51.191 INFO:tasks.rados.rados.0.trial178.stdout:1727: finishing write tid 2 to trial17818531-11 2026-02-21T22:22:51.191 INFO:tasks.rados.rados.0.trial178.stdout:1727: finishing write tid 3 to trial17818531-11 2026-02-21T22:22:51.191 INFO:tasks.rados.rados.0.trial178.stdout:1727: finishing write tid 4 to trial17818531-11 2026-02-21T22:22:51.192 INFO:tasks.rados.rados.0.trial178.stdout:1727: oid 11 updating version 0 to 777 2026-02-21T22:22:51.192 INFO:tasks.rados.rados.0.trial178.stdout:1727: oid 11 version 777 is already newer than 775 2026-02-21T22:22:51.192 INFO:tasks.rados.rados.0.trial178.stdout:1727: oid 11 version 777 is already newer than 776 2026-02-21T22:22:51.192 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 777 (ObjNum 611 snap 171 seq_num 611) dirty exists 2026-02-21T22:22:51.192 INFO:tasks.rados.rados.0.trial178.stdout:1727: left oid 11 (ObjNum 611 snap 171 seq_num 611) 2026-02-21T22:22:52.171 INFO:tasks.rados.rados.0.trial178.stdout:1726: done (2 left) 2026-02-21T22:22:52.171 INFO:tasks.rados.rados.0.trial178.stdout:1727: done (1 left) 2026-02-21T22:22:52.171 INFO:tasks.rados.rados.0.trial178.stdout:1728: done (0 left) 2026-02-21T22:22:52.171 INFO:tasks.rados.rados.0.trial178.stdout:1729: write oid 8 current snap is 172 2026-02-21T22:22:52.171 INFO:tasks.rados.rados.0.trial178.stdout:1729: seq_num 612 ranges {685884=560628,2046274=239610} 2026-02-21T22:22:52.179 INFO:tasks.rados.rados.0.trial178.stdout:1729: writing trial17818531-8 from 685884 to 1246512 tid 1 2026-02-21T22:22:52.187 INFO:tasks.rados.rados.0.trial178.stdout:1729: writing trial17818531-8 from 2046274 to 2285884 tid 2 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1730: read oid 49 snap -1 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1730: expect (ObjNum 486 snap 135 seq_num 486) 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1731: read oid 18 snap -1 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1731: expect deleted 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1732: write oid 34 current snap is 172 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1729: finishing write tid 1 to trial17818531-8 2026-02-21T22:22:52.188 INFO:tasks.rados.rados.0.trial178.stdout:1732: seq_num 613 ranges {702691=635877,2104756=197935} 2026-02-21T22:22:52.196 INFO:tasks.rados.rados.0.trial178.stdout:1732: writing trial17818531-34 from 702691 to 1338568 tid 1 2026-02-21T22:22:52.203 INFO:tasks.rados.rados.0.trial178.stdout:1732: writing trial17818531-34 from 2104756 to 2302691 tid 2 2026-02-21T22:22:52.203 INFO:tasks.rados.rados.0.trial178.stdout:1733: write oid 17 current snap is 172 2026-02-21T22:22:52.203 INFO:tasks.rados.rados.0.trial178.stdout:1733: seq_num 614 ranges {719498=711126,2046885=272613} 2026-02-21T22:22:52.212 INFO:tasks.rados.rados.0.trial178.stdout:1733: writing trial17818531-17 from 719498 to 1430624 tid 1 2026-02-21T22:22:52.218 INFO:tasks.rados.rados.0.trial178.stdout:1733: writing trial17818531-17 from 2046885 to 2319498 tid 2 2026-02-21T22:22:52.218 INFO:tasks.rados.rados.0.trial178.stdout:1734: snap_create 2026-02-21T22:22:52.218 INFO:tasks.rados.rados.0.trial178.stdout:1729: finishing write tid 2 to trial17818531-8 2026-02-21T22:22:52.218 INFO:tasks.rados.rados.0.trial178.stdout:1729: finishing write tid 3 to trial17818531-8 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1729: finishing write tid 4 to trial17818531-8 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1729: oid 8 updating version 0 to 950 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1729: oid 8 version 950 is already newer than 949 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1729: oid 8 updating version 950 to 951 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 951 (ObjNum 612 snap 172 seq_num 612) dirty exists 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1729: left oid 8 (ObjNum 612 snap 172 seq_num 612) 2026-02-21T22:22:52.219 INFO:tasks.rados.rados.0.trial178.stdout:1730: expect (ObjNum 486 snap 135 seq_num 486) 2026-02-21T22:22:52.282 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:52.304 INFO:tasks.rados.rados.0.trial178.stdout:1732: finishing write tid 1 to trial17818531-34 2026-02-21T22:22:52.304 INFO:tasks.rados.rados.0.trial178.stdout:1732: finishing write tid 2 to trial17818531-34 2026-02-21T22:22:52.304 INFO:tasks.rados.rados.0.trial178.stdout:1732: finishing write tid 3 to trial17818531-34 2026-02-21T22:22:52.304 INFO:tasks.rados.rados.0.trial178.stdout:1732: finishing write tid 4 to trial17818531-34 2026-02-21T22:22:52.304 INFO:tasks.rados.rados.0.trial178.stdout:1732: oid 34 updating version 0 to 753 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1732: oid 34 version 753 is already newer than 752 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1732: oid 34 updating version 753 to 754 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 754 (ObjNum 613 snap 172 seq_num 613) dirty exists 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1732: left oid 34 (ObjNum 613 snap 172 seq_num 613) 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: finishing write tid 1 to trial17818531-17 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: finishing write tid 2 to trial17818531-17 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: finishing write tid 3 to trial17818531-17 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: finishing write tid 4 to trial17818531-17 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: oid 17 updating version 0 to 715 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: oid 17 updating version 715 to 716 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:1733: oid 17 version 716 is already newer than 714 2026-02-21T22:22:52.305 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 716 (ObjNum 614 snap 172 seq_num 614) dirty exists 2026-02-21T22:22:52.306 INFO:tasks.rados.rados.0.trial178.stdout:1733: left oid 17 (ObjNum 614 snap 172 seq_num 614) 2026-02-21T22:22:53.172 INFO:tasks.rados.rados.0.trial178.stdout:1729: done (5 left) 2026-02-21T22:22:53.172 INFO:tasks.rados.rados.0.trial178.stdout:1730: done (4 left) 2026-02-21T22:22:53.172 INFO:tasks.rados.rados.0.trial178.stdout:1731: done (3 left) 2026-02-21T22:22:53.173 INFO:tasks.rados.rados.0.trial178.stdout:1732: done (2 left) 2026-02-21T22:22:53.173 INFO:tasks.rados.rados.0.trial178.stdout:1733: done (1 left) 2026-02-21T22:22:53.173 INFO:tasks.rados.rados.0.trial178.stdout:1734: done (0 left) 2026-02-21T22:22:53.173 INFO:tasks.rados.rados.0.trial178.stdout:1735: write oid 21 current snap is 173 2026-02-21T22:22:53.173 INFO:tasks.rados.rados.0.trial178.stdout:1735: seq_num 615 ranges {736305=786375,2105367=230938} 2026-02-21T22:22:53.178 INFO:tasks.rados.rados.0.trial178.stdout:1735: writing trial17818531-21 from 736305 to 1522680 tid 1 2026-02-21T22:22:53.182 INFO:tasks.rados.rados.0.trial178.stdout:1735: writing trial17818531-21 from 2105367 to 2336305 tid 2 2026-02-21T22:22:53.182 INFO:tasks.rados.rados.0.trial178.stdout:1736: copy_from oid 12 from oid 30 current snap is 173 2026-02-21T22:22:53.182 INFO:tasks.rados.rados.0.trial178.stdout:1737: copy_from oid 22 from oid 46 current snap is 173 2026-02-21T22:22:53.182 INFO:tasks.rados.rados.0.trial178.stdout:1738: snap_create 2026-02-21T22:22:53.186 INFO:tasks.rados.rados.0.trial178.stdout:1735: finishing write tid 1 to trial17818531-21 2026-02-21T22:22:53.186 INFO:tasks.rados.rados.0.trial178.stdout:1735: finishing write tid 2 to trial17818531-21 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: finishing write tid 3 to trial17818531-21 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: finishing write tid 4 to trial17818531-21 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: oid 21 updating version 0 to 955 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: oid 21 version 955 is already newer than 953 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: oid 21 version 955 is already newer than 954 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 955 (ObjNum 615 snap 173 seq_num 615) dirty exists 2026-02-21T22:22:53.187 INFO:tasks.rados.rados.0.trial178.stdout:1735: left oid 21 (ObjNum 615 snap 173 seq_num 615) 2026-02-21T22:22:53.192 INFO:tasks.rados.rados.0.trial178.stdout:1736: finishing copy_from racing read to trial17818531-12 2026-02-21T22:22:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1736: finishing copy_from to trial17818531-12 2026-02-21T22:22:53.193 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 498 (ObjNum 544 snap 152 seq_num 544) dirty exists 2026-02-21T22:22:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1737: finishing copy_from racing read to trial17818531-22 2026-02-21T22:22:53.194 INFO:tasks.rados.rados.0.trial178.stdout:1737: finishing copy_from to trial17818531-22 2026-02-21T22:22:53.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 453 (ObjNum 602 snap 168 seq_num 602) dirty exists 2026-02-21T22:22:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:22:53.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:22:53.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:53.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:54.176 INFO:tasks.rados.rados.0.trial178.stdout:1735: done (3 left) 2026-02-21T22:22:54.176 INFO:tasks.rados.rados.0.trial178.stdout:1736: done (2 left) 2026-02-21T22:22:54.176 INFO:tasks.rados.rados.0.trial178.stdout:1737: done (1 left) 2026-02-21T22:22:54.177 INFO:tasks.rados.rados.0.trial178.stdout:1738: done (0 left) 2026-02-21T22:22:54.177 INFO:tasks.rados.rados.0.trial178.stdout:1739: copy_from oid 35 from oid 14 current snap is 174 2026-02-21T22:22:54.177 INFO:tasks.rados.rados.0.trial178.stdout:1740: snap_create 2026-02-21T22:22:54.187 INFO:tasks.rados.rados.0.trial178.stdout:1739: finishing copy_from racing read to trial17818531-35 2026-02-21T22:22:54.187 INFO:tasks.rados.rados.0.trial178.stdout:1739: finishing copy_from to trial17818531-35 2026-02-21T22:22:54.188 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 759 (ObjNum 598 snap 167 seq_num 598) dirty exists 2026-02-21T22:22:55.178 INFO:tasks.rados.rados.0.trial178.stdout:1739: done (1 left) 2026-02-21T22:22:55.178 INFO:tasks.rados.rados.0.trial178.stdout:1740: done (0 left) 2026-02-21T22:22:55.178 INFO:tasks.rados.rados.0.trial178.stdout:1741: write (excl) oid 48 current snap is 175 2026-02-21T22:22:55.178 INFO:tasks.rados.rados.0.trial178.stdout:1741: seq_num 616 ranges {753112=577977,1880202=472910} 2026-02-21T22:22:55.186 INFO:tasks.rados.rados.0.trial178.stdout:1741: writing trial17818531-48 from 753112 to 1331089 tid 1 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:1741: writing trial17818531-48 from 1880202 to 2353112 tid 2 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:1742: rollback oid 17 current snap is 175 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 170 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:1741: finishing write tid 1 to trial17818531-48 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:1743: rollback oid 32 current snap is 175 2026-02-21T22:22:55.193 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 170 2026-02-21T22:22:55.194 INFO:tasks.rados.rados.0.trial178.stdout:1744: read oid 3 snap -1 2026-02-21T22:22:55.194 INFO:tasks.rados.rados.0.trial178.stdout:1744: expect deleted 2026-02-21T22:22:55.194 INFO:tasks.rados.rados.0.trial178.stdout:1745: snap_remove snap 174 2026-02-21T22:22:56.168 INFO:tasks.rados.rados.0.trial178.stdout:1745: done (4 left) 2026-02-21T22:22:56.168 INFO:tasks.rados.rados.0.trial178.stdout:1746: read oid 29 snap -1 2026-02-21T22:22:56.168 INFO:tasks.rados.rados.0.trial178.stdout:1746: expect (ObjNum 592 snap 163 seq_num 592) 2026-02-21T22:22:56.168 INFO:tasks.rados.rados.0.trial178.stdout:1747: write (excl) oid 47 current snap is 175 2026-02-21T22:22:56.168 INFO:tasks.rados.rados.0.trial178.stdout:1747: seq_num 617 ranges {769919=653226,1938684=431235} 2026-02-21T22:22:56.177 INFO:tasks.rados.rados.0.trial178.stdout:1747: writing trial17818531-47 from 769919 to 1423145 tid 1 2026-02-21T22:22:56.184 INFO:tasks.rados.rados.0.trial178.stdout:1747: writing trial17818531-47 from 1938684 to 2369919 tid 2 2026-02-21T22:22:56.184 INFO:tasks.rados.rados.0.trial178.stdout:1748: rollback oid 33 current snap is 175 2026-02-21T22:22:56.184 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 167 2026-02-21T22:22:56.184 INFO:tasks.rados.rados.0.trial178.stdout:1741: finishing write tid 2 to trial17818531-48 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: finishing write tid 3 to trial17818531-48 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: finishing write tid 4 to trial17818531-48 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: oid 48 updating version 0 to 713 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: oid 48 version 713 is already newer than 712 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: oid 48 version 713 is already newer than 711 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 713 (ObjNum 616 snap 175 seq_num 616) dirty exists 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: left oid 48 (ObjNum 616 snap 175 seq_num 616) 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1743: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1743: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1741: done (6 left) 2026-02-21T22:22:56.185 INFO:tasks.rados.rados.0.trial178.stdout:1744: done (5 left) 2026-02-21T22:22:56.186 INFO:tasks.rados.rados.0.trial178.stdout:1749: write (excl) oid 18 current snap is 175 2026-02-21T22:22:56.186 INFO:tasks.rados.rados.0.trial178.stdout:1749: seq_num 618 ranges {786726=728475,2280813=105913} 2026-02-21T22:22:56.194 INFO:tasks.rados.rados.0.trial178.stdout:1749: writing trial17818531-18 from 786726 to 1515201 tid 1 2026-02-21T22:22:56.200 INFO:tasks.rados.rados.0.trial178.stdout:1749: writing trial17818531-18 from 2280813 to 2386726 tid 2 2026-02-21T22:22:56.200 INFO:tasks.rados.rados.0.trial178.stdout:1750: snap_create 2026-02-21T22:22:56.200 INFO:tasks.rados.rados.0.trial178.stdout:1743: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:22:56.200 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 783 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:22:56.201 INFO:tasks.rados.rados.0.trial178.stdout:1742: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:22:56.201 INFO:tasks.rados.rados.0.trial178.stdout:1742: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:22:56.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 718 (ObjNum 241 snap 0 seq_num 99645360) dirty dne 2026-02-21T22:22:56.201 INFO:tasks.rados.rados.0.trial178.stdout:1746: expect (ObjNum 592 snap 163 seq_num 592) 2026-02-21T22:22:56.212 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2948466 2026-02-21T22:22:56.232 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:56.267 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:22:56.297 INFO:tasks.rados.rados.0.trial178.stdout:1747: finishing write tid 1 to trial17818531-47 2026-02-21T22:22:56.297 INFO:tasks.rados.rados.0.trial178.stdout:1747: finishing write tid 2 to trial17818531-47 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: finishing write tid 3 to trial17818531-47 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: finishing write tid 4 to trial17818531-47 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: oid 47 updating version 0 to 958 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: oid 47 version 958 is already newer than 957 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: oid 47 updating version 958 to 959 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 959 (ObjNum 617 snap 175 seq_num 617) dirty exists 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1747: left oid 47 (ObjNum 617 snap 175 seq_num 617) 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1748: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1748: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:1748: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:22:56.298 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 417 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: finishing write tid 1 to trial17818531-18 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: finishing write tid 2 to trial17818531-18 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: finishing write tid 3 to trial17818531-18 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: finishing write tid 4 to trial17818531-18 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: oid 18 updating version 0 to 741 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: oid 18 version 741 is already newer than 739 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: oid 18 version 741 is already newer than 740 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 741 (ObjNum 618 snap 175 seq_num 618) dirty exists 2026-02-21T22:22:56.299 INFO:tasks.rados.rados.0.trial178.stdout:1749: left oid 18 (ObjNum 618 snap 175 seq_num 618) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1742: done (6 left) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1743: done (5 left) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1746: done (4 left) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1747: done (3 left) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1748: done (2 left) 2026-02-21T22:22:57.184 INFO:tasks.rados.rados.0.trial178.stdout:1749: done (1 left) 2026-02-21T22:22:57.185 INFO:tasks.rados.rados.0.trial178.stdout:1750: done (0 left) 2026-02-21T22:22:57.185 INFO:tasks.rados.rados.0.trial178.stdout:1751: delete oid 31 current snap is 176 2026-02-21T22:22:57.188 INFO:tasks.rados.rados.0.trial178.stdout:1751: done (0 left) 2026-02-21T22:22:57.188 INFO:tasks.rados.rados.0.trial178.stdout:1752: write (excl) oid 50 current snap is 176 2026-02-21T22:22:57.188 INFO:tasks.rados.rados.0.trial178.stdout:1752: seq_num 620 ranges {420340=478973,1597777=512212} 2026-02-21T22:22:57.194 INFO:tasks.rados.rados.0.trial178.stdout:1752: writing trial17818531-50 from 420340 to 899313 tid 1 2026-02-21T22:22:57.203 INFO:tasks.rados.rados.0.trial178.stdout:1752: writing trial17818531-50 from 1597777 to 2109989 tid 2 2026-02-21T22:22:57.203 INFO:tasks.rados.rados.0.trial178.stdout:1753: copy_from oid 5 from oid 38 current snap is 176 2026-02-21T22:22:57.203 INFO:tasks.rados.rados.0.trial178.stdout:1752: finishing write tid 1 to trial17818531-50 2026-02-21T22:22:57.203 INFO:tasks.rados.rados.0.trial178.stdout:1754: snap_create 2026-02-21T22:22:57.210 INFO:tasks.rados.rados.0.trial178.stdout:1752: finishing write tid 2 to trial17818531-50 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: finishing write tid 3 to trial17818531-50 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: finishing write tid 4 to trial17818531-50 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: oid 50 updating version 0 to 520 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: oid 50 version 520 is already newer than 519 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: oid 50 version 520 is already newer than 518 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 520 (ObjNum 620 snap 176 seq_num 620) dirty exists 2026-02-21T22:22:57.211 INFO:tasks.rados.rados.0.trial178.stdout:1752: left oid 50 (ObjNum 620 snap 176 seq_num 620) 2026-02-21T22:22:57.214 INFO:tasks.rados.rados.0.trial178.stdout:1753: finishing copy_from racing read to trial17818531-5 2026-02-21T22:22:57.217 INFO:tasks.rados.rados.0.trial178.stdout:1753: finishing copy_from to trial17818531-5 2026-02-21T22:22:57.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 411 (ObjNum 575 snap 158 seq_num 575) dirty exists 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:1752: done (2 left) 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:1753: done (1 left) 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:1754: done (0 left) 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:1755: rollback oid 33 current snap is 177 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 167 2026-02-21T22:22:58.186 INFO:tasks.rados.rados.0.trial178.stdout:1756: snap_remove snap 169 2026-02-21T22:22:58.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:22:58.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:1756: done (1 left) 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:1757: rollback oid 29 current snap is 177 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 172 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:1755: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:1755: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:1755: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:22:59.187 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 420 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:22:59.188 INFO:tasks.rados.rados.0.trial178.stdout:1755: done (1 left) 2026-02-21T22:22:59.188 INFO:tasks.rados.rados.0.trial178.stdout:1758: delete oid 28 current snap is 177 2026-02-21T22:22:59.191 INFO:tasks.rados.rados.0.trial178.stdout:1757: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:22:59.191 INFO:tasks.rados.rados.0.trial178.stdout:1757: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:22:59.191 INFO:tasks.rados.rados.0.trial178.stdout:1757: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:22:59.191 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 757 (ObjNum 592 snap 163 seq_num 592) dirty exists 2026-02-21T22:22:59.192 INFO:tasks.rados.rados.0.trial178.stdout:1757: done (1 left) 2026-02-21T22:22:59.192 INFO:tasks.rados.rados.0.trial178.stdout:1758: done (0 left) 2026-02-21T22:22:59.192 INFO:tasks.rados.rados.0.trial178.stdout:1759: write oid 1 current snap is 177 2026-02-21T22:22:59.192 INFO:tasks.rados.rados.0.trial178.stdout:1759: seq_num 622 ranges {453954=629471,1598388=515881} 2026-02-21T22:22:59.200 INFO:tasks.rados.rados.0.trial178.stdout:1759: writing trial17818531-1 from 453954 to 1083425 tid 1 2026-02-21T22:22:59.206 INFO:tasks.rados.rados.0.trial178.stdout:1759: writing trial17818531-1 from 1598388 to 2114269 tid 2 2026-02-21T22:22:59.207 INFO:tasks.rados.rados.0.trial178.stdout:1760: snap_create 2026-02-21T22:22:59.207 INFO:tasks.rados.rados.0.trial178.stdout:1759: finishing write tid 1 to trial17818531-1 2026-02-21T22:22:59.211 INFO:tasks.rados.rados.0.trial178.stdout:1759: finishing write tid 2 to trial17818531-1 2026-02-21T22:22:59.212 INFO:tasks.rados.rados.0.trial178.stdout:1759: finishing write tid 3 to trial17818531-1 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:1759: finishing write tid 4 to trial17818531-1 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:1759: oid 1 updating version 0 to 714 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:1759: oid 1 updating version 714 to 716 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:1759: oid 1 version 716 is already newer than 715 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 716 (ObjNum 622 snap 177 seq_num 622) dirty exists 2026-02-21T22:22:59.213 INFO:tasks.rados.rados.0.trial178.stdout:1759: left oid 1 (ObjNum 622 snap 177 seq_num 622) 2026-02-21T22:23:00.192 INFO:tasks.rados.rados.0.trial178.stdout:1759: done (1 left) 2026-02-21T22:23:00.192 INFO:tasks.rados.rados.0.trial178.stdout:1760: done (0 left) 2026-02-21T22:23:00.192 INFO:tasks.rados.rados.0.trial178.stdout:1761: snap_remove snap 177 2026-02-21T22:23:01.169 INFO:tasks.rados.rados.0.trial178.stdout:1761: done (0 left) 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1762: read oid 5 snap -1 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1762: expect (ObjNum 575 snap 158 seq_num 575) 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1763: read oid 18 snap -1 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1763: expect (ObjNum 618 snap 175 seq_num 618) 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1764: rollback oid 3 current snap is 178 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 175 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1765: write (excl) oid 22 current snap is 178 2026-02-21T22:23:01.170 INFO:tasks.rados.rados.0.trial178.stdout:1765: seq_num 623 ranges {470761=704720,1656870=659539} 2026-02-21T22:23:01.177 INFO:tasks.rados.rados.0.trial178.stdout:1765: writing trial17818531-22 from 470761 to 1175481 tid 1 2026-02-21T22:23:01.185 INFO:tasks.rados.rados.0.trial178.stdout:1765: writing trial17818531-22 from 1656870 to 2316409 tid 2 2026-02-21T22:23:01.185 INFO:tasks.rados.rados.0.trial178.stdout:1766: read oid 2 snap 173 2026-02-21T22:23:01.185 INFO:tasks.rados.rados.0.trial178.stdout:1766: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:23:01.185 INFO:tasks.rados.rados.0.trial178.stdout:1767: snap_remove snap 170 2026-02-21T22:23:02.170 INFO:tasks.rados.rados.0.trial178.stdout:1767: done (5 left) 2026-02-21T22:23:02.170 INFO:tasks.rados.rados.0.trial178.stdout:1768: delete oid 39 current snap is 178 2026-02-21T22:23:02.171 INFO:tasks.rados.rados.0.trial178.stdout:1764: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:23:02.171 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 0 (ObjNum 33 snap 0 seq_num 99795152) dirty dne 2026-02-21T22:23:02.171 INFO:tasks.rados.rados.0.trial178.stdout:1762: expect (ObjNum 575 snap 158 seq_num 575) 2026-02-21T22:23:02.324 INFO:tasks.rados.rados.0.trial178.stdout:1763: expect (ObjNum 618 snap 175 seq_num 618) 2026-02-21T22:23:02.346 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1546660 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: finishing write tid 1 to trial17818531-22 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: finishing write tid 2 to trial17818531-22 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: finishing write tid 3 to trial17818531-22 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: finishing write tid 4 to trial17818531-22 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: oid 22 updating version 0 to 523 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: oid 22 version 523 is already newer than 522 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: oid 22 version 523 is already newer than 521 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 523 (ObjNum 623 snap 178 seq_num 623) dirty exists 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1765: left oid 22 (ObjNum 623 snap 178 seq_num 623) 2026-02-21T22:23:02.370 INFO:tasks.rados.rados.0.trial178.stdout:1766: expect (ObjNum 544 snap 152 seq_num 544) 2026-02-21T22:23:02.441 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 641733 2026-02-21T22:23:02.477 INFO:tasks.rados.rados.0.trial178.stdout:1762: done (5 left) 2026-02-21T22:23:02.477 INFO:tasks.rados.rados.0.trial178.stdout:1763: done (4 left) 2026-02-21T22:23:02.477 INFO:tasks.rados.rados.0.trial178.stdout:1764: done (3 left) 2026-02-21T22:23:02.477 INFO:tasks.rados.rados.0.trial178.stdout:1765: done (2 left) 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1766: done (1 left) 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1768: done (0 left) 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1769: rollback oid 3 current snap is 178 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 173 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1770: read oid 10 snap -1 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1770: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1771: write (excl) oid 39 current snap is 178 2026-02-21T22:23:02.478 INFO:tasks.rados.rados.0.trial178.stdout:1771: seq_num 625 ranges {504375=571571,1490187=663208} 2026-02-21T22:23:02.483 INFO:tasks.rados.rados.0.trial178.stdout:1771: writing trial17818531-39 from 504375 to 1075946 tid 1 2026-02-21T22:23:02.490 INFO:tasks.rados.rados.0.trial178.stdout:1771: writing trial17818531-39 from 1490187 to 2153395 tid 2 2026-02-21T22:23:02.491 INFO:tasks.rados.rados.0.trial178.stdout:1772: rollback oid 32 current snap is 178 2026-02-21T22:23:02.491 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 162 2026-02-21T22:23:02.491 INFO:tasks.rados.rados.0.trial178.stdout:1769: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:23:02.491 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 0 (ObjNum 33 snap 0 seq_num 99795152) dirty dne 2026-02-21T22:23:02.491 INFO:tasks.rados.rados.0.trial178.stdout:1770: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:02.526 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:02.555 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: finishing write tid 1 to trial17818531-39 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1772: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1772: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: finishing write tid 2 to trial17818531-39 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: finishing write tid 3 to trial17818531-39 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1772: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 802 (ObjNum 338 snap 95 seq_num 338) dirty exists 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: finishing write tid 4 to trial17818531-39 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: oid 39 updating version 0 to 766 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: oid 39 version 766 is already newer than 764 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:1771: oid 39 version 766 is already newer than 765 2026-02-21T22:23:02.580 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 766 (ObjNum 625 snap 178 seq_num 625) dirty exists 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1771: left oid 39 (ObjNum 625 snap 178 seq_num 625) 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1769: done (3 left) 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1770: done (2 left) 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1771: done (1 left) 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1772: done (0 left) 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1773: copy_from oid 32 from oid 9 current snap is 178 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1774: write (excl) oid 40 current snap is 178 2026-02-21T22:23:02.581 INFO:tasks.rados.rados.0.trial178.stdout:1774: seq_num 626 ranges {521182=646820,1948669=523219} 2026-02-21T22:23:02.588 INFO:tasks.rados.rados.0.trial178.stdout:1774: writing trial17818531-40 from 521182 to 1168002 tid 1 2026-02-21T22:23:02.597 INFO:tasks.rados.rados.0.trial178.stdout:1774: writing trial17818531-40 from 1948669 to 2471888 tid 2 2026-02-21T22:23:02.597 INFO:tasks.rados.rados.0.trial178.stdout:1775: snap_remove snap 173 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1775: done (2 left) 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1776: snap_create 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1773: finishing copy_from to trial17818531-32 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1773: got expected ENOENT (src dne) 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1773: finishing copy_from racing read to trial17818531-32 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1774: finishing write tid 1 to trial17818531-40 2026-02-21T22:23:03.425 INFO:tasks.rados.rados.0.trial178.stdout:1774: finishing write tid 2 to trial17818531-40 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: finishing write tid 3 to trial17818531-40 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: finishing write tid 4 to trial17818531-40 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: oid 40 updating version 0 to 526 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: oid 40 version 526 is already newer than 524 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: oid 40 version 526 is already newer than 525 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 526 (ObjNum 626 snap 178 seq_num 626) dirty exists 2026-02-21T22:23:03.426 INFO:tasks.rados.rados.0.trial178.stdout:1774: left oid 40 (ObjNum 626 snap 178 seq_num 626) 2026-02-21T22:23:03.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:03.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:04.428 INFO:tasks.rados.rados.0.trial178.stdout:1773: done (2 left) 2026-02-21T22:23:04.428 INFO:tasks.rados.rados.0.trial178.stdout:1774: done (1 left) 2026-02-21T22:23:04.428 INFO:tasks.rados.rados.0.trial178.stdout:1776: done (0 left) 2026-02-21T22:23:04.428 INFO:tasks.rados.rados.0.trial178.stdout:1777: read oid 34 snap 162 2026-02-21T22:23:04.428 INFO:tasks.rados.rados.0.trial178.stdout:1777: expect (ObjNum 497 snap 139 seq_num 497) 2026-02-21T22:23:04.429 INFO:tasks.rados.rados.0.trial178.stdout:1778: rollback oid 5 current snap is 179 2026-02-21T22:23:04.429 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 167 2026-02-21T22:23:04.429 INFO:tasks.rados.rados.0.trial178.stdout:1779: write oid 19 current snap is 179 2026-02-21T22:23:04.429 INFO:tasks.rados.rados.0.trial178.stdout:1779: seq_num 627 ranges {537989=722069,1890798=647191} 2026-02-21T22:23:04.436 INFO:tasks.rados.rados.0.trial178.stdout:1779: writing trial17818531-19 from 537989 to 1260058 tid 1 2026-02-21T22:23:04.444 INFO:tasks.rados.rados.0.trial178.stdout:1779: writing trial17818531-19 from 1890798 to 2537989 tid 2 2026-02-21T22:23:04.444 INFO:tasks.rados.rados.0.trial178.stdout:1780: write oid 48 current snap is 179 2026-02-21T22:23:04.444 INFO:tasks.rados.rados.0.trial178.stdout:1780: seq_num 628 ranges {554796=797318,1949280=526888} 2026-02-21T22:23:04.453 INFO:tasks.rados.rados.0.trial178.stdout:1780: writing trial17818531-48 from 554796 to 1352114 tid 1 2026-02-21T22:23:04.461 INFO:tasks.rados.rados.0.trial178.stdout:1780: writing trial17818531-48 from 1949280 to 2476168 tid 2 2026-02-21T22:23:04.461 INFO:tasks.rados.rados.0.trial178.stdout:1781: read oid 17 snap -1 2026-02-21T22:23:04.461 INFO:tasks.rados.rados.0.trial178.stdout:1781: expect deleted 2026-02-21T22:23:04.461 INFO:tasks.rados.rados.0.trial178.stdout:1782: snap_remove snap 171 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:1782: done (5 left) 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:1783: copy_from oid 20 from oid 38 current snap is 179 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:1778: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:1778: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 743 (ObjNum 145 snap 0 seq_num 99818864) dirty dne 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:1777: expect (ObjNum 497 snap 139 seq_num 497) 2026-02-21T22:23:05.431 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:05.496 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 422793 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: finishing write tid 1 to trial17818531-19 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: finishing write tid 2 to trial17818531-19 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: finishing write tid 3 to trial17818531-19 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: finishing write tid 4 to trial17818531-19 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: oid 19 updating version 0 to 975 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: oid 19 updating version 975 to 976 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: oid 19 updating version 976 to 977 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 977 (ObjNum 627 snap 179 seq_num 627) dirty exists 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: left oid 19 (ObjNum 627 snap 179 seq_num 627) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1780: finishing write tid 1 to trial17818531-48 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1780: finishing write tid 2 to trial17818531-48 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1777: done (5 left) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1778: done (4 left) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1779: done (3 left) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1781: done (2 left) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1784: read oid 26 snap -1 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1784: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:23:05.519 INFO:tasks.rados.rados.0.trial178.stdout:1785: snap_remove snap 175 2026-02-21T22:23:06.171 INFO:tasks.rados.rados.0.trial178.stdout:1785: done (3 left) 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1786: copy_from oid 18 from oid 46 current snap is 179 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: finishing write tid 3 to trial17818531-48 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: finishing write tid 4 to trial17818531-48 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: oid 48 updating version 0 to 717 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: oid 48 updating version 717 to 719 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: oid 48 version 719 is already newer than 718 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 719 (ObjNum 628 snap 179 seq_num 628) dirty exists 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1780: left oid 48 (ObjNum 628 snap 179 seq_num 628) 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1783: finishing copy_from racing read to trial17818531-20 2026-02-21T22:23:06.172 INFO:tasks.rados.rados.0.trial178.stdout:1783: finishing copy_from to trial17818531-20 2026-02-21T22:23:06.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 411 (ObjNum 575 snap 158 seq_num 575) dirty exists 2026-02-21T22:23:06.173 INFO:tasks.rados.rados.0.trial178.stdout:1784: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:23:06.173 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:06.224 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:06.277 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 380014 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1786: finishing copy_from racing read to trial17818531-18 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1786: finishing copy_from to trial17818531-18 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 453 (ObjNum 602 snap 168 seq_num 602) dirty exists 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1780: done (3 left) 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1783: done (2 left) 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1784: done (1 left) 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1786: done (0 left) 2026-02-21T22:23:06.329 INFO:tasks.rados.rados.0.trial178.stdout:1787: read oid 5 snap -1 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1787: expect deleted 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1788: read oid 28 snap -1 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1788: expect deleted 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1789: read oid 21 snap 168 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1789: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:23:06.330 INFO:tasks.rados.rados.0.trial178.stdout:1790: delete oid 41 current snap is 179 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1787: done (3 left) 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1788: done (2 left) 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1790: done (1 left) 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1791: copy_from oid 40 from oid 6 current snap is 179 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1792: read oid 33 snap 162 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1792: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:06.331 INFO:tasks.rados.rados.0.trial178.stdout:1793: write oid 37 current snap is 179 2026-02-21T22:23:06.332 INFO:tasks.rados.rados.0.trial178.stdout:1793: seq_num 630 ranges {588410=547816,1666244=414204,2484603=103807} 2026-02-21T22:23:06.338 INFO:tasks.rados.rados.0.trial178.stdout:1793: writing trial17818531-37 from 588410 to 1136226 tid 1 2026-02-21T22:23:06.344 INFO:tasks.rados.rados.0.trial178.stdout:1793: writing trial17818531-37 from 1666244 to 2080448 tid 2 2026-02-21T22:23:06.348 INFO:tasks.rados.rados.0.trial178.stdout:1793: writing trial17818531-37 from 2484603 to 2588410 tid 3 2026-02-21T22:23:06.348 INFO:tasks.rados.rados.0.trial178.stdout:1794: snap_remove snap 176 2026-02-21T22:23:07.172 INFO:tasks.rados.rados.0.trial178.stdout:1794: done (4 left) 2026-02-21T22:23:07.172 INFO:tasks.rados.rados.0.trial178.stdout:1795: rollback oid 28 current snap is 179 2026-02-21T22:23:07.173 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 172 2026-02-21T22:23:07.173 INFO:tasks.rados.rados.0.trial178.stdout:1789: expect (ObjNum 471 snap 133 seq_num 471) 2026-02-21T22:23:07.284 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1854454 2026-02-21T22:23:07.309 INFO:tasks.rados.rados.0.trial178.stdout:1792: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:07.310 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:07.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:07.386 INFO:tasks.rados.rados.0.trial178.stdout:1793: finishing write tid 1 to trial17818531-37 2026-02-21T22:23:07.386 INFO:tasks.rados.rados.0.trial178.stdout:1791: finishing copy_from racing read to trial17818531-40 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1791: finishing copy_from to trial17818531-40 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 758 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1793: finishing write tid 2 to trial17818531-37 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1793: finishing write tid 3 to trial17818531-37 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1793: finishing write tid 4 to trial17818531-37 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1789: done (4 left) 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1791: done (3 left) 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1792: done (2 left) 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1796: delete oid 22 current snap is 179 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1793: finishing write tid 5 to trial17818531-37 2026-02-21T22:23:07.387 INFO:tasks.rados.rados.0.trial178.stdout:1793: oid 37 updating version 0 to 723 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1793: oid 37 version 723 is already newer than 721 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1793: oid 37 version 723 is already newer than 720 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1793: oid 37 version 723 is already newer than 722 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 723 (ObjNum 630 snap 179 seq_num 630) dirty exists 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1793: left oid 37 (ObjNum 630 snap 179 seq_num 630) 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1795: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:1795: finishing rollback tid 2 to trial17818531-28 2026-02-21T22:23:07.388 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 423 (ObjNum 430 snap 123 seq_num 430) dirty exists 2026-02-21T22:23:07.389 INFO:tasks.rados.rados.0.trial178.stdout:1793: done (2 left) 2026-02-21T22:23:07.389 INFO:tasks.rados.rados.0.trial178.stdout:1795: done (1 left) 2026-02-21T22:23:07.389 INFO:tasks.rados.rados.0.trial178.stdout:1796: done (0 left) 2026-02-21T22:23:07.389 INFO:tasks.rados.rados.0.trial178.stdout:1797: write (excl) oid 50 current snap is 179 2026-02-21T22:23:07.389 INFO:tasks.rados.rados.0.trial178.stdout:1797: seq_num 632 ranges {622024=414667,1783208=417873} 2026-02-21T22:23:07.394 INFO:tasks.rados.rados.0.trial178.stdout:1797: writing trial17818531-50 from 622024 to 1036691 tid 1 2026-02-21T22:23:07.403 INFO:tasks.rados.rados.0.trial178.stdout:1797: writing trial17818531-50 from 1783208 to 2201081 tid 2 2026-02-21T22:23:07.403 INFO:tasks.rados.rados.0.trial178.stdout:1798: snap_remove snap 172 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1798: done (1 left) 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1799: delete oid 4 current snap is 179 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1797: finishing write tid 1 to trial17818531-50 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1797: finishing write tid 2 to trial17818531-50 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1797: finishing write tid 3 to trial17818531-50 2026-02-21T22:23:08.177 INFO:tasks.rados.rados.0.trial178.stdout:1797: finishing write tid 4 to trial17818531-50 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1797: oid 50 updating version 0 to 761 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1797: oid 50 version 761 is already newer than 760 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1797: oid 50 updating version 761 to 762 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 762 (ObjNum 632 snap 179 seq_num 632) dirty exists 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1797: left oid 50 (ObjNum 632 snap 179 seq_num 632) 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1797: done (1 left) 2026-02-21T22:23:08.178 INFO:tasks.rados.rados.0.trial178.stdout:1799: done (0 left) 2026-02-21T22:23:08.179 INFO:tasks.rados.rados.0.trial178.stdout:1800: snap_create 2026-02-21T22:23:08.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:08.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:09.176 INFO:tasks.rados.rados.0.trial178.stdout:1800: done (0 left) 2026-02-21T22:23:09.176 INFO:tasks.rados.rados.0.trial178.stdout:1801: copy_from oid 25 from oid 16 current snap is 180 2026-02-21T22:23:09.176 INFO:tasks.rados.rados.0.trial178.stdout:1802: write oid 33 current snap is 180 2026-02-21T22:23:09.176 INFO:tasks.rados.rados.0.trial178.stdout:1802: seq_num 634 ranges {655638=565165,1900172=421542} 2026-02-21T22:23:09.184 INFO:tasks.rados.rados.0.trial178.stdout:1802: writing trial17818531-33 from 655638 to 1220803 tid 1 2026-02-21T22:23:09.192 INFO:tasks.rados.rados.0.trial178.stdout:1802: writing trial17818531-33 from 1900172 to 2321714 tid 2 2026-02-21T22:23:09.192 INFO:tasks.rados.rados.0.trial178.stdout:1803: write oid 11 current snap is 180 2026-02-21T22:23:09.192 INFO:tasks.rados.rados.0.trial178.stdout:1803: seq_num 635 ranges {672445=640414,1842301=681553} 2026-02-21T22:23:09.201 INFO:tasks.rados.rados.0.trial178.stdout:1803: writing trial17818531-11 from 672445 to 1312859 tid 1 2026-02-21T22:23:09.210 INFO:tasks.rados.rados.0.trial178.stdout:1803: writing trial17818531-11 from 1842301 to 2523854 tid 2 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1804: rollback oid 14 current snap is 180 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 167 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1801: finishing copy_from to trial17818531-25 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1801: got expected ENOENT (src dne) 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1801: finishing copy_from racing read to trial17818531-25 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: finishing write tid 1 to trial17818531-33 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: finishing write tid 2 to trial17818531-33 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: finishing write tid 3 to trial17818531-33 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: finishing write tid 4 to trial17818531-33 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: oid 33 updating version 0 to 425 2026-02-21T22:23:09.211 INFO:tasks.rados.rados.0.trial178.stdout:1802: oid 33 updating version 425 to 426 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1802: oid 33 version 426 is already newer than 424 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 426 (ObjNum 634 snap 180 seq_num 634) dirty exists 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1802: left oid 33 (ObjNum 634 snap 180 seq_num 634) 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1803: finishing write tid 1 to trial17818531-11 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1801: done (3 left) 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1802: done (2 left) 2026-02-21T22:23:09.212 INFO:tasks.rados.rados.0.trial178.stdout:1805: delete oid 42 current snap is 180 2026-02-21T22:23:09.213 INFO:tasks.rados.rados.0.trial178.stdout:1803: finishing write tid 2 to trial17818531-11 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: finishing write tid 3 to trial17818531-11 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: finishing write tid 4 to trial17818531-11 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: oid 11 updating version 0 to 822 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: oid 11 version 822 is already newer than 821 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: oid 11 version 822 is already newer than 820 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 822 (ObjNum 635 snap 180 seq_num 635) dirty exists 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: left oid 11 (ObjNum 635 snap 180 seq_num 635) 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1803: done (2 left) 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1805: done (1 left) 2026-02-21T22:23:09.214 INFO:tasks.rados.rados.0.trial178.stdout:1806: snap_remove snap 162 2026-02-21T22:23:10.182 INFO:tasks.rados.rados.0.trial178.stdout:1806: done (1 left) 2026-02-21T22:23:10.182 INFO:tasks.rados.rados.0.trial178.stdout:1807: snap_remove snap 179 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:1807: done (1 left) 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:1808: copy_from oid 45 from oid 40 current snap is 180 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:1804: finishing rollback tid 0 to trial17818531-14 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:1804: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:1804: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:23:11.171 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 769 (ObjNum 598 snap 167 seq_num 598) dirty exists 2026-02-21T22:23:11.172 INFO:tasks.rados.rados.0.trial178.stdout:1804: done (1 left) 2026-02-21T22:23:11.172 INFO:tasks.rados.rados.0.trial178.stdout:1809: write oid 37 current snap is 180 2026-02-21T22:23:11.172 INFO:tasks.rados.rados.0.trial178.stdout:1809: seq_num 637 ranges {706059=790912,1959265=685222} 2026-02-21T22:23:11.178 INFO:tasks.rados.rados.0.trial178.stdout:1809: writing trial17818531-37 from 706059 to 1496971 tid 1 2026-02-21T22:23:11.184 INFO:tasks.rados.rados.0.trial178.stdout:1809: writing trial17818531-37 from 1959265 to 2644487 tid 2 2026-02-21T22:23:11.184 INFO:tasks.rados.rados.0.trial178.stdout:1810: copy_from oid 25 from oid 10 current snap is 180 2026-02-21T22:23:11.185 INFO:tasks.rados.rados.0.trial178.stdout:1811: copy_from oid 32 from oid 19 current snap is 180 2026-02-21T22:23:11.185 INFO:tasks.rados.rados.0.trial178.stdout:1812: write oid 48 current snap is 180 2026-02-21T22:23:11.185 INFO:tasks.rados.rados.0.trial178.stdout:1812: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2026-02-21T22:23:11.192 INFO:tasks.rados.rados.0.trial178.stdout:1812: writing trial17818531-48 from 722866 to 1189027 tid 1 2026-02-21T22:23:11.198 INFO:tasks.rados.rados.0.trial178.stdout:1812: writing trial17818531-48 from 1617747 to 2046627 tid 2 2026-02-21T22:23:11.203 INFO:tasks.rados.rados.0.trial178.stdout:1812: writing trial17818531-48 from 2587634 to 2722866 tid 3 2026-02-21T22:23:11.203 INFO:tasks.rados.rados.0.trial178.stdout:1813: write oid 6 current snap is 180 2026-02-21T22:23:11.203 INFO:tasks.rados.rados.0.trial178.stdout:1813: seq_num 639 ranges {739673=657763,2076229=663444} 2026-02-21T22:23:11.212 INFO:tasks.rados.rados.0.trial178.stdout:1813: writing trial17818531-6 from 739673 to 1397436 tid 1 2026-02-21T22:23:11.218 INFO:tasks.rados.rados.0.trial178.stdout:1813: writing trial17818531-6 from 2076229 to 2739673 tid 2 2026-02-21T22:23:11.218 INFO:tasks.rados.rados.0.trial178.stdout:1814: rollback oid 44 current snap is 180 2026-02-21T22:23:11.218 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 167 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1808: finishing copy_from racing read to trial17818531-45 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1809: finishing write tid 1 to trial17818531-37 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1808: finishing copy_from to trial17818531-45 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 758 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1811: finishing copy_from racing read to trial17818531-32 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1810: finishing copy_from racing read to trial17818531-25 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1811: finishing copy_from to trial17818531-32 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 977 (ObjNum 627 snap 179 seq_num 627) dirty exists 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1810: finishing copy_from to trial17818531-25 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 761 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1809: finishing write tid 2 to trial17818531-37 2026-02-21T22:23:11.219 INFO:tasks.rados.rados.0.trial178.stdout:1808: done (6 left) 2026-02-21T22:23:11.220 INFO:tasks.rados.rados.0.trial178.stdout:1810: done (5 left) 2026-02-21T22:23:11.220 INFO:tasks.rados.rados.0.trial178.stdout:1811: done (4 left) 2026-02-21T22:23:11.220 INFO:tasks.rados.rados.0.trial178.stdout:1815: write oid 32 current snap is 180 2026-02-21T22:23:11.220 INFO:tasks.rados.rados.0.trial178.stdout:1815: seq_num 640 ranges {756480=733012,2134711=432549} 2026-02-21T22:23:11.224 INFO:tasks.rados.rados.0.trial178.stdout:1815: writing trial17818531-32 from 756480 to 1489492 tid 1 2026-02-21T22:23:11.228 INFO:tasks.rados.rados.0.trial178.stdout:1815: writing trial17818531-32 from 2134711 to 2567260 tid 2 2026-02-21T22:23:11.228 INFO:tasks.rados.rados.0.trial178.stdout:1816: read oid 18 snap -1 2026-02-21T22:23:11.228 INFO:tasks.rados.rados.0.trial178.stdout:1816: expect (ObjNum 602 snap 168 seq_num 602) 2026-02-21T22:23:11.229 INFO:tasks.rados.rados.0.trial178.stdout:1817: write (excl) oid 41 current snap is 180 2026-02-21T22:23:11.229 INFO:tasks.rados.rados.0.trial178.stdout:1817: seq_num 641 ranges {773287=408261,1793193=576207} 2026-02-21T22:23:11.233 INFO:tasks.rados.rados.0.trial178.stdout:1817: writing trial17818531-41 from 773287 to 1181548 tid 1 2026-02-21T22:23:11.237 INFO:tasks.rados.rados.0.trial178.stdout:1817: writing trial17818531-41 from 1793193 to 2369400 tid 2 2026-02-21T22:23:11.238 INFO:tasks.rados.rados.0.trial178.stdout:1818: snap_remove snap 168 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1818: done (7 left) 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1819: rollback oid 27 current snap is 180 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 167 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1809: finishing write tid 3 to trial17818531-37 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1809: finishing write tid 4 to trial17818531-37 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1809: oid 37 updating version 0 to 724 2026-02-21T22:23:12.191 INFO:tasks.rados.rados.0.trial178.stdout:1809: oid 37 updating version 724 to 760 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:1809: oid 37 version 760 is already newer than 759 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 760 (ObjNum 637 snap 180 seq_num 637) dirty exists 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:1809: left oid 37 (ObjNum 637 snap 180 seq_num 637) 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:1809: done (7 left) 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:1820: write oid 36 current snap is 180 2026-02-21T22:23:12.192 INFO:tasks.rados.rados.0.trial178.stdout:1820: seq_num 642 ranges {790094=483510,1851675=436218} 2026-02-21T22:23:12.196 INFO:tasks.rados.rados.0.trial178.stdout:1820: writing trial17818531-36 from 790094 to 1273604 tid 1 2026-02-21T22:23:12.200 INFO:tasks.rados.rados.0.trial178.stdout:1820: writing trial17818531-36 from 1851675 to 2287893 tid 2 2026-02-21T22:23:12.200 INFO:tasks.rados.rados.0.trial178.stdout:1821: write (excl) oid 46 current snap is 180 2026-02-21T22:23:12.200 INFO:tasks.rados.rados.0.trial178.stdout:1821: seq_num 643 ranges {406901=558759,1393804=579876,2608396=198505} 2026-02-21T22:23:12.203 INFO:tasks.rados.rados.0.trial178.stdout:1821: writing trial17818531-46 from 406901 to 965660 tid 1 2026-02-21T22:23:12.207 INFO:tasks.rados.rados.0.trial178.stdout:1821: writing trial17818531-46 from 1393804 to 1973680 tid 2 2026-02-21T22:23:12.211 INFO:tasks.rados.rados.0.trial178.stdout:1821: writing trial17818531-46 from 2608396 to 2806901 tid 3 2026-02-21T22:23:12.211 INFO:tasks.rados.rados.0.trial178.stdout:1822: write (excl) oid 24 current snap is 180 2026-02-21T22:23:12.211 INFO:tasks.rados.rados.0.trial178.stdout:1822: seq_num 644 ranges {423708=634008,1852286=439887,2752172=71536} 2026-02-21T22:23:12.215 INFO:tasks.rados.rados.0.trial178.stdout:1822: writing trial17818531-24 from 423708 to 1057716 tid 1 2026-02-21T22:23:12.220 INFO:tasks.rados.rados.0.trial178.stdout:1822: writing trial17818531-24 from 1852286 to 2292173 tid 2 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1822: writing trial17818531-24 from 2752172 to 2823708 tid 3 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1823: copy_from oid 2 from oid 37 current snap is 180 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: finishing write tid 1 to trial17818531-48 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: finishing write tid 2 to trial17818531-48 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: finishing write tid 3 to trial17818531-48 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: finishing write tid 4 to trial17818531-48 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: finishing write tid 5 to trial17818531-48 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: oid 48 updating version 0 to 763 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: oid 48 version 763 is already newer than 761 2026-02-21T22:23:12.223 INFO:tasks.rados.rados.0.trial178.stdout:1812: oid 48 updating version 763 to 764 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1812: oid 48 version 764 is already newer than 762 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 764 (ObjNum 638 snap 180 seq_num 638) dirty exists 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1812: left oid 48 (ObjNum 638 snap 180 seq_num 638) 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1813: finishing write tid 1 to trial17818531-6 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1812: done (10 left) 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1824: read oid 29 snap -1 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1824: expect (ObjNum 592 snap 163 seq_num 592) 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1825: read oid 13 snap 178 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1825: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1826: write (excl) oid 1 current snap is 180 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1813: finishing write tid 2 to trial17818531-6 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1813: finishing write tid 3 to trial17818531-6 2026-02-21T22:23:12.224 INFO:tasks.rados.rados.0.trial178.stdout:1826: seq_num 645 ranges {440515=709257,1910768=583545} 2026-02-21T22:23:12.226 INFO:tasks.rados.rados.0.trial178.stdout:1826: writing trial17818531-1 from 440515 to 1149772 tid 1 2026-02-21T22:23:12.232 INFO:tasks.rados.rados.0.trial178.stdout:1826: writing trial17818531-1 from 1910768 to 2494313 tid 2 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1827: rollback oid 35 current snap is 180 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 35 to 178 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: finishing write tid 4 to trial17818531-6 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: oid 6 updating version 0 to 763 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: oid 6 updating version 763 to 764 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: oid 6 version 764 is already newer than 762 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 764 (ObjNum 639 snap 180 seq_num 639) dirty exists 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: left oid 6 (ObjNum 639 snap 180 seq_num 639) 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1814: finishing rollback tid 0 to trial17818531-44 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1813: done (13 left) 2026-02-21T22:23:12.233 INFO:tasks.rados.rados.0.trial178.stdout:1828: copy_from oid 12 from oid 6 current snap is 180 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1814: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1814: finishing rollback tid 2 to trial17818531-44 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 516 (ObjNum 588 snap 162 seq_num 588) dirty exists 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: finishing write tid 1 to trial17818531-32 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: finishing write tid 2 to trial17818531-32 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: finishing write tid 3 to trial17818531-32 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: finishing write tid 4 to trial17818531-32 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: oid 32 updating version 0 to 978 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: oid 32 updating version 978 to 979 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: oid 32 updating version 979 to 980 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 980 (ObjNum 640 snap 180 seq_num 640) dirty exists 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1815: left oid 32 (ObjNum 640 snap 180 seq_num 640) 2026-02-21T22:23:12.234 INFO:tasks.rados.rados.0.trial178.stdout:1816: expect (ObjNum 602 snap 168 seq_num 602) 2026-02-21T22:23:12.235 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: finishing write tid 1 to trial17818531-41 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: finishing write tid 2 to trial17818531-41 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: finishing write tid 3 to trial17818531-41 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: finishing write tid 4 to trial17818531-41 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: oid 41 updating version 0 to 720 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: oid 41 updating version 720 to 721 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: oid 41 updating version 721 to 722 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 722 (ObjNum 641 snap 180 seq_num 641) dirty exists 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: left oid 41 (ObjNum 641 snap 180 seq_num 641) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1819: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1819: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1819: finishing rollback tid 2 to trial17818531-27 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 772 (ObjNum 590 snap 163 seq_num 590) dirty exists 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: finishing write tid 1 to trial17818531-36 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: finishing write tid 2 to trial17818531-36 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: finishing write tid 3 to trial17818531-36 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1814: done (13 left) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1815: done (12 left) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1816: done (11 left) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1817: done (10 left) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1819: done (9 left) 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1829: read oid 17 snap -1 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1829: expect deleted 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1830: snap_create 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: finishing write tid 4 to trial17818531-36 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: oid 36 updating version 0 to 372 2026-02-21T22:23:12.314 INFO:tasks.rados.rados.0.trial178.stdout:1820: oid 36 version 372 is already newer than 370 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1820: oid 36 version 372 is already newer than 371 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 372 (ObjNum 642 snap 180 seq_num 642) dirty exists 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1820: left oid 36 (ObjNum 642 snap 180 seq_num 642) 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: finishing write tid 1 to trial17818531-46 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: finishing write tid 2 to trial17818531-46 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: finishing write tid 3 to trial17818531-46 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: finishing write tid 4 to trial17818531-46 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: finishing write tid 5 to trial17818531-46 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: oid 46 updating version 0 to 457 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: oid 46 version 457 is already newer than 455 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: oid 46 version 457 is already newer than 454 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: oid 46 version 457 is already newer than 456 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 457 (ObjNum 643 snap 180 seq_num 643) dirty exists 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1821: left oid 46 (ObjNum 643 snap 180 seq_num 643) 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: finishing write tid 1 to trial17818531-24 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: finishing write tid 2 to trial17818531-24 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: finishing write tid 3 to trial17818531-24 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: finishing write tid 4 to trial17818531-24 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: finishing write tid 5 to trial17818531-24 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: oid 24 updating version 0 to 1026 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: oid 24 updating version 1026 to 1027 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: oid 24 updating version 1027 to 1028 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: oid 24 version 1028 is already newer than 1025 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1028 (ObjNum 644 snap 180 seq_num 644) dirty exists 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1822: left oid 24 (ObjNum 644 snap 180 seq_num 644) 2026-02-21T22:23:12.315 INFO:tasks.rados.rados.0.trial178.stdout:1824: expect (ObjNum 592 snap 163 seq_num 592) 2026-02-21T22:23:12.365 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:12.395 INFO:tasks.rados.rados.0.trial178.stdout:1825: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:12.431 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2424259 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: finishing write tid 1 to trial17818531-1 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1823: finishing copy_from racing read to trial17818531-2 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1827: finishing rollback tid 0 to trial17818531-35 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1823: finishing copy_from to trial17818531-2 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 760 (ObjNum 637 snap 180 seq_num 637) dirty exists 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1827: finishing rollback tid 1 to trial17818531-35 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1827: finishing rollback tid 2 to trial17818531-35 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 983 (ObjNum 598 snap 167 seq_num 598) dirty exists 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: finishing write tid 2 to trial17818531-1 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: finishing write tid 3 to trial17818531-1 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: finishing write tid 4 to trial17818531-1 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: oid 1 updating version 0 to 766 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: oid 1 updating version 766 to 767 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: oid 1 version 767 is already newer than 765 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 767 (ObjNum 645 snap 180 seq_num 645) dirty exists 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1826: left oid 1 (ObjNum 645 snap 180 seq_num 645) 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1828: finishing copy_from racing read to trial17818531-12 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:1828: finishing copy_from to trial17818531-12 2026-02-21T22:23:12.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 764 (ObjNum 639 snap 180 seq_num 639) dirty exists 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1820: done (10 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1821: done (9 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1822: done (8 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1823: done (7 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1824: done (6 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1825: done (5 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1826: done (4 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1827: done (3 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1828: done (2 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1829: done (1 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1830: done (0 left) 2026-02-21T22:23:13.195 INFO:tasks.rados.rados.0.trial178.stdout:1831: snap_create 2026-02-21T22:23:13.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:13.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1831: done (0 left) 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1832: read oid 6 snap -1 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1832: expect (ObjNum 639 snap 180 seq_num 639) 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1833: read oid 7 snap -1 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1833: expect deleted 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1834: read oid 40 snap -1 2026-02-21T22:23:14.197 INFO:tasks.rados.rados.0.trial178.stdout:1834: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:14.198 INFO:tasks.rados.rados.0.trial178.stdout:1835: read oid 47 snap -1 2026-02-21T22:23:14.198 INFO:tasks.rados.rados.0.trial178.stdout:1835: expect (ObjNum 617 snap 175 seq_num 617) 2026-02-21T22:23:14.198 INFO:tasks.rados.rados.0.trial178.stdout:1836: write oid 19 current snap is 182 2026-02-21T22:23:14.198 INFO:tasks.rados.rados.0.trial178.stdout:1836: seq_num 646 ranges {457322=784506,1969250=443556} 2026-02-21T22:23:14.205 INFO:tasks.rados.rados.0.trial178.stdout:1836: writing trial17818531-19 from 457322 to 1241828 tid 1 2026-02-21T22:23:14.213 INFO:tasks.rados.rados.0.trial178.stdout:1836: writing trial17818531-19 from 1969250 to 2412806 tid 2 2026-02-21T22:23:14.213 INFO:tasks.rados.rados.0.trial178.stdout:1837: rollback oid 42 current snap is 182 2026-02-21T22:23:14.213 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 180 2026-02-21T22:23:14.213 INFO:tasks.rados.rados.0.trial178.stdout:1835: expect (ObjNum 617 snap 175 seq_num 617) 2026-02-21T22:23:14.266 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1659168 2026-02-21T22:23:14.284 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1008128 2026-02-21T22:23:14.303 INFO:tasks.rados.rados.0.trial178.stdout:1832: expect (ObjNum 639 snap 180 seq_num 639) 2026-02-21T22:23:14.339 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:14.428 INFO:tasks.rados.rados.0.trial178.stdout:1834: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:14.467 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3302365 2026-02-21T22:23:14.488 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1588163 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: finishing write tid 1 to trial17818531-19 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: finishing write tid 2 to trial17818531-19 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: finishing write tid 3 to trial17818531-19 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: finishing write tid 4 to trial17818531-19 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: oid 19 updating version 0 to 1032 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: oid 19 version 1032 is already newer than 1030 2026-02-21T22:23:14.583 INFO:tasks.rados.rados.0.trial178.stdout:1836: oid 19 version 1032 is already newer than 1031 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1032 (ObjNum 646 snap 182 seq_num 646) dirty exists 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1836: left oid 19 (ObjNum 646 snap 182 seq_num 646) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1837: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 997 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1832: done (5 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1833: done (4 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1834: done (3 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1835: done (2 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1836: done (1 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1837: done (0 left) 2026-02-21T22:23:14.584 INFO:tasks.rados.rados.0.trial178.stdout:1838: delete oid 32 current snap is 182 2026-02-21T22:23:14.585 INFO:tasks.rados.rados.0.trial178.stdout:1838: done (0 left) 2026-02-21T22:23:14.585 INFO:tasks.rados.rados.0.trial178.stdout:1839: snap_remove snap 180 2026-02-21T22:23:15.200 INFO:tasks.rados.rados.0.trial178.stdout:1839: done (0 left) 2026-02-21T22:23:15.200 INFO:tasks.rados.rados.0.trial178.stdout:1840: write oid 17 current snap is 182 2026-02-21T22:23:15.200 INFO:tasks.rados.rados.0.trial178.stdout:1840: seq_num 648 ranges {490936=651357,1686214=447225,2861864=29072} 2026-02-21T22:23:15.207 INFO:tasks.rados.rados.0.trial178.stdout:1840: writing trial17818531-17 from 490936 to 1142293 tid 1 2026-02-21T22:23:15.212 INFO:tasks.rados.rados.0.trial178.stdout:1840: writing trial17818531-17 from 1686214 to 2133439 tid 2 2026-02-21T22:23:15.215 INFO:tasks.rados.rados.0.trial178.stdout:1840: writing trial17818531-17 from 2861864 to 2890936 tid 3 2026-02-21T22:23:15.216 INFO:tasks.rados.rados.0.trial178.stdout:1841: snap_remove snap 167 2026-02-21T22:23:16.171 INFO:tasks.rados.rados.0.trial178.stdout:1841: done (1 left) 2026-02-21T22:23:16.171 INFO:tasks.rados.rados.0.trial178.stdout:1842: write oid 31 current snap is 182 2026-02-21T22:23:16.172 INFO:tasks.rados.rados.0.trial178.stdout:1842: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2026-02-21T22:23:16.179 INFO:tasks.rados.rados.0.trial178.stdout:1842: writing trial17818531-31 from 507743 to 1234349 tid 1 2026-02-21T22:23:16.183 INFO:tasks.rados.rados.0.trial178.stdout:1842: writing trial17818531-31 from 1744696 to 2335579 tid 2 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1842: writing trial17818531-31 from 2772934 to 2907743 tid 3 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1843: delete oid 41 current snap is 182 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: finishing write tid 1 to trial17818531-17 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: finishing write tid 2 to trial17818531-17 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: finishing write tid 3 to trial17818531-17 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: finishing write tid 4 to trial17818531-17 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: finishing write tid 5 to trial17818531-17 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: oid 17 updating version 0 to 723 2026-02-21T22:23:16.186 INFO:tasks.rados.rados.0.trial178.stdout:1840: oid 17 updating version 723 to 726 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:1840: oid 17 version 726 is already newer than 724 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:1840: oid 17 version 726 is already newer than 725 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 726 (ObjNum 648 snap 182 seq_num 648) dirty exists 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:1840: left oid 17 (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:1842: finishing write tid 1 to trial17818531-31 2026-02-21T22:23:16.187 INFO:tasks.rados.rados.0.trial178.stdout:1842: finishing write tid 2 to trial17818531-31 2026-02-21T22:23:16.188 INFO:tasks.rados.rados.0.trial178.stdout:1842: finishing write tid 3 to trial17818531-31 2026-02-21T22:23:16.188 INFO:tasks.rados.rados.0.trial178.stdout:1840: done (2 left) 2026-02-21T22:23:16.188 INFO:tasks.rados.rados.0.trial178.stdout:1843: done (1 left) 2026-02-21T22:23:16.188 INFO:tasks.rados.rados.0.trial178.stdout:1844: read oid 42 snap -1 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1844: expect deleted 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1845: snap_create 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: finishing write tid 4 to trial17818531-31 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: finishing write tid 5 to trial17818531-31 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: oid 31 updating version 0 to 765 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: oid 31 version 765 is already newer than 762 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: oid 31 version 765 is already newer than 764 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: oid 31 version 765 is already newer than 763 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 765 (ObjNum 649 snap 182 seq_num 649) dirty exists 2026-02-21T22:23:16.189 INFO:tasks.rados.rados.0.trial178.stdout:1842: left oid 31 (ObjNum 649 snap 182 seq_num 649) 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1842: done (2 left) 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1844: done (1 left) 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1845: done (0 left) 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1846: read oid 40 snap -1 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1846: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:17.172 INFO:tasks.rados.rados.0.trial178.stdout:1847: delete oid 18 current snap is 183 2026-02-21T22:23:17.174 INFO:tasks.rados.rados.0.trial178.stdout:1847: done (1 left) 2026-02-21T22:23:17.174 INFO:tasks.rados.rados.0.trial178.stdout:1848: delete oid 32 current snap is 183 2026-02-21T22:23:17.190 INFO:tasks.rados.rados.0.trial178.stdout:1846: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:17.320 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:17.376 INFO:tasks.rados.rados.0.trial178.stdout:1846: done (1 left) 2026-02-21T22:23:17.376 INFO:tasks.rados.rados.0.trial178.stdout:1848: done (0 left) 2026-02-21T22:23:17.377 INFO:tasks.rados.rados.0.trial178.stdout:1849: snap_remove snap 181 2026-02-21T22:23:18.173 INFO:tasks.rados.rados.0.trial178.stdout:1849: done (0 left) 2026-02-21T22:23:18.174 INFO:tasks.rados.rados.0.trial178.stdout:1850: snap_remove snap 178 2026-02-21T22:23:18.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:18.559 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:19.178 INFO:tasks.rados.rados.0.trial178.stdout:1850: done (0 left) 2026-02-21T22:23:19.178 INFO:tasks.rados.rados.0.trial178.stdout:1851: write (excl) oid 47 current snap is 183 2026-02-21T22:23:19.178 INFO:tasks.rados.rados.0.trial178.stdout:1851: seq_num 653 ranges {574971=627602,1862271=598221} 2026-02-21T22:23:19.186 INFO:tasks.rados.rados.0.trial178.stdout:1851: writing trial17818531-47 from 574971 to 1202573 tid 1 2026-02-21T22:23:19.192 INFO:tasks.rados.rados.0.trial178.stdout:1851: writing trial17818531-47 from 1862271 to 2460492 tid 2 2026-02-21T22:23:19.192 INFO:tasks.rados.rados.0.trial178.stdout:1852: snap_create 2026-02-21T22:23:19.192 INFO:tasks.rados.rados.0.trial178.stdout:1851: finishing write tid 1 to trial17818531-47 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: finishing write tid 2 to trial17818531-47 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: finishing write tid 3 to trial17818531-47 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: finishing write tid 4 to trial17818531-47 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: oid 47 updating version 0 to 1060 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: oid 47 updating version 1060 to 1061 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: oid 47 version 1061 is already newer than 1059 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1061 (ObjNum 653 snap 183 seq_num 653) dirty exists 2026-02-21T22:23:19.195 INFO:tasks.rados.rados.0.trial178.stdout:1851: left oid 47 (ObjNum 653 snap 183 seq_num 653) 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1851: done (1 left) 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1852: done (0 left) 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1853: copy_from oid 46 from oid 27 current snap is 184 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1854: read oid 14 snap -1 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1854: expect (ObjNum 598 snap 167 seq_num 598) 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1855: read oid 19 snap -1 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1855: expect (ObjNum 646 snap 182 seq_num 646) 2026-02-21T22:23:20.184 INFO:tasks.rados.rados.0.trial178.stdout:1856: snap_create 2026-02-21T22:23:20.186 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:23:20.188 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:20.189 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:20.189 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:20.189 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:20.189 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:20.189 INFO:tasks.rados.rados.0.trial178.stdout:1854: expect (ObjNum 598 snap 167 seq_num 598) 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:20.192 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:23:20.193 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:20.194 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:20.195 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:20.195 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:20.195 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:23:20.253 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1764270 2026-02-21T22:23:20.296 INFO:tasks.rados.rados.0.trial178.stdout:1855: expect (ObjNum 646 snap 182 seq_num 646) 2026-02-21T22:23:20.296 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:20.373 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1461242 2026-02-21T22:23:20.397 INFO:tasks.rados.rados.0.trial178.stdout:1853: finishing copy_from racing read to trial17818531-46 2026-02-21T22:23:20.397 INFO:tasks.rados.rados.0.trial178.stdout:1853: finishing copy_from to trial17818531-46 2026-02-21T22:23:20.397 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 772 (ObjNum 590 snap 163 seq_num 590) dirty exists 2026-02-21T22:23:21.173 INFO:tasks.rados.rados.0.trial178.stdout:1853: done (3 left) 2026-02-21T22:23:21.173 INFO:tasks.rados.rados.0.trial178.stdout:1854: done (2 left) 2026-02-21T22:23:21.173 INFO:tasks.rados.rados.0.trial178.stdout:1855: done (1 left) 2026-02-21T22:23:21.174 INFO:tasks.rados.rados.0.trial178.stdout:1856: done (0 left) 2026-02-21T22:23:21.174 INFO:tasks.rados.rados.0.trial178.stdout:1857: write (excl) oid 8 current snap is 185 2026-02-21T22:23:21.174 INFO:tasks.rados.rados.0.trial178.stdout:1857: seq_num 654 ranges {591778=419204,1637106=741879,2910049=81729} 2026-02-21T22:23:21.180 INFO:tasks.rados.rados.0.trial178.stdout:1857: writing trial17818531-8 from 591778 to 1010982 tid 1 2026-02-21T22:23:21.189 INFO:tasks.rados.rados.0.trial178.stdout:1857: writing trial17818531-8 from 1637106 to 2378985 tid 2 2026-02-21T22:23:21.193 INFO:tasks.rados.rados.0.trial178.stdout:1857: writing trial17818531-8 from 2910049 to 2991778 tid 3 2026-02-21T22:23:21.193 INFO:tasks.rados.rados.0.trial178.stdout:1858: write (excl) oid 6 current snap is 185 2026-02-21T22:23:21.193 INFO:tasks.rados.rados.0.trial178.stdout:1858: seq_num 655 ranges {608585=494453,1579235=601890,2821119=187466} 2026-02-21T22:23:21.201 INFO:tasks.rados.rados.0.trial178.stdout:1858: writing trial17818531-6 from 608585 to 1103038 tid 1 2026-02-21T22:23:21.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: writing trial17818531-6 from 1579235 to 2181125 tid 2 2026-02-21T22:23:21.214 INFO:tasks.rados.rados.0.trial178.stdout:1858: writing trial17818531-6 from 2821119 to 3008585 tid 3 2026-02-21T22:23:21.214 INFO:tasks.rados.rados.0.trial178.stdout:1859: write (excl) oid 33 current snap is 185 2026-02-21T22:23:21.214 INFO:tasks.rados.rados.0.trial178.stdout:1859: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2026-02-21T22:23:21.220 INFO:tasks.rados.rados.0.trial178.stdout:1859: writing trial17818531-33 from 625392 to 1195094 tid 1 2026-02-21T22:23:21.224 INFO:tasks.rados.rados.0.trial178.stdout:1859: writing trial17818531-33 from 1637717 to 2383265 tid 2 2026-02-21T22:23:21.227 INFO:tasks.rados.rados.0.trial178.stdout:1859: writing trial17818531-33 from 2848542 to 3025392 tid 3 2026-02-21T22:23:21.227 INFO:tasks.rados.rados.0.trial178.stdout:1860: snap_remove snap 184 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1860: done (3 left) 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1861: read oid 27 snap 182 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1861: expect (ObjNum 590 snap 163 seq_num 590) 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1862: read oid 34 snap -1 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1862: expect (ObjNum 613 snap 172 seq_num 613) 2026-02-21T22:23:22.192 INFO:tasks.rados.rados.0.trial178.stdout:1863: read oid 3 snap 183 2026-02-21T22:23:22.193 INFO:tasks.rados.rados.0.trial178.stdout:1863: expect deleted 2026-02-21T22:23:22.193 INFO:tasks.rados.rados.0.trial178.stdout:1864: read oid 17 snap -1 2026-02-21T22:23:22.193 INFO:tasks.rados.rados.0.trial178.stdout:1864: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:23:22.193 INFO:tasks.rados.rados.0.trial178.stdout:1865: write oid 24 current snap is 185 2026-02-21T22:23:22.193 INFO:tasks.rados.rados.0.trial178.stdout:1865: seq_num 657 ranges {642199=644951,1696199=605559,2992318=49881} 2026-02-21T22:23:22.199 INFO:tasks.rados.rados.0.trial178.stdout:1865: writing trial17818531-24 from 642199 to 1287150 tid 1 2026-02-21T22:23:22.203 INFO:tasks.rados.rados.0.trial178.stdout:1865: writing trial17818531-24 from 1696199 to 2301758 tid 2 2026-02-21T22:23:22.206 INFO:tasks.rados.rados.0.trial178.stdout:1865: writing trial17818531-24 from 2992318 to 3042199 tid 3 2026-02-21T22:23:22.206 INFO:tasks.rados.rados.0.trial178.stdout:1866: delete oid 30 current snap is 185 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: finishing write tid 1 to trial17818531-8 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: finishing write tid 2 to trial17818531-8 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: finishing write tid 3 to trial17818531-8 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: finishing write tid 4 to trial17818531-8 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: finishing write tid 5 to trial17818531-8 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: oid 8 updating version 0 to 1066 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: oid 8 version 1066 is already newer than 1064 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: oid 8 version 1066 is already newer than 1063 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: oid 8 version 1066 is already newer than 1065 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1066 (ObjNum 654 snap 185 seq_num 654) dirty exists 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1857: left oid 8 (ObjNum 654 snap 185 seq_num 654) 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1858: finishing write tid 1 to trial17818531-6 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1858: finishing write tid 2 to trial17818531-6 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1858: finishing write tid 3 to trial17818531-6 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1858: finishing write tid 4 to trial17818531-6 2026-02-21T22:23:22.207 INFO:tasks.rados.rados.0.trial178.stdout:1858: finishing write tid 5 to trial17818531-6 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: oid 6 updating version 0 to 768 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: oid 6 version 768 is already newer than 765 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: oid 6 version 768 is already newer than 766 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: oid 6 version 768 is already newer than 767 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 768 (ObjNum 655 snap 185 seq_num 655) dirty exists 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1858: left oid 6 (ObjNum 655 snap 185 seq_num 655) 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: finishing write tid 1 to trial17818531-33 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: finishing write tid 2 to trial17818531-33 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: finishing write tid 3 to trial17818531-33 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: finishing write tid 4 to trial17818531-33 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: finishing write tid 5 to trial17818531-33 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: oid 33 updating version 0 to 430 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: oid 33 version 430 is already newer than 427 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: oid 33 version 430 is already newer than 428 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: oid 33 version 430 is already newer than 429 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 430 (ObjNum 656 snap 185 seq_num 656) dirty exists 2026-02-21T22:23:22.208 INFO:tasks.rados.rados.0.trial178.stdout:1859: left oid 33 (ObjNum 656 snap 185 seq_num 656) 2026-02-21T22:23:22.209 INFO:tasks.rados.rados.0.trial178.stdout:1861: expect (ObjNum 590 snap 163 seq_num 590) 2026-02-21T22:23:22.377 INFO:tasks.rados.rados.0.trial178.stdout:1864: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:23:22.377 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:22.440 INFO:tasks.rados.rados.0.trial178.stdout:1862: expect (ObjNum 613 snap 172 seq_num 613) 2026-02-21T22:23:22.446 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 739985 2026-02-21T22:23:22.469 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: finishing write tid 1 to trial17818531-24 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: finishing write tid 2 to trial17818531-24 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: finishing write tid 3 to trial17818531-24 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: finishing write tid 4 to trial17818531-24 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: finishing write tid 5 to trial17818531-24 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: oid 24 updating version 0 to 1072 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: oid 24 version 1072 is already newer than 1071 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: oid 24 version 1072 is already newer than 1070 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: oid 24 updating version 1072 to 1073 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1073 (ObjNum 657 snap 185 seq_num 657) dirty exists 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: left oid 24 (ObjNum 657 snap 185 seq_num 657) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1857: done (8 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1858: done (7 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1859: done (6 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1861: done (5 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1862: done (4 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1863: done (3 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1864: done (2 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1865: done (1 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1866: done (0 left) 2026-02-21T22:23:22.528 INFO:tasks.rados.rados.0.trial178.stdout:1867: write (excl) oid 36 current snap is 185 2026-02-21T22:23:22.529 INFO:tasks.rados.rados.0.trial178.stdout:1867: seq_num 659 ranges {675813=795449,2096810=609228} 2026-02-21T22:23:22.538 INFO:tasks.rados.rados.0.trial178.stdout:1867: writing trial17818531-36 from 675813 to 1471262 tid 1 2026-02-21T22:23:22.547 INFO:tasks.rados.rados.0.trial178.stdout:1867: writing trial17818531-36 from 2096810 to 2706038 tid 2 2026-02-21T22:23:22.547 INFO:tasks.rados.rados.0.trial178.stdout:1868: copy_from oid 37 from oid 20 current snap is 185 2026-02-21T22:23:22.548 INFO:tasks.rados.rados.0.trial178.stdout:1867: finishing write tid 1 to trial17818531-36 2026-02-21T22:23:22.548 INFO:tasks.rados.rados.0.trial178.stdout:1869: rollback oid 26 current snap is 185 2026-02-21T22:23:22.548 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 182 2026-02-21T22:23:22.548 INFO:tasks.rados.rados.0.trial178.stdout:1870: snap_remove snap 183 2026-02-21T22:23:23.197 INFO:tasks.rados.rados.0.trial178.stdout:1870: done (3 left) 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1871: read oid 10 snap -1 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1871: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1872: write (excl) oid 49 current snap is 185 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1867: finishing write tid 2 to trial17818531-36 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1867: finishing write tid 3 to trial17818531-36 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1867: finishing write tid 4 to trial17818531-36 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1867: oid 36 updating version 0 to 375 2026-02-21T22:23:23.198 INFO:tasks.rados.rados.0.trial178.stdout:1867: oid 36 version 375 is already newer than 373 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1867: oid 36 version 375 is already newer than 374 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 375 (ObjNum 659 snap 185 seq_num 659) dirty exists 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1867: left oid 36 (ObjNum 659 snap 185 seq_num 659) 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1869: finishing rollback tid 0 to trial17818531-26 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1869: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1869: finishing rollback tid 2 to trial17818531-26 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 771 (ObjNum 564 snap 158 seq_num 564) dirty exists 2026-02-21T22:23:23.199 INFO:tasks.rados.rados.0.trial178.stdout:1868: finishing copy_from racing read to trial17818531-37 2026-02-21T22:23:23.200 INFO:tasks.rados.rados.0.trial178.stdout:1868: finishing copy_from to trial17818531-37 2026-02-21T22:23:23.200 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 411 (ObjNum 575 snap 158 seq_num 575) dirty exists 2026-02-21T22:23:23.200 INFO:tasks.rados.rados.0.trial178.stdout:1872: seq_num 660 ranges {692620=470698,1755292=752886} 2026-02-21T22:23:23.205 INFO:tasks.rados.rados.0.trial178.stdout:1872: writing trial17818531-49 from 692620 to 1163318 tid 1 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1872: writing trial17818531-49 from 1755292 to 2508178 tid 2 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1867: done (4 left) 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1868: done (3 left) 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1869: done (2 left) 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1873: write (excl) oid 48 current snap is 185 2026-02-21T22:23:23.214 INFO:tasks.rados.rados.0.trial178.stdout:1873: seq_num 661 ranges {709427=545947,1813774=612897,2985657=123770} 2026-02-21T22:23:23.222 INFO:tasks.rados.rados.0.trial178.stdout:1873: writing trial17818531-48 from 709427 to 1255374 tid 1 2026-02-21T22:23:23.230 INFO:tasks.rados.rados.0.trial178.stdout:1873: writing trial17818531-48 from 1813774 to 2426671 tid 2 2026-02-21T22:23:23.235 INFO:tasks.rados.rados.0.trial178.stdout:1873: writing trial17818531-48 from 2985657 to 3109427 tid 3 2026-02-21T22:23:23.235 INFO:tasks.rados.rados.0.trial178.stdout:1874: snap_create 2026-02-21T22:23:23.235 INFO:tasks.rados.rados.0.trial178.stdout:1871: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:23.259 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:23.316 INFO:tasks.rados.rados.0.trial178.stdout:1872: finishing write tid 1 to trial17818531-49 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: finishing write tid 2 to trial17818531-49 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: finishing write tid 3 to trial17818531-49 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: finishing write tid 4 to trial17818531-49 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: oid 49 updating version 0 to 767 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: oid 49 updating version 767 to 768 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: oid 49 version 768 is already newer than 766 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 768 (ObjNum 660 snap 185 seq_num 660) dirty exists 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1872: left oid 49 (ObjNum 660 snap 185 seq_num 660) 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: finishing write tid 1 to trial17818531-48 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: finishing write tid 2 to trial17818531-48 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: finishing write tid 3 to trial17818531-48 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: finishing write tid 4 to trial17818531-48 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: finishing write tid 5 to trial17818531-48 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: oid 48 updating version 0 to 770 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: oid 48 version 770 is already newer than 769 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: oid 48 version 770 is already newer than 768 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: oid 48 updating version 770 to 771 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 771 (ObjNum 661 snap 185 seq_num 661) dirty exists 2026-02-21T22:23:23.317 INFO:tasks.rados.rados.0.trial178.stdout:1873: left oid 48 (ObjNum 661 snap 185 seq_num 661) 2026-02-21T22:23:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:23:23] 2026-02-21T22:23:23.527 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:23:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:23:23, mode upmap, max misplaced 0.050000 2026-02-21T22:23:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:23:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:23:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', 'unique_pool_0', '.mgr'] 2026-02-21T22:23:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:23:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:23:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:23:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:23:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:23:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:23:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:23:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:23:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:23:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:23:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:23:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:23:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:23:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:23:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:23:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:23:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:23:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "393", 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:23:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:23:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:23:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:23:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 393, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 190, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:23:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:23:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:23:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:23:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:23:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00031122278284143517 of space, bias 1.0, pg target 0.09336683485243055 quantized to 32 (current 16) 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:23:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:23.822 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:23:23.822 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:23:23.823 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-21T22:23:23.823 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-21T22:23:23.836 INFO:tasks.ceph.mon.c:Started 2026-02-21T22:23:23.836 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:23:23.836 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: get_monmap_and_config 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: build_initial_monmap 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: monmap: 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:23.899244+0000 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:23.899244+0000 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 1 Processor -- start 2026-02-21T22:23:23.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 1 -- start start 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: init 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-a con 0x7fd9dc091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-b con 0x7fd9dc0909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-c con 0x7fd9dc1854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc057a20 con 0x7fd9dc091240 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc091d20 con 0x7fd9dc1854b0 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e0ee9640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd9dc091240 0x7fd9dc091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40894/0 (socket says 10.20.193.102:40894) 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e0ee9640 1 -- 10.20.193.102:0/1897088726 learned_addr learned my addr 10.20.193.102:0/1897088726 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc057740 con 0x7fd9dc0909a0 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:23.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.895+0000 7fd9e3174640 10 monclient(hunting): authenticate will time out at 1613.135761s 2026-02-21T22:23:23.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d3fff640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.102:6790/0 conn(0x7fd9dc0909a0 legacy=0x7fd9dc183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e0ee9640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.102:6789/0 conn(0x7fd9dc091240 legacy=0x7fd9dc091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356425912 0 0) 0x7fd9dc057a20 con 0x7fd9dc091240 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient(hunting): my global_id is 34371 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c8003620 con 0x7fd9dc091240 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e16ea640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.178:6789/0 conn(0x7fd9dc1854b0 legacy=0x7fd9dc1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2559684596 0 0) 0x7fd9c8003620 con 0x7fd9dc091240 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.102:6790/0 conn(0x7fd9dc0909a0 legacy=0x7fd9dc183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.178:6789/0 conn(0x7fd9dc1854b0 legacy=0x7fd9dc1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 monclient: found mon.noname-a 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc090dc0 con 0x7fd9dc091240 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: _finish_auth 0 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 5 monclient: authenticate success, global_id 34371 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9cc003040 con 0x7fd9dc091240 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: dump: 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:23.901 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9cc003310 con 0x7fd9dc091240 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 1 -- 10.20.193.102:0/1897088726 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9cc0049a0 con 0x7fd9dc091240 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9d37fe640 10 monclient: dump: 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:23.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 4 set_mon_vals no callback set 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: shutdown 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- 10.20.193.102:0/1897088726 >> v1:10.20.193.102:6789/0 conn(0x7fd9dc091240 legacy=0x7fd9dc091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- 10.20.193.102:0/1897088726 shutdown_connections 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- 10.20.193.102:0/1897088726 >> 10.20.193.102:0/1897088726 conn(0x7fd9dc05e660 msgr2=0x7fd9dc095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- 10.20.193.102:0/1897088726 shutdown_connections 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- 10.20.193.102:0/1897088726 wait complete. 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: build_initial_monmap 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: monmap: 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 Processor -- start 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- start start 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: init 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-c con 0x7fd9dc0909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-b con 0x7fd9dc091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:23.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient(hunting): picked mon.noname-a con 0x7fd9dc1854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc1121d0 con 0x7fd9dc1854b0 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc12bdf0 con 0x7fd9dc091240 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc0592d0 con 0x7fd9dc0909a0 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e3174640 10 monclient(hunting): authenticate will time out at 1613.139768s 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e16ea640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd9dc1854b0 0x7fd9dc1741c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40896/0 (socket says 10.20.193.102:40896) 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.899+0000 7fd9e16ea640 1 -- 10.20.193.102:0/1567956347 learned_addr learned my addr 10.20.193.102:0/1567956347 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e0ee9640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.102:6790/0 conn(0x7fd9dc0909a0 legacy=0x7fd9dc12b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e16ea640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.102:6789/0 conn(0x7fd9dc1854b0 legacy=0x7fd9dc1741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670177855 0 0) 0x7fd9dc1121d0 con 0x7fd9dc1854b0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): my global_id is 34374 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9ac003620 con 0x7fd9dc1854b0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d3fff640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.178:6789/0 conn(0x7fd9dc091240 legacy=0x7fd9dc172aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e16ea640 1 --1- 10.20.193.102:0/1567956347 >> v1:10.20.193.102:6789/0 conn(0x7fd9dc1854b0 0x7fd9dc1741c0 :-1 s=OPENED pgs=310 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fd9d8004930 mon_map magic: 0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3257213494 0 0) 0x7fd9ac003620 con 0x7fd9dc1854b0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9dc1121d0 con 0x7fd9dc1854b0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9d8004930 con 0x7fd9dc1854b0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): dump: 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977193762 0 0) 0x7fd9dc12bdf0 con 0x7fd9dc091240 2026-02-21T22:23:23.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): my global_id is 54193 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9ac003620 con 0x7fd9dc091240 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1104841093 0 0) 0x7fd9ac003620 con 0x7fd9dc091240 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9dc12bdf0 con 0x7fd9dc091240 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9c4003310 con 0x7fd9dc091240 2026-02-21T22:23:23.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient(hunting): dump: 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4261198038 0 0) 0x7fd9dc12bdf0 con 0x7fd9dc091240 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.102:6790/0 conn(0x7fd9dc0909a0 legacy=0x7fd9dc12b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.102:6789/0 conn(0x7fd9dc1854b0 legacy=0x7fd9dc1741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: _finish_hunting 0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 monclient: found mon.b 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc110ce0 con 0x7fd9dc091240 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: _finish_auth 0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: _check_auth_tickets 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 596851645 0 0) 0x7fd9dc1121d0 con 0x7fd9dc1854b0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 5 monclient: authenticate success, global_id 54193 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _renew_subs 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc1130a0 con 0x7fd9dc091240 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _renew_subs 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd9dc111a10 con 0x7fd9dc091240 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9c40039c0 con 0x7fd9dc091240 2026-02-21T22:23:23.907 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9c4003d20 con 0x7fd9dc091240 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d17fa640 10 monclient: dump: 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9d2ffd640 4 set_mon_vals no callback set 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.903+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc0008e0 con 0x7fd9dc091240 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd9c4003fc0 con 0x7fd9dc091240 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(393..393 src has 112..393) ==== 3975+0+0 (unknown 2931935947 0 0) 0x7fd9c404c490 con 0x7fd9dc091240 2026-02-21T22:23:23.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9e0ee9640 10 monclient: get_auth_request con 0x7fd9ac03d650 auth_method 0 2026-02-21T22:23:23.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fd9dc1130a0 con 0x7fd9dc091240 2026-02-21T22:23:23.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9d17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:23.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:23.907+0000 7fd9d17fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:24.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:23:24.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:24.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:24.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd9dc095010 con 0x7fd9dc091240 2026-02-21T22:23:24.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9d17fa640 1 -- 10.20.193.102:0/1567956347 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1718071305) 0x7fd9c40174c0 con 0x7fd9dc091240 2026-02-21T22:23:24.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9d17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:24.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9d17fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:24.025 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:23:24.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd9ac03d650 legacy=0x7fd9ac05db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 10 monclient: shutdown 2026-02-21T22:23:24.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 >> v1:10.20.193.178:6789/0 conn(0x7fd9dc091240 legacy=0x7fd9dc172aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 shutdown_connections 2026-02-21T22:23:24.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 >> 10.20.193.102:0/1567956347 conn(0x7fd9dc05e660 msgr2=0x7fd9dc08bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 shutdown_connections 2026-02-21T22:23:24.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.023+0000 7fd9e3174640 1 -- 10.20.193.102:0/1567956347 wait complete. 2026-02-21T22:23:24.034 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:23:24.035 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:23:24.035 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:23:24.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: get_monmap_and_config 2026-02-21T22:23:24.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: monmap: 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:24.105589+0000 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:24.105589+0000 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:24.106 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 1 Processor -- start 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 1 -- start start 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: init 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-b con 0x7f5edc091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-c con 0x7f5edc0909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-a con 0x7f5edc1854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc057a20 con 0x7f5edc1854b0 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc091d20 con 0x7f5edc0909a0 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc057740 con 0x7f5edc091240 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5ee0c2e640 10 monclient(hunting): authenticate will time out at 1613.341687s 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5eda575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5edc091240 0x7f5edc091610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38094/0 (socket says 10.20.193.102:38094) 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5eda575640 1 -- 10.20.193.102:0/979371377 learned_addr learned my addr 10.20.193.102:0/979371377 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5edad76640 1 --1- 10.20.193.102:0/979371377 >> v1:10.20.193.102:6789/0 conn(0x7f5edc1854b0 0x7f5edc1a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40900/0 (socket says 10.20.193.102:40900) 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5eda575640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.102:6790/0 conn(0x7f5edc091240 legacy=0x7f5edc091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.103+0000 7f5edad76640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.102:6789/0 conn(0x7f5edc1854b0 legacy=0x7f5edc1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9d74640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.178:6789/0 conn(0x7f5edc0909a0 legacy=0x7f5edc183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422550582 0 0) 0x7f5edc057a20 con 0x7f5edc1854b0 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): my global_id is 34377 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec8003620 con 0x7f5edc1854b0 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 727582251 0 0) 0x7f5edc091d20 con 0x7f5edc0909a0 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient(hunting): my global_id is 54196 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc057a20 con 0x7f5edc0909a0 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 282131357 0 0) 0x7f5edc057a20 con 0x7f5edc0909a0 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.102:6790/0 conn(0x7f5edc091240 legacy=0x7f5edc091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.102:6789/0 conn(0x7f5edc1854b0 legacy=0x7f5edc1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 monclient: found mon.noname-c 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc090dc0 con 0x7f5edc0909a0 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ec4003160 con 0x7f5edc0909a0 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 5 monclient: authenticate success, global_id 54196 2026-02-21T22:23:24.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: dump: 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.119 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5ec4003430 con 0x7f5edc0909a0 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 1 -- 10.20.193.102:0/979371377 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ec40038c0 con 0x7f5edc0909a0 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ed9573640 10 monclient: dump: 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.120 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:24.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 4 set_mon_vals no callback set 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: shutdown 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/979371377 >> v1:10.20.193.178:6789/0 conn(0x7f5edc0909a0 legacy=0x7f5edc183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/979371377 shutdown_connections 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/979371377 >> 10.20.193.102:0/979371377 conn(0x7f5edc05e660 msgr2=0x7f5edc095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/979371377 shutdown_connections 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/979371377 wait complete. 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: monmap: 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:24.122 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 Processor -- start 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- start start 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: init 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-c con 0x7f5edc0909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-a con 0x7f5edc091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient(hunting): picked mon.noname-b con 0x7f5edc1854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc12bc80 con 0x7f5edc091240 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc112ed0 con 0x7f5edc1854b0 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc112060 con 0x7f5edc0909a0 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5ee0c2e640 10 monclient(hunting): authenticate will time out at 1613.346590s 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5eda575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5edc0909a0 0x7f5edc12b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38098/0 (socket says 10.20.193.102:38098) 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.107+0000 7f5eda575640 1 -- 10.20.193.102:0/2862695397 learned_addr learned my addr 10.20.193.102:0/2862695397 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5eda575640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6790/0 conn(0x7f5edc0909a0 legacy=0x7f5edc12b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ed9d74640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc091240 legacy=0x7f5edc172930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119271680 0 0) 0x7f5edc112060 con 0x7f5edc0909a0 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): my global_id is 74099 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003620 con 0x7f5edc0909a0 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481323146 0 0) 0x7f5edc12bc80 con 0x7f5edc091240 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): my global_id is 34380 2026-02-21T22:23:24.125 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc112060 con 0x7f5edc091240 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5edad76640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.178:6789/0 conn(0x7f5edc1854b0 legacy=0x7f5edc174050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1258924607 0 0) 0x7f5edc112060 con 0x7f5edc091240 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5edc12bc80 con 0x7f5edc091240 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 762210899 0 0) 0x7f5ea8003620 con 0x7f5edc0909a0 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5edc112060 con 0x7f5edc0909a0 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ec4003710 con 0x7f5edc091240 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.126 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ed0003330 con 0x7f5edc0909a0 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:24.127 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912343618 0 0) 0x7f5edc112ed0 con 0x7f5edc1854b0 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient(hunting): my global_id is 54199 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003620 con 0x7f5edc1854b0 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1159054943 0 0) 0x7f5edc12bc80 con 0x7f5edc091240 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6790/0 conn(0x7f5edc0909a0 legacy=0x7f5edc12b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.178:6789/0 conn(0x7f5edc1854b0 legacy=0x7f5edc174050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 monclient: found mon.a 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc110b70 con 0x7f5edc091240 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 5 monclient: authenticate success, global_id 34380 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 10 monclient: _renew_subs 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc112630 con 0x7f5edc091240 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 10 monclient: _renew_subs 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5edc1118a0 con 0x7f5edc091240 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5ec4003430 con 0x7f5edc091240 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ec4004090 con 0x7f5edc091240 2026-02-21T22:23:24.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 10 monclient: dump: 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ed8d72640 4 set_mon_vals no callback set 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5edc110520 con 0x7f5edc12e510 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5ec40042f0 con 0x7f5edc091240 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(393..393 src has 112..393) ==== 3975+0+0 (unknown 2931935947 0 0) 0x7f5ec404c630 con 0x7f5edc091240 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.111+0000 7f5edad76640 10 monclient: get_auth_request con 0x7f5ea803d870 auth_method 0 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5eda575640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc12e510 legacy=0x7f5edc08ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894960926 0 0) 0x7f5edc110520 con 0x7f5edc12e510 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient(hunting): my global_id is 34383 2026-02-21T22:23:24.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc12bc80 con 0x7f5edc12e510 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4023747751 0 0) 0x7f5edc12bc80 con 0x7f5edc12e510 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5edc110520 con 0x7f5edc12e510 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ed0004c80 con 0x7f5edc12e510 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient: dump: 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.133 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3698115676 0 0) 0x7f5edc110520 con 0x7f5edc12e510 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5edc0008e0 con 0x7f5edc12e510 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f5ed0006140 con 0x7f5edc12e510 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc12e510 legacy=0x7f5edc08ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.115+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc12e510 legacy=0x7f5edc08ca00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:24.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5edc12e8e0 con 0x7f5edc08d110 2026-02-21T22:23:24.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5eda575640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc08d110 legacy=0x7f5edc176780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827647677 0 0) 0x7f5edc12e8e0 con 0x7f5edc08d110 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient(hunting): my global_id is 34386 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc110520 con 0x7f5edc08d110 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 225855698 0 0) 0x7f5edc110520 con 0x7f5edc08d110 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5edc12e8e0 con 0x7f5edc08d110 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ed0006810 con 0x7f5edc08d110 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient: dump: 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.137 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2949211654 0 0) 0x7f5edc12e8e0 con 0x7f5edc08d110 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5edc095010 con 0x7f5edc08d110 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4516 (unknown 0 0 2413050609) 0x7f5ed0006ae0 con 0x7f5edc08d110 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:24.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc08d110 legacy=0x7f5edc176780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ec2ffd640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc08d110 legacy=0x7f5edc176780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:24.139 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 352988, 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.140 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.141 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:24.142 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.143 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.144 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:24.145 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.146 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.147 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.148 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:24.149 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5ea803d870 legacy=0x7f5ea805dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 10 monclient: shutdown 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 >> v1:10.20.193.102:6789/0 conn(0x7f5edc091240 legacy=0x7f5edc172930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5edad76640 1 -- 10.20.193.102:0/2862695397 reap_dead start 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5edad76640 1 -- 10.20.193.102:0/2862695397 reap_dead start 2026-02-21T22:23:24.150 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 shutdown_connections 2026-02-21T22:23:24.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 >> 10.20.193.102:0/2862695397 conn(0x7f5edc05e660 msgr2=0x7f5edc0895d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 shutdown_connections 2026-02-21T22:23:24.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.119+0000 7f5ee0c2e640 1 -- 10.20.193.102:0/2862695397 wait complete. 2026-02-21T22:23:24.151 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:24.151 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:23:24.151 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:23:24.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: get_monmap_and_config 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: monmap: 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:24.223129+0000 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:24.223129+0000 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:24.223 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 1 Processor -- start 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 1 -- start start 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: init 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-a con 0x7f255808ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-c con 0x7f255808b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-b con 0x7f255808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2558086a10 con 0x7f255808ce00 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255808b130 con 0x7f255808b7f0 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255808bd60 con 0x7f255808d8e0 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f255ce91640 10 monclient(hunting): authenticate will time out at 1613.459272s 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f2556575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f255808ce00 0x7f255808d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40914/0 (socket says 10.20.193.102:40914) 2026-02-21T22:23:24.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.219+0000 7f2556575640 1 -- 10.20.193.102:0/3217262005 learned_addr learned my addr 10.20.193.102:0/3217262005 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2556575640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.102:6789/0 conn(0x7f255808ce00 legacy=0x7f255808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2556d76640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.102:6790/0 conn(0x7f255808d8e0 legacy=0x7f25581a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035228295 0 0) 0x7f2558086a10 con 0x7f255808ce00 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): my global_id is 34389 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2544003620 con 0x7f255808ce00 2026-02-21T22:23:24.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262360750 0 0) 0x7f255808bd60 con 0x7f255808d8e0 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient(hunting): my global_id is 74102 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2558086a10 con 0x7f255808d8e0 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555d74640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.178:6789/0 conn(0x7f255808b7f0 legacy=0x7f2558181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 732789450 0 0) 0x7f2544003620 con 0x7f255808ce00 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.102:6790/0 conn(0x7f255808d8e0 legacy=0x7f25581a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.178:6789/0 conn(0x7f255808b7f0 legacy=0x7f2558181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 monclient: found mon.noname-a 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255808b370 con 0x7f255808ce00 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f254c003110 con 0x7f255808ce00 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 5 monclient: authenticate success, global_id 34389 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:24.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: dump: 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f254c0033e0 con 0x7f255808ce00 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 1 -- 10.20.193.102:0/3217262005 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f254c004b50 con 0x7f255808ce00 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2555573640 10 monclient: dump: 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.239 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 4 set_mon_vals no callback set 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: shutdown 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- 10.20.193.102:0/3217262005 >> v1:10.20.193.102:6789/0 conn(0x7f255808ce00 legacy=0x7f255808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- 10.20.193.102:0/3217262005 shutdown_connections 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- 10.20.193.102:0/3217262005 >> 10.20.193.102:0/3217262005 conn(0x7f2558091a50 msgr2=0x7f2558091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- 10.20.193.102:0/3217262005 shutdown_connections 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- 10.20.193.102:0/3217262005 wait complete. 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: monmap: 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:24.241 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 Processor -- start 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- start start 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: init 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-a con 0x7f255808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-b con 0x7f255808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient(hunting): picked mon.noname-c con 0x7f255808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2558058a20 con 0x7f255808b7f0 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255805e170 con 0x7f255808ce00 2026-02-21T22:23:24.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2558088a70 con 0x7f255808d8e0 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f255ce91640 10 monclient(hunting): authenticate will time out at 1613.463923s 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2556575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f255808b7f0 0x7f25581532d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40916/0 (socket says 10.20.193.102:40916) 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.223+0000 7f2556575640 1 -- 10.20.193.102:0/730641946 learned_addr learned my addr 10.20.193.102:0/730641946 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f2556575640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6789/0 conn(0x7f255808b7f0 legacy=0x7f25581532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f2556d76640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255808d8e0 legacy=0x7f2558195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750880768 0 0) 0x7f255808b370 con 0x7f255808b7f0 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): my global_id is 34392 2026-02-21T22:23:24.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2524003620 con 0x7f255808b7f0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1905524488 0 0) 0x7f2558086a10 con 0x7f255808d8e0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): my global_id is 74105 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f255808b370 con 0x7f255808d8e0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f2555d74640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.178:6789/0 conn(0x7f255808ce00 legacy=0x7f25580849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3596112296 0 0) 0x7f2524003620 con 0x7f255808b7f0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2558086a10 con 0x7f255808b7f0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f254c002b70 con 0x7f255808b7f0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:24.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): dump: 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 686648167 0 0) 0x7f255808b370 con 0x7f255808d8e0 2026-02-21T22:23:24.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2524003620 con 0x7f255808d8e0 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2548004900 con 0x7f255808d8e0 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient(hunting): dump: 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.246 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1353283332 0 0) 0x7f2558086a10 con 0x7f255808b7f0 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255808d8e0 legacy=0x7f2558195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.178:6789/0 conn(0x7f255808ce00 legacy=0x7f25580849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 monclient: found mon.a 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558082f10 con 0x7f255808b7f0 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 5 monclient: authenticate success, global_id 34392 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 10 monclient: _renew_subs 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f25580831f0 con 0x7f255808b7f0 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 10 monclient: _renew_subs 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2558196d80 con 0x7f255808b7f0 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f254c004b10 con 0x7f255808b7f0 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f254c005170 con 0x7f255808b7f0 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 10 monclient: dump: 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.249 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f2554d72640 4 set_mon_vals no callback set 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f25580835d0 con 0x7f255805b850 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f254c011800 con 0x7f255808b7f0 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(393..393 src has 112..393) ==== 3975+0+0 (unknown 2931935947 0 0) 0x7f254c04c720 con 0x7f255808b7f0 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.227+0000 7f2556d76640 10 monclient: get_auth_request con 0x7f252403d6a0 auth_method 0 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f2555d74640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805b850 legacy=0x7f255805bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346740695 0 0) 0x7f255808b130 con 0x7f255805b850 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient(hunting): my global_id is 74108 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2558086a10 con 0x7f255805b850 2026-02-21T22:23:24.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2463068232 0 0) 0x7f2558086a10 con 0x7f255805b850 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f255808b130 con 0x7f255805b850 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2540003110 con 0x7f255805b850 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient: dump: 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.251 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 483473627 0 0) 0x7f255808b130 con 0x7f255805b850 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25580008e0 con 0x7f255805b850 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2540003530 con 0x7f255805b850 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805b850 legacy=0x7f255805bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.231+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805b850 legacy=0x7f255805bc20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.235+0000 7f255ce91640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.235+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f255805c730 con 0x7f255805c330 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f2555d74640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805c330 legacy=0x7f25581980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219541868 0 0) 0x7f255805c730 con 0x7f255805c330 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient(hunting): my global_id is 74111 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f255808b130 con 0x7f255805c330 2026-02-21T22:23:24.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 545951842 0 0) 0x7f255808b130 con 0x7f255805c330 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f255805c730 con 0x7f255805c330 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2540003b60 con 0x7f255805c330 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient: dump: 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.255 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 41859720 0 0) 0x7f255805c730 con 0x7f255805c330 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2558090810 con 0x7f255805c330 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4344 (unknown 0 0 1869436854) 0x7f25400058c0 con 0x7f255805c330 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805c330 legacy=0x7f25581980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f253effd640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6790/0 conn(0x7f255805c330 legacy=0x7f25581980d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:24.256 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 363, 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.257 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.258 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:24.259 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.260 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.261 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.262 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.263 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.264 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:24.265 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.178:6800/2549949439 conn(0x7f252403d6a0 legacy=0x7f252405db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 10 monclient: shutdown 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 >> v1:10.20.193.102:6789/0 conn(0x7f255808b7f0 legacy=0x7f25581532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f2556d76640 1 -- 10.20.193.102:0/730641946 reap_dead start 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f2556d76640 1 -- 10.20.193.102:0/730641946 reap_dead start 2026-02-21T22:23:24.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 shutdown_connections 2026-02-21T22:23:24.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 >> 10.20.193.102:0/730641946 conn(0x7f2558091a50 msgr2=0x7f25580575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 shutdown_connections 2026-02-21T22:23:24.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.239+0000 7f255ce91640 1 -- 10.20.193.102:0/730641946 wait complete. 2026-02-21T22:23:24.267 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:24.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:23:24.267 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:23:24.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: get_monmap_and_config 2026-02-21T22:23:24.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: monmap: 2026-02-21T22:23:24.336 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:24.337036+0000 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:24.337036+0000 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 1 Processor -- start 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 1 -- start start 2026-02-21T22:23:24.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: init 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-c con 0x7f9ca408ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-a con 0x7f9ca408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-b con 0x7f9ca408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca4086a10 con 0x7f9ca408b7f0 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca408b130 con 0x7f9ca408ce00 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca408bd60 con 0x7f9ca408d8e0 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient(hunting): authenticate will time out at 1613.573010s 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca27fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 0x7f9ca4181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40922/0 (socket says 10.20.193.102:40922) 2026-02-21T22:23:24.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca27fc640 1 -- 10.20.193.102:0/2133557117 learned_addr learned my addr 10.20.193.102:0/2133557117 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca27fc640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 legacy=0x7f9ca4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca37fe640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.102:6790/0 conn(0x7f9ca408d8e0 legacy=0x7f9ca41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671517951 0 0) 0x7f9ca4086a10 con 0x7f9ca408b7f0 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient(hunting): my global_id is 34395 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca2ffd640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.178:6789/0 conn(0x7f9ca408ce00 legacy=0x7f9ca408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c90003620 con 0x7f9ca408b7f0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3667230876 0 0) 0x7f9c90003620 con 0x7f9ca408b7f0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.102:6790/0 conn(0x7f9ca408d8e0 legacy=0x7f9ca41a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.178:6789/0 conn(0x7f9ca408ce00 legacy=0x7f9ca408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 monclient: found mon.noname-a 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca408b370 con 0x7f9ca408b7f0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c8c003110 con 0x7f9ca408b7f0 2026-02-21T22:23:24.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: dump: 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 5 monclient: authenticate success, global_id 34395 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c8c0033e0 con 0x7f9ca408b7f0 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca99fa640 4 set_mon_vals no callback set 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 1 -- 10.20.193.102:0/2133557117 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c8c004b50 con 0x7f9ca408b7f0 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.335+0000 7f9ca1ffb640 10 monclient: dump: 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.345 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: shutdown 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- 10.20.193.102:0/2133557117 >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 legacy=0x7f9ca4181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- 10.20.193.102:0/2133557117 shutdown_connections 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- 10.20.193.102:0/2133557117 >> 10.20.193.102:0/2133557117 conn(0x7f9ca4091a50 msgr2=0x7f9ca4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- 10.20.193.102:0/2133557117 shutdown_connections 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- 10.20.193.102:0/2133557117 wait complete. 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: build_initial_monmap 2026-02-21T22:23:24.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: monmap: 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 Processor -- start 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- start start 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: init 2026-02-21T22:23:24.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-a con 0x7f9ca408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-c con 0x7f9ca408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient(hunting): picked mon.noname-b con 0x7f9ca408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca4083230 con 0x7f9ca408b7f0 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca4088a70 con 0x7f9ca408d8e0 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca4080ed0 con 0x7f9ca408ce00 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca99fa640 10 monclient(hunting): authenticate will time out at 1613.577556s 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca2ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 0x7f9ca4084760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40926/0 (socket says 10.20.193.102:40926) 2026-02-21T22:23:24.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca2ffd640 1 -- 10.20.193.102:0/3222584108 learned_addr learned my addr 10.20.193.102:0/3222584108 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca2ffd640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 legacy=0x7f9ca4084760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca27fc640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.102:6790/0 conn(0x7f9ca408ce00 legacy=0x7f9ca4194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 832751495 0 0) 0x7f9ca4083230 con 0x7f9ca408b7f0 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 10 monclient(hunting): my global_id is 34398 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c74003620 con 0x7f9ca408b7f0 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.339+0000 7f9ca37fe640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca408d8e0 legacy=0x7f9ca4195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4255856865 0 0) 0x7f9c74003620 con 0x7f9ca408b7f0 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ca4083230 con 0x7f9ca408b7f0 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c98003420 con 0x7f9ca408b7f0 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient(hunting): dump: 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.352 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1284759439 0 0) 0x7f9ca4083230 con 0x7f9ca408b7f0 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.102:6790/0 conn(0x7f9ca408ce00 legacy=0x7f9ca4194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca408d8e0 legacy=0x7f9ca4195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 monclient: found mon.a 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4082f50 con 0x7f9ca408b7f0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: _finish_auth 0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 5 monclient: authenticate success, global_id 34398 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 10 monclient: _renew_subs 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca4084e70 con 0x7f9ca408b7f0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 10 monclient: _renew_subs 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:24.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9ca4153270 con 0x7f9ca408b7f0 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c980039c0 con 0x7f9ca408b7f0 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c98003d20 con 0x7f9ca408b7f0 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 10 monclient: dump: 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.355 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca17fa640 4 set_mon_vals no callback set 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9ca405e360 con 0x7f9ca407b4f0 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9c98003ff0 con 0x7f9ca408b7f0 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(393..393 src has 112..393) ==== 3975+0+0 (unknown 2931935947 0 0) 0x7f9c9804c7b0 con 0x7f9ca408b7f0 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca37fe640 10 monclient: get_auth_request con 0x7f9c7403d450 auth_method 0 2026-02-21T22:23:24.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.343+0000 7f9ca27fc640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b4f0 legacy=0x7f9ca404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734337273 0 0) 0x7f9ca405e360 con 0x7f9ca407b4f0 2026-02-21T22:23:24.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 10 monclient(hunting): my global_id is 54202 2026-02-21T22:23:24.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.791+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca4083230 con 0x7f9ca407b4f0 2026-02-21T22:23:24.793 INFO:tasks.rados.rados.0.trial178.stdout:1871: done (3 left) 2026-02-21T22:23:24.794 INFO:tasks.rados.rados.0.trial178.stdout:1872: done (2 left) 2026-02-21T22:23:24.794 INFO:tasks.rados.rados.0.trial178.stdout:1873: done (1 left) 2026-02-21T22:23:24.794 INFO:tasks.rados.rados.0.trial178.stdout:1874: done (0 left) 2026-02-21T22:23:24.794 INFO:tasks.rados.rados.0.trial178.stdout:1875: copy_from oid 45 from oid 9 current snap is 186 2026-02-21T22:23:24.794 INFO:tasks.rados.rados.0.trial178.stdout:1876: delete oid 11 current snap is 186 2026-02-21T22:23:24.795 INFO:tasks.rados.rados.0.trial178.stdout:1876: done (1 left) 2026-02-21T22:23:24.795 INFO:tasks.rados.rados.0.trial178.stdout:1877: snap_remove snap 185 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1470452344 0 0) 0x7f9ca4083230 con 0x7f9ca407b4f0 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ca405e360 con 0x7f9ca407b4f0 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c8c002d90 con 0x7f9ca407b4f0 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 10 monclient: dump: 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.955 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2800822732 0 0) 0x7f9ca405e360 con 0x7f9ca407b4f0 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ca40008e0 con 0x7f9ca407b4f0 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f9c8c002fc0 con 0x7f9ca407b4f0 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b4f0 legacy=0x7f9ca404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.955+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b4f0 legacy=0x7f9ca404d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.959+0000 7f9ca99fa640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:24.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.959+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9ca404d990 con 0x7f9ca407b8c0 2026-02-21T22:23:24.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca27fc640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b8c0 legacy=0x7f9ca4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:24.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814980595 0 0) 0x7f9ca404d990 con 0x7f9ca407b8c0 2026-02-21T22:23:24.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient(hunting): my global_id is 54205 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca405e360 con 0x7f9ca407b8c0 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3269692960 0 0) 0x7f9ca405e360 con 0x7f9ca407b8c0 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ca404d990 con 0x7f9ca407b8c0 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c8c004c80 con 0x7f9ca407b8c0 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient: dump: 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:24.964 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4166264632 0 0) 0x7f9ca404d990 con 0x7f9ca407b8c0 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ca4090810 con 0x7f9ca407b8c0 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4681 (unknown 0 0 1099580185) 0x7f9c8c004f80 con 0x7f9ca407b8c0 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b8c0 legacy=0x7f9ca4198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9c7bfff640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6789/0 conn(0x7f9ca407b8c0 legacy=0x7f9ca4198250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 108609, 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:24.965 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.966 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:24.967 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:24.968 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.969 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.970 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9c7403d450 legacy=0x7f9c7405d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 10 monclient: shutdown 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 >> v1:10.20.193.102:6789/0 conn(0x7f9ca408b7f0 legacy=0x7f9ca4084760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca37fe640 1 -- 10.20.193.102:0/3222584108 reap_dead start 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca37fe640 1 -- 10.20.193.102:0/3222584108 reap_dead start 2026-02-21T22:23:24.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 shutdown_connections 2026-02-21T22:23:24.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 >> 10.20.193.102:0/3222584108 conn(0x7f9ca4091a50 msgr2=0x7f9ca4089890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:24.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 shutdown_connections 2026-02-21T22:23:24.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:24.963+0000 7f9ca99fa640 1 -- 10.20.193.102:0/3222584108 wait complete. 2026-02-21T22:23:24.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:24.978 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:23:24.978 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 10 monclient: get_monmap_and_config 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 10 monclient: build_initial_monmap 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 10 monclient: monmap: 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:25.039827+0000 2026-02-21T22:23:25.038 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:25.039827+0000 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:25.039 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 1 Processor -- start 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 1 -- start start 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 10 monclient: init 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.035+0000 7fa456d57640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-a con 0x7fa450088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-c con 0x7fa45008a1d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-b con 0x7fa450089860 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa450085920 con 0x7fa450088d80 2026-02-21T22:23:25.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45005e780 con 0x7fa45008a1d0 2026-02-21T22:23:25.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa450086330 con 0x7fa450089860 2026-02-21T22:23:25.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:25.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient(hunting): authenticate will time out at 1614.276097s 2026-02-21T22:23:25.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa454acc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa450088d80 0x7fa450089150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40936/0 (socket says 10.20.193.102:40936) 2026-02-21T22:23:25.042 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa454acc640 1 -- 10.20.193.102:0/735372050 learned_addr learned my addr 10.20.193.102:0/735372050 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa454acc640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.102:6789/0 conn(0x7fa450088d80 legacy=0x7fa450089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4552cd640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.102:6790/0 conn(0x7fa450089860 legacy=0x7fa4501a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744576820 0 0) 0x7fa450085920 con 0x7fa450088d80 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _negotiate 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): my global_id is 34401 2026-02-21T22:23:25.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa43c0058b0 con 0x7fa450088d80 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143617097 0 0) 0x7fa450086330 con 0x7fa450089860 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _negotiate 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient(hunting): my global_id is 74114 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa450085920 con 0x7fa450089860 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa447fff640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.178:6789/0 conn(0x7fa45008a1d0 legacy=0x7fa450180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4025134334 0 0) 0x7fa43c0058b0 con 0x7fa450088d80 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.102:6790/0 conn(0x7fa450089860 legacy=0x7fa4501a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.178:6789/0 conn(0x7fa45008a1d0 legacy=0x7fa450180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: _finish_hunting 0 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 monclient: found mon.noname-a 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45008c6b0 con 0x7fa450088d80 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: _finish_auth 0 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: _check_auth_tickets 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa440003110 con 0x7fa450088d80 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 5 monclient: authenticate success, global_id 34401 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:25.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: dump: 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:25.050 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa440003450 con 0x7fa450088d80 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 1 -- 10.20.193.102:0/735372050 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa440004b50 con 0x7fa450088d80 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa4477fe640 10 monclient: dump: 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.051 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 4 set_mon_vals no callback set 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient: shutdown 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- 10.20.193.102:0/735372050 >> v1:10.20.193.102:6789/0 conn(0x7fa450088d80 legacy=0x7fa450089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- 10.20.193.102:0/735372050 shutdown_connections 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- 10.20.193.102:0/735372050 >> 10.20.193.102:0/735372050 conn(0x7fa450091a50 msgr2=0x7fa450091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:25.052 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- 10.20.193.102:0/735372050 shutdown_connections 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 1 -- 10.20.193.102:0/735372050 wait complete. 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient: build_initial_monmap 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.039+0000 7fa456d57640 10 monclient: monmap: 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:25.053 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 1 Processor -- start 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 1 -- start start 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient: init 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-c con 0x7fa450088d80 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-a con 0x7fa450089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient(hunting): picked mon.noname-b con 0x7fa45008a1d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa450080c00 con 0x7fa450089860 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45005e350 con 0x7fa45008a1d0 2026-02-21T22:23:25.054 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa450088a70 con 0x7fa450088d80 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa456d57640 10 monclient(hunting): authenticate will time out at 1614.280653s 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa454acc640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa450088d80 0x7fa450084a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38150/0 (socket says 10.20.193.102:38150) 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa454acc640 1 -- 10.20.193.102:0/1039886383 learned_addr learned my addr 10.20.193.102:0/1039886383 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa454acc640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.102:6790/0 conn(0x7fa450088d80 legacy=0x7fa450084a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa447fff640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.102:6789/0 conn(0x7fa450089860 legacy=0x7fa45011d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3676645660 0 0) 0x7fa450088a70 con 0x7fa450088d80 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:25.055 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): my global_id is 74117 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa420003620 con 0x7fa450088d80 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367336650 0 0) 0x7fa450080c00 con 0x7fa450089860 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 10 monclient(hunting): my global_id is 34404 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa450088a70 con 0x7fa450089860 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4552cd640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.178:6789/0 conn(0x7fa45008a1d0 legacy=0x7fa45011f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:25.056 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.043+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 558643412 0 0) 0x7fa420003620 con 0x7fa450088d80 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa450080c00 con 0x7fa450088d80 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2789670531 0 0) 0x7fa450088a70 con 0x7fa450089860 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa420003620 con 0x7fa450089860 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428191396 0 0) 0x7fa45005e350 con 0x7fa45008a1d0 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:25.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): my global_id is 54211 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa450088a70 con 0x7fa45008a1d0 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa440002b70 con 0x7fa450088d80 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): dump: 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.060 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa438003380 con 0x7fa450089860 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient(hunting): dump: 2026-02-21T22:23:25.061 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 199438515 0 0) 0x7fa450080c00 con 0x7fa450088d80 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.178:6789/0 conn(0x7fa45008a1d0 legacy=0x7fa45011f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.102:6789/0 conn(0x7fa450089860 legacy=0x7fa45011d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: _finish_hunting 0 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 monclient: found mon.c 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa450081220 con 0x7fa450088d80 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: _finish_auth 0 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: _check_auth_tickets 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 5 monclient: authenticate success, global_id 74117 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _renew_subs 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa450083580 con 0x7fa450088d80 2026-02-21T22:23:25.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _renew_subs 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa450081d80 con 0x7fa450088d80 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa440003290 con 0x7fa450088d80 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa440005170 con 0x7fa450088d80 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 10 monclient: dump: 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.064 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa446ffd640 4 set_mon_vals no callback set 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4500008e0 con 0x7fa450088d80 2026-02-21T22:23:25.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa440011800 con 0x7fa450088d80 2026-02-21T22:23:25.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(394..394 src has 112..394) ==== 3947+0+0 (unknown 1286226823 0 0) 0x7fa44004c630 con 0x7fa450088d80 2026-02-21T22:23:25.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.047+0000 7fa447fff640 10 monclient: get_auth_request con 0x7fa42003d820 auth_method 0 2026-02-21T22:23:25.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.051+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fa440017680 con 0x7fa450088d80 2026-02-21T22:23:25.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.051+0000 7fa4457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:25.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.051+0000 7fa4457fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:25.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.167+0000 7fa456d57640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:23:25.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.167+0000 7fa456d57640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:23:25.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.167+0000 7fa456d57640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:25.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.167+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa450090810 con 0x7fa450088d80 2026-02-21T22:23:25.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa4457fa640 1 -- 10.20.193.102:0/1039886383 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7fa440016f60 con 0x7fa450088d80 2026-02-21T22:23:25.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa4457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:23:25.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa4457fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:25.180 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.181 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=10,logm=61} crc {auth=687427969,config=1618278320,health=2872322161,kv=3483615117,logm=3698183590}) 2026-02-21T22:23:25.181 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=1,health=11,kv=11,logm=63} crc {auth=3320316033,config=2457401934,health=3910066772,kv=2407103666,logm=2800385566}) 2026-02-21T22:23:25.181 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.181 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=10,logm=61} crc {auth=687427969,config=1618278320,health=2872322161,kv=3483615117,logm=3698183590}) 2026-02-21T22:23:25.181 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=13,config=2,health=13,kv=12,logm=60} crc {auth=3049958568,config=1618278320,health=2716147082,kv=1080533729,logm=1886775552}) 2026-02-21T22:23:25.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa42003d820 legacy=0x7fa42005dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 10 monclient: shutdown 2026-02-21T22:23:25.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:25.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 >> v1:10.20.193.102:6790/0 conn(0x7fa450088d80 legacy=0x7fa450084a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:25.182 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.183 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=461617200}) 2026-02-21T22:23:25.183 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3727832123}) 2026-02-21T22:23:25.183 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.183 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=461617200}) 2026-02-21T22:23:25.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 shutdown_connections 2026-02-21T22:23:25.184 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3942835488}) 2026-02-21T22:23:25.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 >> 10.20.193.102:0/1039886383 conn(0x7fa450091a50 msgr2=0x7fa4500575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:25.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 shutdown_connections 2026-02-21T22:23:25.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:25.179+0000 7fa456d57640 1 -- 10.20.193.102:0/1039886383 wait complete. 2026-02-21T22:23:25.185 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.185 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302894110}) 2026-02-21T22:23:25.185 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1565838869}) 2026-02-21T22:23:25.185 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.186 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=302894110}) 2026-02-21T22:23:25.186 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.179+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3384645865}) 2026-02-21T22:23:25.192 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:25.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:25.552 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:23:25.553 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:23:25.555 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:25.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:25.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:23:25.716 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.716 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381243182}) 2026-02-21T22:23:25.716 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2782419728}) 2026-02-21T22:23:25.716 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.716 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381243182}) 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3378836260}) 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1926565258}) 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=370818512}) 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1926565258}) 2026-02-21T22:23:25.717 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2978277141}) 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328932891}) 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1583992938}) 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2328932891}) 2026-02-21T22:23:25.718 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.715+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2438757417}) 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2480277537}) 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1332612126}) 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2480277537}) 2026-02-21T22:23:25.719 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=678622410}) 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162276531}) 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1127853967}) 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3162276531}) 2026-02-21T22:23:25.720 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3948500940}) 2026-02-21T22:23:25.721 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.721 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3035079420}) 2026-02-21T22:23:25.721 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1589035565}) 2026-02-21T22:23:25.721 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.721 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3035079420}) 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1056040357}) 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1217433578}) 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=756398381}) 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1217433578}) 2026-02-21T22:23:25.722 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:25.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4112277649}) 2026-02-21T22:23:26.193 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:23:26.193 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941223983}) 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=398168680}) 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3941223983}) 2026-02-21T22:23:26.253 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.251+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4157693013}) 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=580813260}) 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1642119649}) 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=580813260}) 2026-02-21T22:23:26.258 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.255+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2882849431}) 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: get_monmap_and_config 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: monmap: 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:26.266353+0000 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:26.266353+0000 2026-02-21T22:23:26.264 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.265 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.265 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.265 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:26.265 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:26.265 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 1 Processor -- start 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 1 -- start start 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: init 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c086a10 con 0x7f2a3c08ce00 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c08b130 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c08bd60 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a44453640 10 monclient(hunting): authenticate will time out at 1615.502275s 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a429c9640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08d8e0 0x7f2a3c1a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38182/0 (socket says 10.20.193.102:38182) 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.263+0000 7f2a429c9640 1 -- 10.20.193.102:0/2392946747 learned_addr learned my addr 10.20.193.102:0/2392946747 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a429c9640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a419c7640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a421c8640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2174143636 0 0) 0x7f2a3c086a10 con 0x7f2a3c08ce00 2026-02-21T22:23:26.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient(hunting): my global_id is 34407 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a24003620 con 0x7f2a3c08ce00 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976598198 0 0) 0x7f2a24003620 con 0x7f2a3c08ce00 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 monclient: found mon.noname-a 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c08b370 con 0x7f2a3c08ce00 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a2c003160 con 0x7f2a3c08ce00 2026-02-21T22:23:26.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 5 monclient: authenticate success, global_id 34407 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: dump: 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a2c003430 con 0x7f2a3c08ce00 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 1 -- 10.20.193.102:0/2392946747 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a2c0038c0 con 0x7f2a3c08ce00 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a411c6640 10 monclient: dump: 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.273 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 4 set_mon_vals no callback set 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 10 monclient: shutdown 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 1 -- 10.20.193.102:0/2392946747 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 1 -- 10.20.193.102:0/2392946747 shutdown_connections 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 1 -- 10.20.193.102:0/2392946747 >> 10.20.193.102:0/2392946747 conn(0x7f2a3c091a50 msgr2=0x7f2a3c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 1 -- 10.20.193.102:0/2392946747 shutdown_connections 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 1 -- 10.20.193.102:0/2392946747 wait complete. 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.267+0000 7f2a44453640 10 monclient: monmap: 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:26.274 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:26.275 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 1 Processor -- start 2026-02-21T22:23:26.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 1 -- start start 2026-02-21T22:23:26.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient: init 2026-02-21T22:23:26.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-a con 0x7f2a3c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-c con 0x7f2a3c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient(hunting): picked mon.noname-b con 0x7f2a3c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c0834a0 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c089bd0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c058940 con 0x7f2a3c08ce00 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a421c8640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 0x7f2a3c088db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40986/0 (socket says 10.20.193.102:40986) 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a421c8640 1 -- 10.20.193.102:0/2542657477 learned_addr learned my addr 10.20.193.102:0/2542657477 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a44453640 10 monclient(hunting): authenticate will time out at 1615.508719s 2026-02-21T22:23:26.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a421c8640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a419c7640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c0894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631869449 0 0) 0x7f2a3c0834a0 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): my global_id is 34410 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a429c9640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a0c003620 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a421c8640 1 --1- 10.20.193.102:0/2542657477 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 0x7f2a3c088db0 :-1 s=OPENED pgs=329 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f2a0c003620 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2192979833 0 0) 0x7f2a0c003620 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a3c0834a0 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a2c003710 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.282 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.271+0000 7f2a22ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006095958 0 0) 0x7f2a3c089bd0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): my global_id is 54214 2026-02-21T22:23:26.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a0c003620 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=775118602}) 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1456408330}) 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=775118602}) 2026-02-21T22:23:26.285 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.275+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2693653702}) 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1877: done (1 left) 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1878: read oid 40 snap -1 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1878: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1879: rollback oid 21 current snap is 186 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 182 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1875: finishing copy_from to trial17818531-45 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1875: got expected ENOENT (src dne) 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1875: finishing copy_from racing read to trial17818531-45 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1875: done (2 left) 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1880: write oid 49 current snap is 186 2026-02-21T22:23:26.286 INFO:tasks.rados.rados.0.trial178.stdout:1880: seq_num 663 ranges {743041=412798,1930738=616566,3040503=102538} 2026-02-21T22:23:26.287 INFO:tasks.rados.rados.0.trial178.stdout:1880: writing trial17818531-49 from 743041 to 1155839 tid 1 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 722881365 0 0) 0x7f2a0c003620 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a3c089bd0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a38004810 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:26.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.290 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1597966803 0 0) 0x7f2a3c089bd0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.102:6790/0 conn(0x7f2a3c08ce00 legacy=0x7f2a3c0894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.102:6789/0 conn(0x7f2a3c08b7f0 legacy=0x7f2a3c088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 monclient: found mon.b 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c05e0e0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1694264004 0 0) 0x7f2a3c0834a0 con 0x7f2a3c08b7f0 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 5 monclient: authenticate success, global_id 54214 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _renew_subs 2026-02-21T22:23:26.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c0831c0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _renew_subs 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2a3c084f60 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a38004a70 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a38006250 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 10 monclient: dump: 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.295 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a409c5640 4 set_mon_vals no callback set 2026-02-21T22:23:26.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2a380064a0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:26.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7f2a3804d9d0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3c0008e0 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.275+0000 7f2a421c8640 10 monclient: get_auth_request con 0x7f2a0c03d650 auth_method 0 2026-02-21T22:23:26.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.283+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f2a38018940 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.283+0000 7f2a22ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.283+0000 7f2a22ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:26.300 INFO:tasks.rados.rados.0.trial178.stdout:1880: writing trial17818531-49 from 1930738 to 2547304 tid 2 2026-02-21T22:23:26.300 INFO:tasks.rados.rados.0.trial178.stdout:1880: writing trial17818531-49 from 3040503 to 3143041 tid 3 2026-02-21T22:23:26.300 INFO:tasks.rados.rados.0.trial178.stdout:1881: read oid 10 snap 182 2026-02-21T22:23:26.300 INFO:tasks.rados.rados.0.trial178.stdout:1881: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1882: read oid 13 snap -1 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1882: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1883: read oid 17 snap -1 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1883: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1884: read oid 50 snap -1 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1884: expect (ObjNum 632 snap 179 seq_num 632) 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1885: copy_from oid 32 from oid 33 current snap is 186 2026-02-21T22:23:26.301 INFO:tasks.rados.rados.0.trial178.stdout:1879: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:23:26.302 INFO:tasks.rados.rados.0.trial178.stdout:1879: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:23:26.302 INFO:tasks.rados.rados.0.trial178.stdout:1879: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:23:26.302 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1081 (ObjNum 615 snap 173 seq_num 615) dirty exists 2026-02-21T22:23:26.302 INFO:tasks.rados.rados.0.trial178.stdout:1880: finishing write tid 1 to trial17818531-49 2026-02-21T22:23:26.302 INFO:tasks.rados.rados.0.trial178.stdout:1878: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:26.367 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.377 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.378 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1567386303}) 2026-02-21T22:23:26.378 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4106100219}) 2026-02-21T22:23:26.378 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.378 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1567386303}) 2026-02-21T22:23:26.378 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.375+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1223016604}) 2026-02-21T22:23:26.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:23:26.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:26.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:26.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 --> v1:10.20.193.178:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2a3c090810 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a22ffd640 1 -- 10.20.193.102:0/2542657477 <== mon.1 v1:10.20.193.178:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1227047113) 0x7f2a38039e20 con 0x7f2a3c08d8e0 2026-02-21T22:23:26.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a22ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:26.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a22ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:26.400 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3892390332}) 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=71012591}) 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3892390332}) 2026-02-21T22:23:26.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.399+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=635783284}) 2026-02-21T22:23:26.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.178:6800/2549949439 conn(0x7f2a0c03d650 legacy=0x7f2a0c05db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 10 monclient: shutdown 2026-02-21T22:23:26.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 >> v1:10.20.193.178:6789/0 conn(0x7f2a3c08d8e0 legacy=0x7f2a3c195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 shutdown_connections 2026-02-21T22:23:26.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 >> 10.20.193.102:0/2542657477 conn(0x7f2a3c091a50 msgr2=0x7f2a3c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 shutdown_connections 2026-02-21T22:23:26.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.399+0000 7f2a44453640 1 -- 10.20.193.102:0/2542657477 wait complete. 2026-02-21T22:23:26.412 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:23:26.412 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:23:26.412 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:23:26.412 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=825736957}) 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3965103687}) 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=825736957}) 2026-02-21T22:23:26.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.419+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1105393861}) 2026-02-21T22:23:26.424 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: get_monmap_and_config 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: monmap: 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:26.474374+0000 2026-02-21T22:23:26.473 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:26.474374+0000 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 1 Processor -- start 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 1 -- start start 2026-02-21T22:23:26.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: init 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-c con 0x7f982c13fb80 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-b con 0x7f982c13ff50 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f982c1855d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c093e30 con 0x7f982c1855d0 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c092470 con 0x7f982c13fb80 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c056470 con 0x7f982c13ff50 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9830d4a640 10 monclient(hunting): authenticate will time out at 1615.710786s 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9829d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f982c13ff50 0x7f982c183eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38194/0 (socket says 10.20.193.102:38194) 2026-02-21T22:23:26.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.471+0000 7f9829d74640 1 -- 10.20.193.102:0/1436878618 learned_addr learned my addr 10.20.193.102:0/1436878618 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829d74640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.102:6790/0 conn(0x7f982c13ff50 legacy=0x7f982c183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f982ad76640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.102:6789/0 conn(0x7f982c1855d0 legacy=0x7f982c1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878007445 0 0) 0x7f982c056470 con 0x7f982c13ff50 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): my global_id is 74120 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9818003620 con 0x7f982c13ff50 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955166897 0 0) 0x7f982c093e30 con 0x7f982c1855d0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient(hunting): my global_id is 34413 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c056470 con 0x7f982c1855d0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f982a575640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.178:6789/0 conn(0x7f982c13fb80 legacy=0x7f982c058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707884363 0 0) 0x7f9818003620 con 0x7f982c13ff50 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.178:6789/0 conn(0x7f982c13fb80 legacy=0x7f982c058530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.102:6789/0 conn(0x7f982c1855d0 legacy=0x7f982c1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 monclient: found mon.noname-b 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c057510 con 0x7f982c13ff50 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9814003110 con 0x7f982c13ff50 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: dump: 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.476 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 5 monclient: authenticate success, global_id 74120 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829d74640 1 --1- 10.20.193.102:0/1436878618 >> v1:10.20.193.102:6790/0 conn(0x7f982c13ff50 0x7f982c183eb0 :-1 s=OPENED pgs=351 gs=2 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f9814004b50 mon_map magic: 0 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f98140033e0 con 0x7f982c13ff50 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 4 set_mon_vals no callback set 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: shutdown 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- 10.20.193.102:0/1436878618 >> v1:10.20.193.102:6790/0 conn(0x7f982c13ff50 legacy=0x7f982c183eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 1 -- 10.20.193.102:0/1436878618 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9814004b50 con 0x7f982c13ff50 2026-02-21T22:23:26.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829573640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- 10.20.193.102:0/1436878618 shutdown_connections 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- 10.20.193.102:0/1436878618 >> 10.20.193.102:0/1436878618 conn(0x7f982c096bc0 msgr2=0x7f982c092a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- 10.20.193.102:0/1436878618 shutdown_connections 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- 10.20.193.102:0/1436878618 wait complete. 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: monmap: 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 Processor -- start 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- start start 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: init 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-b con 0x7f982c13fb80 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-a con 0x7f982c13ff50 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient(hunting): picked mon.noname-c con 0x7f982c1855d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c10ca20 con 0x7f982c13ff50 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c12bf40 con 0x7f982c13fb80 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f982c059240 con 0x7f982c1855d0 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9830d4a640 10 monclient(hunting): authenticate will time out at 1615.714723s 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f982c13ff50 0x7f982c12b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:40998/0 (socket says 10.20.193.102:40998) 2026-02-21T22:23:26.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.475+0000 7f9829d74640 1 -- 10.20.193.102:0/1808671088 learned_addr learned my addr 10.20.193.102:0/1808671088 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9829d74640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c13ff50 legacy=0x7f982c12b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f982ad76640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6790/0 conn(0x7f982c1855d0 legacy=0x7f982c184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452479496 0 0) 0x7f982c10ca20 con 0x7f982c13ff50 2026-02-21T22:23:26.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): my global_id is 34416 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97f8003620 con 0x7f982c13ff50 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f982a575640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.178:6789/0 conn(0x7f982c13fb80 legacy=0x7f982c12b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4281539359 0 0) 0x7f982c059240 con 0x7f982c1855d0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): my global_id is 74123 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c10ca20 con 0x7f982c1855d0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2179927379 0 0) 0x7f97f8003620 con 0x7f982c13ff50 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f982c059240 con 0x7f982c13ff50 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9814002f60 con 0x7f982c13ff50 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239665488 0 0) 0x7f982c12bf40 con 0x7f982c13fb80 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): my global_id is 54220 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97f8003620 con 0x7f982c13fb80 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 853991263 0 0) 0x7f982c10ca20 con 0x7f982c1855d0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f982c12bf40 con 0x7f982c1855d0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f981c0043d0 con 0x7f982c1855d0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:26.481 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3786058803 0 0) 0x7f982c059240 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6790/0 conn(0x7f982c1855d0 legacy=0x7f982c184c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.178:6789/0 conn(0x7f982c13fb80 legacy=0x7f982c12b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 monclient: found mon.a 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c08f580 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 5 monclient: authenticate success, global_id 34416 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 10 monclient: _renew_subs 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f982c08e090 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 10 monclient: _renew_subs 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f982c08edc0 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f98140032b0 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9814005390 con 0x7f982c13ff50 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 10 monclient: dump: 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9828d72640 4 set_mon_vals no callback set 2026-02-21T22:23:26.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f982c095010 con 0x7f982c128080 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9814011a20 con 0x7f982c13ff50 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7f981404c840 con 0x7f982c13ff50 2026-02-21T22:23:26.483 INFO:tasks.rados.rados.0.trial178.stdout:1880: finishing write tid 2 to trial17818531-49 2026-02-21T22:23:26.483 INFO:tasks.rados.rados.0.trial178.stdout:1882: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.479+0000 7f982ad76640 10 monclient: get_auth_request con 0x7f97f803d880 auth_method 0 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f982a575640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c128080 legacy=0x7f982c088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036725182 0 0) 0x7f982c095010 con 0x7f982c128080 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient(hunting): my global_id is 34419 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c059240 con 0x7f982c128080 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1725314554 0 0) 0x7f982c059240 con 0x7f982c128080 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f982c095010 con 0x7f982c128080 2026-02-21T22:23:26.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9820003210 con 0x7f982c128080 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient: dump: 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 834578066 0 0) 0x7f982c095010 con 0x7f982c128080 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f982c128450 con 0x7f982c128080 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f98200058c0 con 0x7f982c128080 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c128080 legacy=0x7f982c088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.483+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c128080 legacy=0x7f982c088de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:26.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f982c051070 con 0x7f982c08d6c0 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f982a575640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c08d6c0 legacy=0x7f982c0894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977170054 0 0) 0x7f982c051070 con 0x7f982c08d6c0 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient(hunting): my global_id is 34422 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f982c095010 con 0x7f982c08d6c0 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1694665120 0 0) 0x7f982c095010 con 0x7f982c08d6c0 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f982c051070 con 0x7f982c08d6c0 2026-02-21T22:23:26.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9820003ab0 con 0x7f982c08d6c0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient: dump: 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 150659024 0 0) 0x7f982c051070 con 0x7f982c08d6c0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f982c1403b0 con 0x7f982c08d6c0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4516 (unknown 0 0 2405139196) 0x7f9820005b20 con 0x7f982c08d6c0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c08d6c0 legacy=0x7f982c0894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9812ffd640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c08d6c0 legacy=0x7f982c0894f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.490 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 355356, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:26.491 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:26.492 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.493 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.494 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.495 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.178:6800/2549949439 conn(0x7f97f803d880 legacy=0x7f97f805dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 10 monclient: shutdown 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 >> v1:10.20.193.102:6789/0 conn(0x7f982c13ff50 legacy=0x7f982c12b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f982ad76640 1 -- 10.20.193.102:0/1808671088 reap_dead start 2026-02-21T22:23:26.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f982ad76640 1 -- 10.20.193.102:0/1808671088 reap_dead start 2026-02-21T22:23:26.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 shutdown_connections 2026-02-21T22:23:26.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 >> 10.20.193.102:0/1808671088 conn(0x7f982c096bc0 msgr2=0x7f982c141750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.487+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 shutdown_connections 2026-02-21T22:23:26.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.491+0000 7f9830d4a640 1 -- 10.20.193.102:0/1808671088 wait complete. 2026-02-21T22:23:26.501 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:26.501 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:23:26.501 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:23:26.562 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1502020 2026-02-21T22:23:26.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: get_monmap_and_config 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: monmap: 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:26.608077+0000 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:26.608077+0000 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 1 Processor -- start 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 1 -- start start 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: init 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:23:26.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-c con 0x7f76d008e6c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-a con 0x7f76d0059100 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-b con 0x7f76d008f1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d00904a0 con 0x7f76d0059100 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d0090260 con 0x7f76d008e6c0 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d0058820 con 0x7f76d008f1a0 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.603+0000 7f76d5f90640 10 monclient(hunting): authenticate will time out at 1615.843982s 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ceffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 0x7f76d0184140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41016/0 (socket says 10.20.193.102:41016) 2026-02-21T22:23:26.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ceffd640 1 -- 10.20.193.102:0/2576490564 learned_addr learned my addr 10.20.193.102:0/2576490564 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ceffd640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 legacy=0x7f76d0184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76cffff640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.102:6790/0 conn(0x7f76d008f1a0 legacy=0x7f76d01a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138196999 0 0) 0x7f76d00904a0 con 0x7f76d0059100 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): my global_id is 34425 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76cf7fe640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.178:6789/0 conn(0x7f76d008e6c0 legacy=0x7f76d008ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76b4003620 con 0x7f76d0059100 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158328290 0 0) 0x7f76d0058820 con 0x7f76d008f1a0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): my global_id is 74126 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d00904a0 con 0x7f76d008f1a0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128163260 0 0) 0x7f76bc002cb0 con 0x7f76d008e6c0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient(hunting): my global_id is 54223 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d0058820 con 0x7f76d008e6c0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 773595691 0 0) 0x7f76b4003620 con 0x7f76d0059100 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.102:6790/0 conn(0x7f76d008f1a0 legacy=0x7f76d01a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.178:6789/0 conn(0x7f76d008e6c0 legacy=0x7f76d008ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 monclient: found mon.noname-a 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d0057f80 con 0x7f76d0059100 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 5 monclient: authenticate success, global_id 34425 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76c0003110 con 0x7f76d0059100 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: dump: 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f76c00033e0 con 0x7f76d0059100 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 1 -- 10.20.193.102:0/2576490564 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76c0004b50 con 0x7f76d0059100 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76ce7fc640 10 monclient: dump: 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.611 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 4 set_mon_vals no callback set 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 10 monclient: shutdown 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 1 -- 10.20.193.102:0/2576490564 >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 legacy=0x7f76d0184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 1 -- 10.20.193.102:0/2576490564 shutdown_connections 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 1 -- 10.20.193.102:0/2576490564 >> 10.20.193.102:0/2576490564 conn(0x7f76d0056b30 msgr2=0x7f76d0093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 1 -- 10.20.193.102:0/2576490564 shutdown_connections 2026-02-21T22:23:26.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 1 -- 10.20.193.102:0/2576490564 wait complete. 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.607+0000 7f76d5f90640 10 monclient: monmap: 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:26.613 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 Processor -- start 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- start start 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: init 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-a con 0x7f76d0059100 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-b con 0x7f76d008e6c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient(hunting): picked mon.noname-c con 0x7f76d008f1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d012bbc0 con 0x7f76d0059100 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d005ea30 con 0x7f76d008e6c0 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d012c0d0 con 0x7f76d008f1a0 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient(hunting): authenticate will time out at 1615.848696s 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cf7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 0x7f76d013ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41028/0 (socket says 10.20.193.102:41028) 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cf7fe640 1 -- 10.20.193.102:0/301425462 learned_addr learned my addr 10.20.193.102:0/301425462 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f76d008f1a0 0x7f76d0195bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38242/0 (socket says 10.20.193.102:38242) 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cffff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d008f1a0 legacy=0x7f76d0195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cf7fe640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 legacy=0x7f76d013ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 146987418 0 0) 0x7f76d012c0d0 con 0x7f76d008f1a0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): my global_id is 74129 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a0003620 con 0x7f76d008f1a0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76ceffd640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.178:6789/0 conn(0x7f76d008e6c0 legacy=0x7f76d0153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099136072 0 0) 0x7f76d012bbc0 con 0x7f76d0059100 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): my global_id is 34428 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d012c0d0 con 0x7f76d0059100 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1627547020 0 0) 0x7f76a0003620 con 0x7f76d008f1a0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76d012bbc0 con 0x7f76d008f1a0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76c4004900 con 0x7f76d008f1a0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): dump: 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.616 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3980094652 0 0) 0x7f76d005ea30 con 0x7f76d008e6c0 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): my global_id is 54226 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a0003620 con 0x7f76d008e6c0 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2483840391 0 0) 0x7f76d012c0d0 con 0x7f76d0059100 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76d005ea30 con 0x7f76d0059100 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76bc003710 con 0x7f76d0059100 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient(hunting): dump: 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.617 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3686775310 0 0) 0x7f76d012bbc0 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.178:6789/0 conn(0x7f76d008e6c0 legacy=0x7f76d0153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6789/0 conn(0x7f76d0059100 legacy=0x7f76d013ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 monclient: found mon.c 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d012e430 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 5 monclient: authenticate success, global_id 74129 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _renew_subs 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f76d013d140 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _renew_subs 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f76d0154120 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f76c4004ed0 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76c4005270 con 0x7f76d008f1a0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76cdffb640 4 set_mon_vals no callback set 2026-02-21T22:23:26.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76abfff640 10 monclient: dump: 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.611+0000 7f76d5f90640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f76d00008e0 con 0x7f76d0092c10 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f76c4005450 con 0x7f76d008f1a0 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7f76c404daf0 con 0x7f76d008f1a0 2026-02-21T22:23:26.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76ceffd640 10 monclient: get_auth_request con 0x7f76a003d8c0 auth_method 0 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76cf7fe640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d0092c10 legacy=0x7f76d0089f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301875352 0 0) 0x7f76d00008e0 con 0x7f76d0092c10 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient(hunting): my global_id is 74132 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d012bbc0 con 0x7f76d0092c10 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2527120008 0 0) 0x7f76d012bbc0 con 0x7f76d0092c10 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76d00008e0 con 0x7f76d0092c10 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76bc003c90 con 0x7f76d0092c10 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient: dump: 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.620 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 632077542 0 0) 0x7f76d00008e0 con 0x7f76d0092c10 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76d00949d0 con 0x7f76d0092c10 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f76bc0058c0 con 0x7f76d0092c10 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d0092c10 legacy=0x7f76d0089f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.615+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d0092c10 legacy=0x7f76d0089f60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.619+0000 7f76d5f90640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:26.621 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.619+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f76d004d650 con 0x7f76d004d280 2026-02-21T22:23:26.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76cf7fe640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d004d280 legacy=0x7f76d01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619731969 0 0) 0x7f76d004d650 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient(hunting): my global_id is 74135 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d00008e0 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2240178064 0 0) 0x7f76d00008e0 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76d004d650 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f76bc004210 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient: dump: 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2724613458 0 0) 0x7f76d004d650 con 0x7f76d004d280 2026-02-21T22:23:26.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f76d004d820 con 0x7f76d004d280 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 1275881029) 0x7f76bc005a70 con 0x7f76d004d280 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d004d280 legacy=0x7f76d01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76abfff640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d004d280 legacy=0x7f76d01982e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 2747, 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:26.625 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.626 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.627 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.628 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.629 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.630 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.178:6800/2549949439 conn(0x7f76a003d8c0 legacy=0x7f76a005dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 10 monclient: shutdown 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 >> v1:10.20.193.102:6790/0 conn(0x7f76d008f1a0 legacy=0x7f76d0195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76cffff640 1 -- 10.20.193.102:0/301425462 reap_dead start 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 shutdown_connections 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 >> 10.20.193.102:0/301425462 conn(0x7f76d0056b30 msgr2=0x7f76d0141ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 shutdown_connections 2026-02-21T22:23:26.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.623+0000 7f76d5f90640 1 -- 10.20.193.102:0/301425462 wait complete. 2026-02-21T22:23:26.633 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:26.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:23:26.633 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:23:26.674 INFO:tasks.rados.rados.0.trial178.stdout:1880: finishing write tid 3 to trial17818531-49 2026-02-21T22:23:26.674 INFO:tasks.rados.rados.0.trial178.stdout:1880: finishing write tid 4 to trial17818531-49 2026-02-21T22:23:26.674 INFO:tasks.rados.rados.0.trial178.stdout:1880: finishing write tid 5 to trial17818531-49 2026-02-21T22:23:26.674 INFO:tasks.rados.rados.0.trial178.stdout:1880: oid 49 updating version 0 to 770 2026-02-21T22:23:26.674 INFO:tasks.rados.rados.0.trial178.stdout:1880: oid 49 updating version 770 to 771 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:1880: oid 49 version 771 is already newer than 769 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:1880: oid 49 updating version 771 to 772 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 772 (ObjNum 663 snap 186 seq_num 663) dirty exists 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:1880: left oid 49 (ObjNum 663 snap 186 seq_num 663) 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:1884: expect (ObjNum 632 snap 179 seq_num 632) 2026-02-21T22:23:26.675 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: get_monmap_and_config 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: monmap: 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:26.702995+0000 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:26.702995+0000 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:26.701 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 1 Processor -- start 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 1 -- start start 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: init 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient(hunting): picked mon.noname-b con 0x7f624808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient(hunting): picked mon.noname-a con 0x7f624808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient(hunting): picked mon.noname-c con 0x7f624808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248086a10 con 0x7f624808b7f0 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624808b130 con 0x7f624808d8e0 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624808bd60 con 0x7f624808ce00 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f624e212640 10 monclient(hunting): authenticate will time out at 1615.939277s 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f62477fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f624808ce00 0x7f624808d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:38286/0 (socket says 10.20.193.102:38286) 2026-02-21T22:23:26.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.699+0000 7f62477fe640 1 -- 10.20.193.102:0/1688168264 learned_addr learned my addr 10.20.193.102:0/1688168264 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62477fe640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.102:6790/0 conn(0x7f624808ce00 legacy=0x7f624808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f6246ffd640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.102:6789/0 conn(0x7f624808b7f0 legacy=0x7f6248181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164207293 0 0) 0x7f624808bd60 con 0x7f624808ce00 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): my global_id is 74138 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f622c003620 con 0x7f624808ce00 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142837234 0 0) 0x7f6248086a10 con 0x7f624808b7f0 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient(hunting): my global_id is 34431 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f624808bd60 con 0x7f624808b7f0 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f6247fff640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.178:6789/0 conn(0x7f624808d8e0 legacy=0x7f62481a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 263695192 0 0) 0x7f622c003620 con 0x7f624808ce00 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.178:6789/0 conn(0x7f624808d8e0 legacy=0x7f62481a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.102:6789/0 conn(0x7f624808b7f0 legacy=0x7f6248181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 monclient: found mon.noname-b 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624808b370 con 0x7f624808ce00 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 5 monclient: authenticate success, global_id 74138 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6234003110 con 0x7f624808ce00 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: dump: 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f62340033e0 con 0x7f624808ce00 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 1 -- 10.20.193.102:0/1688168264 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6234004b50 con 0x7f624808ce00 2026-02-21T22:23:26.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62467fc640 10 monclient: dump: 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 4 set_mon_vals no callback set 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: shutdown 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- 10.20.193.102:0/1688168264 >> v1:10.20.193.102:6790/0 conn(0x7f624808ce00 legacy=0x7f624808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- 10.20.193.102:0/1688168264 shutdown_connections 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- 10.20.193.102:0/1688168264 >> 10.20.193.102:0/1688168264 conn(0x7f6248091a50 msgr2=0x7f6248091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- 10.20.193.102:0/1688168264 shutdown_connections 2026-02-21T22:23:26.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- 10.20.193.102:0/1688168264 wait complete. 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: build_initial_monmap 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: monmap: 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 Processor -- start 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- start start 2026-02-21T22:23:26.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: init 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient(hunting): picked mon.noname-a con 0x7f624808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient(hunting): picked mon.noname-c con 0x7f624808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient(hunting): picked mon.noname-b con 0x7f624808d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248083460 con 0x7f624808b7f0 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62480808e0 con 0x7f624808d8e0 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248081640 con 0x7f624808ce00 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f624e212640 10 monclient(hunting): authenticate will time out at 1615.943859s 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62477fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f624808b7f0 0x7f624807fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41054/0 (socket says 10.20.193.102:41054) 2026-02-21T22:23:26.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.703+0000 7f62477fe640 1 -- 10.20.193.102:0/3709759471 learned_addr learned my addr 10.20.193.102:0/3709759471 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f62477fe640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.102:6789/0 conn(0x7f624808b7f0 legacy=0x7f624807fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6246ffd640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.102:6790/0 conn(0x7f624808ce00 legacy=0x7f62480801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372597800 0 0) 0x7f6248081640 con 0x7f624808ce00 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): my global_id is 74141 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6218003620 con 0x7f624808ce00 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082440927 0 0) 0x7f6248083460 con 0x7f624808b7f0 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): my global_id is 34434 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6248081640 con 0x7f624808b7f0 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6247fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f624808d8e0 legacy=0x7f6248195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2566786745 0 0) 0x7f6218003620 con 0x7f624808ce00 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6248083460 con 0x7f624808ce00 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6238003330 con 0x7f624808ce00 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): dump: 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2229538829 0 0) 0x7f6248081640 con 0x7f624808b7f0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6218003620 con 0x7f624808b7f0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f62340048e0 con 0x7f624808b7f0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): dump: 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422132980 0 0) 0x7f62480808e0 con 0x7f624808d8e0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient(hunting): my global_id is 54232 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6248081640 con 0x7f624808d8e0 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 283051880 0 0) 0x7f6248083460 con 0x7f624808ce00 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f624808d8e0 legacy=0x7f6248195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.102:6789/0 conn(0x7f624808b7f0 legacy=0x7f624807fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: _finish_hunting 0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 monclient: found mon.c 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6248083740 con 0x7f624808ce00 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: _finish_auth 0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: _check_auth_tickets 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 5 monclient: authenticate success, global_id 74141 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 10 monclient: _renew_subs 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6248058ad0 con 0x7f624808ce00 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 10 monclient: _renew_subs 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6248084280 con 0x7f624808ce00 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f62380039c0 con 0x7f624808ce00 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6238003550 con 0x7f624808ce00 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 10 monclient: dump: 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.713 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6245ffb640 4 set_mon_vals no callback set 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f6238003e80 con 0x7f624808ce00 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6248085200 con 0x7f6248084e30 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7f623804c650 con 0x7f624808ce00 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.707+0000 7f6247fff640 10 monclient: get_auth_request con 0x7f621803d880 auth_method 0 2026-02-21T22:23:26.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f62477fe640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f6248084e30 legacy=0x7f624805a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721783113 0 0) 0x7f6248085200 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient(hunting): my global_id is 54235 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6248083460 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4284813483 0 0) 0x7f6248083460 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6248085200 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6234003570 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient: dump: 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1540194435 0 0) 0x7f6248085200 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6248088a70 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f6234004fc0 con 0x7f6248084e30 2026-02-21T22:23:26.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:26.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:26.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f6248084e30 legacy=0x7f624805a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.711+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f6248084e30 legacy=0x7f624805a330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.715+0000 7f624e212640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:26.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.715+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6248059e30 con 0x7f624805aa40 2026-02-21T22:23:26.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f62477fe640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f624805aa40 legacy=0x7f62481986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939988669 0 0) 0x7f6248059e30 con 0x7f624805aa40 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient(hunting): my global_id is 54238 2026-02-21T22:23:26.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6248085200 con 0x7f624805aa40 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 183657185 0 0) 0x7f6248085200 con 0x7f624805aa40 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6248059e30 con 0x7f624805aa40 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f62340055c0 con 0x7f624805aa40 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient: dump: 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:26.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2737089168 0 0) 0x7f6248059e30 con 0x7f624805aa40 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6248062ce0 con 0x7f624805aa40 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4681 (unknown 0 0 1451593877) 0x7f6234006980 con 0x7f624805aa40 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f624805aa40 legacy=0x7f62481986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6223fff640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6789/0 conn(0x7f624805aa40 legacy=0x7f62481986b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 30, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 110365, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:26.723 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.724 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:26.725 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:26.726 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:26.727 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.728 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:26.729 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.178:6800/2549949439 conn(0x7f621803d880 legacy=0x7f621805dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 10 monclient: shutdown 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 >> v1:10.20.193.102:6790/0 conn(0x7f624808ce00 legacy=0x7f62480801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6247fff640 1 -- 10.20.193.102:0/3709759471 reap_dead start 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f6247fff640 1 -- 10.20.193.102:0/3709759471 reap_dead start 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 shutdown_connections 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 >> 10.20.193.102:0/3709759471 conn(0x7f6248091a50 msgr2=0x7f62480575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:26.730 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.730 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.719+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3547116083}) 2026-02-21T22:23:26.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.719+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 shutdown_connections 2026-02-21T22:23:26.731 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1360759069}) 2026-02-21T22:23:26.731 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.731 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:26.723+0000 7f624e212640 1 -- 10.20.193.102:0/3709759471 wait complete. 2026-02-21T22:23:26.731 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1443201 2026-02-21T22:23:26.732 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3547116083}) 2026-02-21T22:23:26.732 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1861216818}) 2026-02-21T22:23:26.732 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=4} crc {logm=3202596653,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1994559070,mon_config_key=3290694530}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=71,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8} crc {logm=513231302,mdsmap=3074481156,mgr=2974333102,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1223489744,mon_config_key=3868534179}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=4} crc {logm=3202596653,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1994559070,mon_config_key=3290694530}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8} crc {logm=2100763276,mdsmap=3074481157,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1994559070,mon_config_key=3868534179}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=76,osdmap=7} crc {mon_config_key=2818177826,monmap=2662863928,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=897875103,osd_snap=3858372265,osdmap=16707852}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=6,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=74,osdmap=8} crc {mon_config_key=3505689774,monmap=2662863928,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=897875103,osd_snap=4001011794,osdmap=3588617803}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=2,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=76,osdmap=7} crc {mon_config_key=2818177826,monmap=2662863928,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=897875103,osd_snap=3858372265,osdmap=16707852}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=4} crc {mon_config_key=2609376702,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=897875103,osd_snap=1490093202,osdmap=3578609481}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=45461706}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3856222621}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=45461706}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=670750987}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4017498663}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3771061922}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4017498663}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.723+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3903899814}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2399491767}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4155658485}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2399491767}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3379536413}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3180772000}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3609646979}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3180772000}) 2026-02-21T22:23:26.733 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:23:26.727+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=74512616}) 2026-02-21T22:23:26.734 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:23:26.734 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-21T22:23:26.734 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2026-02-21T22:23:26.734 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-21T22:23:26.734 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-21T22:23:26.734 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-21T22:23:26.734 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:23:26.745 INFO:tasks.ceph.mon.c:Stopped 2026-02-21T22:23:26.745 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.a 2026-02-21T22:23:26.745 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-21T22:23:26.745 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-21T22:23:26.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:23:26.786 INFO:tasks.rados.rados.0.trial178.stdout:1883: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:23:26.789 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 893406 2026-02-21T22:23:26.811 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2499491 2026-02-21T22:23:26.818 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.840 INFO:tasks.rados.rados.0.trial178.stdout:1881: expect (ObjNum 558 snap 158 seq_num 558) 2026-02-21T22:23:26.840 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.865 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:26.897 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1765328 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1885: finishing copy_from racing read to trial17818531-32 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1885: finishing copy_from to trial17818531-32 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 430 (ObjNum 656 snap 185 seq_num 656) dirty exists 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1878: done (7 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1879: done (6 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1880: done (5 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1881: done (4 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1882: done (3 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1883: done (2 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1884: done (1 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1885: done (0 left) 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1886: rollback oid 9 current snap is 186 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 182 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1887: rollback oid 33 current snap is 186 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 182 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1888: read oid 22 snap 182 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1888: expect deleted 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1889: rollback oid 5 current snap is 186 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 182 2026-02-21T22:23:26.909 INFO:tasks.rados.rados.0.trial178.stdout:1890: read oid 47 snap -1 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1890: expect (ObjNum 653 snap 183 seq_num 653) 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1891: read oid 49 snap -1 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1891: expect (ObjNum 663 snap 186 seq_num 663) 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1892: copy_from oid 2 from oid 34 current snap is 186 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1893: write (excl) oid 42 current snap is 186 2026-02-21T22:23:26.910 INFO:tasks.rados.rados.0.trial178.stdout:1893: seq_num 664 ranges {759848=488047,1989220=760224} 2026-02-21T22:23:26.918 INFO:tasks.rados.rados.0.trial178.stdout:1893: writing trial17818531-42 from 759848 to 1247895 tid 1 2026-02-21T22:23:26.928 INFO:tasks.rados.rados.0.trial178.stdout:1893: writing trial17818531-42 from 1989220 to 2749444 tid 2 2026-02-21T22:23:26.928 INFO:tasks.rados.rados.0.trial178.stdout:1894: write (excl) oid 8 current snap is 186 2026-02-21T22:23:26.929 INFO:tasks.rados.rados.0.trial178.stdout:1894: seq_num 665 ranges {776655=563296,2047702=620235,3095349=81306} 2026-02-21T22:23:26.937 INFO:tasks.rados.rados.0.trial178.stdout:1894: writing trial17818531-8 from 776655 to 1339951 tid 1 2026-02-21T22:23:26.946 INFO:tasks.rados.rados.0.trial178.stdout:1894: writing trial17818531-8 from 2047702 to 2667937 tid 2 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1894: writing trial17818531-8 from 3095349 to 3176655 tid 3 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1895: snap_create 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1889: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1887: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1887: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:23:26.950 INFO:tasks.rados.rados.0.trial178.stdout:1887: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:23:26.951 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 433 (ObjNum 634 snap 180 seq_num 634) dirty exists 2026-02-21T22:23:26.951 INFO:tasks.rados.rados.0.trial178.stdout:1891: expect (ObjNum 663 snap 186 seq_num 663) 2026-02-21T22:23:27.043 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2157092 2026-02-21T22:23:27.096 INFO:tasks.rados.rados.0.trial178.stdout:1890: expect (ObjNum 653 snap 183 seq_num 653) 2026-02-21T22:23:27.096 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:27.175 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 355503 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1892: finishing copy_from racing read to trial17818531-2 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1892: finishing copy_from to trial17818531-2 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 754 (ObjNum 613 snap 172 seq_num 613) dirty exists 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1886: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 0 (ObjNum 33 snap 0 seq_num 99503760) dirty dne 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: finishing write tid 1 to trial17818531-42 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: finishing write tid 2 to trial17818531-42 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: finishing write tid 3 to trial17818531-42 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: finishing write tid 4 to trial17818531-42 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: oid 42 updating version 0 to 1082 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: oid 42 updating version 1082 to 1084 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: oid 42 version 1084 is already newer than 1083 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1084 (ObjNum 664 snap 186 seq_num 664) dirty exists 2026-02-21T22:23:27.217 INFO:tasks.rados.rados.0.trial178.stdout:1893: left oid 42 (ObjNum 664 snap 186 seq_num 664) 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: finishing write tid 1 to trial17818531-8 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: finishing write tid 2 to trial17818531-8 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: finishing write tid 3 to trial17818531-8 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: finishing write tid 4 to trial17818531-8 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: finishing write tid 5 to trial17818531-8 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: oid 8 updating version 0 to 1086 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: oid 8 updating version 1086 to 1088 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: oid 8 version 1088 is already newer than 1085 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: oid 8 version 1088 is already newer than 1087 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1088 (ObjNum 665 snap 186 seq_num 665) dirty exists 2026-02-21T22:23:27.218 INFO:tasks.rados.rados.0.trial178.stdout:1894: left oid 8 (ObjNum 665 snap 186 seq_num 665) 2026-02-21T22:23:28.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:28.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:33.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:33.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:38.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:38.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:41.746 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.a 2026-02-21T22:23:41.746 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-21T22:23:41.746 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-21T22:23:41.746 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:23:41.746 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:23:41.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: get_monmap_and_config 2026-02-21T22:23:41.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: build_initial_monmap 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: monmap: 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:41.819573+0000 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:41.819573+0000 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:41.818 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:41.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 1 Processor -- start 2026-02-21T22:23:41.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 1 -- start start 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: init 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-a con 0x7fabf0084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-c con 0x7fabf0086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-b con 0x7fabf0183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf0086a70 con 0x7fabf0084860 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf008b130 con 0x7fabf0086e90 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf0085340 con 0x7fabf0183350 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf8501640 10 monclient(hunting): authenticate will time out at 1631.055840s 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf6a77640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabf01a3730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf6276640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fabf0084860 0x7fabf0084c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51864/0 (socket says 10.20.193.102:51864) 2026-02-21T22:23:41.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.815+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 learned_addr learned my addr 10.20.193.102:0/2600003745 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:41.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.819+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6789/0 conn(0x7fabf0084860 legacy=0x7fabf0084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:41.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:41.819+0000 7fabf5a75640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabf0181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:42.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:42.019+0000 7fabf6a77640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabf01a3730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:42.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:42.423+0000 7fabf6a77640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabf01a3730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:43.227+0000 7fabf6a77640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabf01a3730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:43.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:43.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:44.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): tick 2026-02-21T22:23:44.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 monclient(hunting): continuing hunt 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabf01a3730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabf0181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6789/0 conn(0x7fabf0084860 legacy=0x7fabf0084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): picked mon.noname-b con 0x7fabf0183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): picked mon.noname-c con 0x7fabf0086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): picked mon.noname-a con 0x7fabf0084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf0086790 con 0x7fabf0084860 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabd00238e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:44.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf0085340 con 0x7fabf0086e90 2026-02-21T22:23:44.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 1 -- 10.20.193.102:0/2600003745 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd006a0e0 con 0x7fabf0183350 2026-02-21T22:23:44.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabd7fff640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:44.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabf6a77640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6789/0 conn(0x7fabf0084860 legacy=0x7fabd00668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:44.821 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:44.819+0000 7fabf5a75640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabd00450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:45.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:45.023+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabd00238e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:45.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:45.427+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabd00238e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:46.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.231+0000 7fabf6276640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabd00238e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:46.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.363+0000 7fabf5a75640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabd00450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 11 2026-02-21T22:23:46.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.363+0000 7fabf5a75640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabd00450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:23:46.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.363+0000 7fabf5a75640 1 --1- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 0x7fabd00450c0 :-1 s=OPENED pgs=300 gs=9 cs=1 l=1).handle_message read tag failed 2026-02-21T22:23:46.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.363+0000 7fabf5a75640 1 --1- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 0x7fabd00450c0 :-1 s=OPENED pgs=300 gs=9 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:23:46.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.363+0000 7fabf5274640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.178:6789/0 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654626434 0 0) 0x7fabf0086790 con 0x7fabf0084860 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient(hunting): _negotiate 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient(hunting): my global_id is 34437 2026-02-21T22:23:46.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabd8003620 con 0x7fabf0084860 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3745892250 0 0) 0x7fabd8003620 con 0x7fabf0084860 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6790/0 conn(0x7fabf0183350 legacy=0x7fabd00238e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.178:6789/0 conn(0x7fabf0086e90 legacy=0x7fabd00450c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: _finish_hunting 0 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 monclient: found mon.noname-a 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd006b440 con 0x7fabf0084860 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: _finish_auth 0 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: _check_auth_tickets 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 5 monclient: authenticate success, global_id 34437 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fabec005a70 con 0x7fabf0084860 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:46.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: dump: 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fabec005d40 con 0x7fabf0084860 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 1 -- 10.20.193.102:0/2600003745 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fabec0060a0 con 0x7fabf0084860 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf5274640 10 monclient: dump: 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:46.799 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 4 set_mon_vals no callback set 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 10 monclient: shutdown 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- 10.20.193.102:0/2600003745 >> v1:10.20.193.102:6789/0 conn(0x7fabf0084860 legacy=0x7fabd00668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- 10.20.193.102:0/2600003745 shutdown_connections 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- 10.20.193.102:0/2600003745 >> 10.20.193.102:0/2600003745 conn(0x7fabf0091a50 msgr2=0x7fabf0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- 10.20.193.102:0/2600003745 shutdown_connections 2026-02-21T22:23:46.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- 10.20.193.102:0/2600003745 wait complete. 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 10 monclient: build_initial_monmap 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 10 monclient: monmap: 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:46.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 Processor -- start 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 1 -- start start 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.795+0000 7fabf8501640 10 monclient: init 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-c con 0x7fabf0084860 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-a con 0x7fabf0086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient(hunting): picked mon.noname-b con 0x7fabf0183350 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf6276640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fabf0084860 legacy=0x7fabf014fc50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf0088a70 con 0x7fabf0086e90 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00820c0 con 0x7fabf0183350 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00823a0 con 0x7fabf0084860 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient(hunting): authenticate will time out at 1636.036401s 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf5a75640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fabf0086e90 0x7fabf0150360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51888/0 (socket says 10.20.193.102:51888) 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf5a75640 1 -- 10.20.193.102:0/294977211 learned_addr learned my addr 10.20.193.102:0/294977211 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf5a75640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.102:6789/0 conn(0x7fabf0086e90 legacy=0x7fabf0150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93513263 0 0) 0x7fabf0085340 con 0x7fabf0086e90 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:46.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): my global_id is 34440 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc0003620 con 0x7fabf0086e90 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf6a77640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.178:6789/0 conn(0x7fabf0183350 legacy=0x7fabf01754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1430153742 0 0) 0x7fabc0003620 con 0x7fabf0086e90 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fabf0085340 con 0x7fabf0086e90 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fabe40033c0 con 0x7fabf0086e90 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): dump: 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:46.808 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971198916 0 0) 0x7fabf00820c0 con 0x7fabf0183350 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient(hunting): my global_id is 54241 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabc0003620 con 0x7fabf0183350 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3000055796 0 0) 0x7fabf0085340 con 0x7fabf0086e90 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.102:6790/0 conn(0x7fabf0084860 legacy=0x7fabf014fc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.178:6789/0 conn(0x7fabf0183350 legacy=0x7fabf01754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 monclient: found mon.a 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf0150a70 con 0x7fabf0086e90 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 5 monclient: authenticate success, global_id 34440 2026-02-21T22:23:46.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _renew_subs 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fabf0080e80 con 0x7fabf0086e90 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _renew_subs 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fabf0176c10 con 0x7fabf0086e90 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fabe4003a30 con 0x7fabf0086e90 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fabe4003d90 con 0x7fabf0086e90 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 10 monclient: dump: 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:46.810 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf4a73640 4 set_mon_vals no callback set 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fabe4004020 con 0x7fabf0086e90 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.799+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf00802e0 con 0x7fabf0086e90 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.803+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7fabe404c760 con 0x7fabf0086e90 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.803+0000 7fabf6276640 10 monclient: get_auth_request con 0x7fabc003d4d0 auth_method 0 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.803+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fabf0088a70 con 0x7fabf0086e90 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.803+0000 7fabd6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:46.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.803+0000 7fabd6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.814 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:46.815 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:23:46.816 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:23:46.817 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:23:46.818 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:23:46.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.919+0000 7fabf8501640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:23:46.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.919+0000 7fabf8501640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:46.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.919+0000 7fabf8501640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:46.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.919+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fabf0062ce0 con 0x7fabf0086e90 2026-02-21T22:23:46.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabd6ffd640 1 -- 10.20.193.102:0/294977211 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 1411863901) 0x7fabe4017730 con 0x7fabf0086e90 2026-02-21T22:23:46.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabd6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:23:46.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabd6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:46.924 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":34,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:23:46.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.178:6800/2549949439 conn(0x7fabc003d4d0 legacy=0x7fabc005d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:46.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 10 monclient: shutdown 2026-02-21T22:23:46.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:46.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 >> v1:10.20.193.102:6789/0 conn(0x7fabf0086e90 legacy=0x7fabf0150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:46.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 shutdown_connections 2026-02-21T22:23:46.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 >> 10.20.193.102:0/294977211 conn(0x7fabf0091a50 msgr2=0x7fabf0087410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:46.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 shutdown_connections 2026-02-21T22:23:46.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:46.923+0000 7fabf8501640 1 -- 10.20.193.102:0/294977211 wait complete. 2026-02-21T22:23:46.934 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:23:46.934 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:23:46.935 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: get_monmap_and_config 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: build_initial_monmap 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: monmap: 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:47.005065+0000 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:47.005065+0000 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:47.003 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 1 Processor -- start 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 1 -- start start 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: init 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-b con 0x7f5874088d80 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-c con 0x7f587408a1d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-a con 0x7f5874089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5874085920 con 0x7f5874089860 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587405e780 con 0x7f587408a1d0 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58737fe640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f5874088d80 legacy=0x7f5874089150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5874086330 con 0x7f5874088d80 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 10 monclient(hunting): authenticate will time out at 1636.241163s 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f5873fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5874089860 0x7f58741a2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51896/0 (socket says 10.20.193.102:51896) 2026-02-21T22:23:47.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f5873fff640 1 -- 10.20.193.102:0/3493489015 learned_addr learned my addr 10.20.193.102:0/3493489015 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f5873fff640 1 -- 10.20.193.102:0/3493489015 >> v1:10.20.193.102:6789/0 conn(0x7f5874089860 legacy=0x7f58741a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f5872ffd640 1 -- 10.20.193.102:0/3493489015 >> v1:10.20.193.178:6789/0 conn(0x7f587408a1d0 legacy=0x7f5874180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265752257 0 0) 0x7f5874085920 con 0x7f5874089860 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): my global_id is 34443 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5858003620 con 0x7f5874089860 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371142885 0 0) 0x7f587405e780 con 0x7f587408a1d0 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient(hunting): my global_id is 54244 2026-02-21T22:23:47.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5874085920 con 0x7f587408a1d0 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106024774 0 0) 0x7f5858003620 con 0x7f5874089860 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 >> v1:10.20.193.102:6790/0 conn(0x7f5874088d80 legacy=0x7f5874089150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 >> v1:10.20.193.178:6789/0 conn(0x7f587408a1d0 legacy=0x7f5874180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: _finish_hunting 0 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 monclient: found mon.noname-a 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587408c6b0 con 0x7f5874089860 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: _finish_auth 0 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: _check_auth_tickets 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5868004620 con 0x7f5874089860 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 5 monclient: authenticate success, global_id 34443 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f58727fc640 10 monclient: dump: 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.008 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.003+0000 7f587a24f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f58680048f0 con 0x7f5874089860 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 1 -- 10.20.193.102:0/3493489015 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5868006060 con 0x7f5874089860 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58727fc640 10 monclient: dump: 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 4 set_mon_vals no callback set 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: shutdown 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- 10.20.193.102:0/3493489015 >> v1:10.20.193.102:6789/0 conn(0x7f5874089860 legacy=0x7f58741a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- 10.20.193.102:0/3493489015 shutdown_connections 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- 10.20.193.102:0/3493489015 >> 10.20.193.102:0/3493489015 conn(0x7f5874091a50 msgr2=0x7f5874091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- 10.20.193.102:0/3493489015 shutdown_connections 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- 10.20.193.102:0/3493489015 wait complete. 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: build_initial_monmap 2026-02-21T22:23:47.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: monmap: 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 Processor -- start 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- start start 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: init 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-a con 0x7f5874088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-b con 0x7f5874089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient(hunting): picked mon.noname-c con 0x7f587408a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587413f640 con 0x7f5874088d80 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587413ccf0 con 0x7f5874089860 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587405dff0 con 0x7f587408a1d0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f587a24f640 10 monclient(hunting): authenticate will time out at 1636.245454s 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f5873fff640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f587408a1d0 legacy=0x7f5874195d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58737fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5874088d80 0x7f587413bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51904/0 (socket says 10.20.193.102:51904) 2026-02-21T22:23:47.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58737fe640 1 -- 10.20.193.102:0/922998002 learned_addr learned my addr 10.20.193.102:0/922998002 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:47.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f58737fe640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f5874088d80 legacy=0x7f587413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396477340 0 0) 0x7f587413f640 con 0x7f5874088d80 2026-02-21T22:23:47.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 10 monclient(hunting): my global_id is 34446 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.007+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5844003620 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 903541180 0 0) 0x7f5844003620 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f5872ffd640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.178:6789/0 conn(0x7f5874089860 legacy=0x7f587413c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f587413f640 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f58600045c0 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): dump: 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2533104764 0 0) 0x7f587413f640 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6790/0 conn(0x7f587408a1d0 legacy=0x7f5874195d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.178:6789/0 conn(0x7f5874089860 legacy=0x7f587413c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 monclient: found mon.a 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58740881e0 con 0x7f5874088d80 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: _finish_auth 0 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:23:47.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 5 monclient: authenticate success, global_id 34446 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 10 monclient: _renew_subs 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f587413fc00 con 0x7f5874088d80 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 10 monclient: _renew_subs 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f587413e140 con 0x7f5874088d80 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5860004890 con 0x7f5874088d80 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5860004c30 con 0x7f5874088d80 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f5871ffb640 4 set_mon_vals no callback set 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: dump: 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f58740008e0 con 0x7f587408cf90 2026-02-21T22:23:47.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5860004f00 con 0x7f5874088d80 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7f586004c780 con 0x7f5874088d80 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f5873fff640 10 monclient: get_auth_request con 0x7f584403d390 auth_method 0 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f5872ffd640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f587408cf90 legacy=0x7f587405a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476791476 0 0) 0x7f58740008e0 con 0x7f587408cf90 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient(hunting): my global_id is 34449 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f587413f640 con 0x7f587408cf90 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3328264132 0 0) 0x7f587413f640 con 0x7f587408cf90 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f58740008e0 con 0x7f587408cf90 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5864003280 con 0x7f587408cf90 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 10 monclient: dump: 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.014 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3531196984 0 0) 0x7f58740008e0 con 0x7f587408cf90 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.011+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5874090810 con 0x7f587408cf90 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f58640058c0 con 0x7f587408cf90 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f584ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f584ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f587408cf90 legacy=0x7f587405a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f587408cf90 legacy=0x7f587405a7c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:47.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f587a24f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:47.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.015+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f58740909c0 con 0x7f5874084e30 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f5872ffd640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f5874084e30 legacy=0x7f587404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852027472 0 0) 0x7f58740909c0 con 0x7f5874084e30 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient(hunting): my global_id is 34452 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58740008e0 con 0x7f5874084e30 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3162904529 0 0) 0x7f58740008e0 con 0x7f5874084e30 2026-02-21T22:23:47.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f58740909c0 con 0x7f5874084e30 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5864003ae0 con 0x7f5874084e30 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient: dump: 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2305702448 0 0) 0x7f58740909c0 con 0x7f5874084e30 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f587404d990 con 0x7f5874084e30 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 1355016147) 0x7f5864005b40 con 0x7f5874084e30 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f5874084e30 legacy=0x7f587404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f584ffff640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f5874084e30 legacy=0x7f587404d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:47.021 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 34, 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 375888, 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.022 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.023 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:47.024 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.025 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "num": 7 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.026 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.178:6800/2549949439 conn(0x7f584403d390 legacy=0x7f584405d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 10 monclient: shutdown 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 >> v1:10.20.193.102:6789/0 conn(0x7f5874088d80 legacy=0x7f587413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f5873fff640 1 -- 10.20.193.102:0/922998002 reap_dead start 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f5873fff640 1 -- 10.20.193.102:0/922998002 reap_dead start 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 shutdown_connections 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 >> 10.20.193.102:0/922998002 conn(0x7f5874091a50 msgr2=0x7f5874086990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 shutdown_connections 2026-02-21T22:23:47.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.019+0000 7f587a24f640 1 -- 10.20.193.102:0/922998002 wait complete. 2026-02-21T22:23:47.029 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:23:47.029 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-21T22:23:47.029 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: get_monmap_and_config 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: build_initial_monmap 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: monmap: 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:23:47.103596+0000 2026-02-21T22:23:47.101 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:23:47.103596+0000 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 1 Processor -- start 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 1 -- start start 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: init 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:47.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-c con 0x7fa7b408cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-b con 0x7fa7b408d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-a con 0x7fa7b408b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b40890e0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b4088ea0 con 0x7fa7b408cd00 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b408aea0 con 0x7fa7b408d7e0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7baed3640 10 monclient(hunting): authenticate will time out at 1636.339475s 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7abfff640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fa7b408d7e0 legacy=0x7fa7b41840b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7b9449640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa7b408b8d0 0x7fa7b41a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51932/0 (socket says 10.20.193.102:51932) 2026-02-21T22:23:47.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.099+0000 7fa7b9449640 1 -- 10.20.193.102:0/104741981 learned_addr learned my addr 10.20.193.102:0/104741981 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7b9449640 1 -- 10.20.193.102:0/104741981 >> v1:10.20.193.102:6789/0 conn(0x7fa7b408b8d0 legacy=0x7fa7b41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291742199 0 0) 0x7fa7b40890e0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient(hunting): my global_id is 34455 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7a0003620 con 0x7fa7b408b8d0 2026-02-21T22:23:47.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7b8c48640 1 -- 10.20.193.102:0/104741981 >> v1:10.20.193.178:6789/0 conn(0x7fa7b408cd00 legacy=0x7fa7b408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1795491882 0 0) 0x7fa7a0003620 con 0x7fa7b408b8d0 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 >> v1:10.20.193.102:6790/0 conn(0x7fa7b408d7e0 legacy=0x7fa7b41840b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 >> v1:10.20.193.178:6789/0 conn(0x7fa7b408cd00 legacy=0x7fa7b408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 monclient: found mon.noname-a 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b408a600 con 0x7fa7b408b8d0 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: _finish_auth 0 2026-02-21T22:23:47.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 5 monclient: authenticate success, global_id 34455 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa7b0004620 con 0x7fa7b408b8d0 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: dump: 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa7b00048f0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 1 -- 10.20.193.102:0/104741981 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa7b0006060 con 0x7fa7b408b8d0 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7ab7fe640 10 monclient: dump: 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.106 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 10 monclient: get_monmap_and_config success 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 4 set_mon_vals no callback set 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 10 monclient: shutdown 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- 10.20.193.102:0/104741981 >> v1:10.20.193.102:6789/0 conn(0x7fa7b408b8d0 legacy=0x7fa7b41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- 10.20.193.102:0/104741981 shutdown_connections 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- 10.20.193.102:0/104741981 >> 10.20.193.102:0/104741981 conn(0x7fa7b4091a50 msgr2=0x7fa7b4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- 10.20.193.102:0/104741981 shutdown_connections 2026-02-21T22:23:47.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- 10.20.193.102:0/104741981 wait complete. 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 10 monclient: build_initial_monmap 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 10 monclient: monmap: 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 Processor -- start 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 1 -- start start 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.103+0000 7fa7baed3640 10 monclient: init 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _reopen_session rank -1 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-b con 0x7fa7b408b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-a con 0x7fa7b408cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient(hunting): picked mon.noname-c con 0x7fa7b408d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b407feb0 con 0x7fa7b408cd00 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b4088a70 con 0x7fa7b408b8d0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7b9449640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7fa7b408d7e0 legacy=0x7fa7b4195d10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b4082210 con 0x7fa7b408d7e0 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient(hunting): _renew_subs 2026-02-21T22:23:47.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient(hunting): authenticate will time out at 1636.344110s 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7abfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa7b408cd00 0x7fa7b41945f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:51940/0 (socket says 10.20.193.102:51940) 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7abfff640 1 -- 10.20.193.102:0/3173941889 learned_addr learned my addr 10.20.193.102:0/3173941889 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7abfff640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.102:6789/0 conn(0x7fa7b408cd00 legacy=0x7fa7b41945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7abfff640 1 --1- 10.20.193.102:0/3173941889 >> v1:10.20.193.102:6789/0 conn(0x7fa7b408cd00 0x7fa7b41945f0 :-1 s=OPENED pgs=362 gs=2 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fa7b407feb0 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7b8c48640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b408b8d0 legacy=0x7fa7b41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818078006 0 0) 0x7fa7b4088a70 con 0x7fa7b408b8d0 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): my global_id is 54253 2026-02-21T22:23:47.109 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa784003620 con 0x7fa7b408b8d0 2026-02-21T22:23:47.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2341944251 0 0) 0x7fa784003620 con 0x7fa7b408b8d0 2026-02-21T22:23:47.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa7b4088a70 con 0x7fa7b408b8d0 2026-02-21T22:23:47.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa7a4003350 con 0x7fa7b408b8d0 2026-02-21T22:23:47.110 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): dump: 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2978519019 0 0) 0x7fa7b4088a70 con 0x7fa7b408b8d0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.102:6790/0 conn(0x7fa7b408d7e0 legacy=0x7fa7b4195d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.102:6789/0 conn(0x7fa7b408cd00 legacy=0x7fa7b41945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: _finish_hunting 0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 monclient: found mon.b 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b4082c10 con 0x7fa7b408b8d0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: _finish_auth 0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: _check_auth_tickets 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587116296 0 0) 0x7fa7b407feb0 con 0x7fa7b408cd00 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 5 monclient: authenticate success, global_id 54253 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _renew_subs 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:47.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b4080a10 con 0x7fa7b408b8d0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _renew_subs 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa7b407f670 con 0x7fa7b408b8d0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7b8c48640 1 --1- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b408b8d0 0x7fa7b41533e0 :-1 s=OPENED pgs=315 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fa7a4003d20 mon_map magic: 0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa7a40039c0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7a97fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7aaffd640 4 set_mon_vals no callback set 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.107+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa7b40008e0 con 0x7fa7b4153af0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa7a4003d20 con 0x7fa7b408b8d0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: dump: 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.112 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa7a4003ff0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7b9449640 10 monclient: get_auth_request con 0x7fa78403d2d0 auth_method 0 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7abfff640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b4153af0 legacy=0x7fa7b407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1864132649 0 0) 0x7fa7b40008e0 con 0x7fa7b4153af0 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient(hunting): my global_id is 54256 2026-02-21T22:23:47.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b407feb0 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 326590840 0 0) 0x7fa7b407feb0 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa7b40008e0 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa79c004420 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: dump: 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2541143573 0 0) 0x7fa7b40008e0 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7b4088560 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fa79c006790 con 0x7fa7b4153af0 2026-02-21T22:23:47.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:23:47.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:23:47.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b4153af0 legacy=0x7fa7b407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.111+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b4153af0 legacy=0x7fa7b407a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:47.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.115+0000 7fa7baed3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:23:47.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.115+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa7b4153ec0 con 0x7fa7b407aa80 2026-02-21T22:23:47.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7abfff640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b407aa80 legacy=0x7fa7b4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:23:47.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825214622 0 0) 0x7fa7b4153ec0 con 0x7fa7b407aa80 2026-02-21T22:23:47.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient(hunting): _negotiate 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient(hunting): my global_id is 54259 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b40008e0 con 0x7fa7b407aa80 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1621719359 0 0) 0x7fa7b40008e0 con 0x7fa7b407aa80 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa7b4153ec0 con 0x7fa7b407aa80 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa79c004b90 con 0x7fa7b407aa80 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient: dump: 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:23:47.121 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1410874223 0 0) 0x7fa7b4153ec0 con 0x7fa7b407aa80 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa7b4090810 con 0x7fa7b407aa80 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4336 (unknown 0 0 2743750519) 0x7fa79c006940 con 0x7fa7b407aa80 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b407aa80 legacy=0x7fa7b4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b407aa80 legacy=0x7fa7b4198440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 34, 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 130768, 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:23:47.122 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.123 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:23:47.124 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.125 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:23:47.126 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:23:47.127 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa78403d2d0 legacy=0x7fa78405d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 10 monclient: shutdown 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 >> v1:10.20.193.178:6789/0 conn(0x7fa7b408b8d0 legacy=0x7fa7b41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7b9449640 1 -- 10.20.193.102:0/3173941889 reap_dead start 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7a97fa640 1 -- 10.20.193.102:0/3173941889 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(395..395 src has 112..395) ==== 3975+0+0 (unknown 1547861635 0 0) 0x7fa7a404c7b0 con 0x7fa7b408b8d0 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7b9449640 1 -- 10.20.193.102:0/3173941889 reap_dead start 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 shutdown_connections 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 >> 10.20.193.102:0/3173941889 conn(0x7fa7b4091a50 msgr2=0x7fa7b405bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 shutdown_connections 2026-02-21T22:23:47.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:23:47.119+0000 7fa7baed3640 1 -- 10.20.193.102:0/3173941889 wait complete. 2026-02-21T22:23:47.132 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:23:47.132 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1886: done (9 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1887: done (8 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1888: done (7 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1889: done (6 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1890: done (5 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1891: done (4 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1892: done (3 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1893: done (2 left) 2026-02-21T22:23:47.802 INFO:tasks.rados.rados.0.trial178.stdout:1894: done (1 left) 2026-02-21T22:23:47.803 INFO:tasks.rados.rados.0.trial178.stdout:1895: done (0 left) 2026-02-21T22:23:47.803 INFO:tasks.rados.rados.0.trial178.stdout:1896: read oid 38 snap -1 2026-02-21T22:23:47.803 INFO:tasks.rados.rados.0.trial178.stdout:1896: expect (ObjNum 575 snap 158 seq_num 575) 2026-02-21T22:23:47.803 INFO:tasks.rados.rados.0.trial178.stdout:1897: snap_create 2026-02-21T22:23:47.803 INFO:tasks.rados.rados.0.trial178.stdout:1896: expect (ObjNum 575 snap 158 seq_num 575) 2026-02-21T22:23:47.871 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2641386 2026-02-21T22:23:47.915 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3124383 2026-02-21T22:23:48.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:48.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:48.802 INFO:tasks.rados.rados.0.trial178.stdout:1896: done (1 left) 2026-02-21T22:23:48.802 INFO:tasks.rados.rados.0.trial178.stdout:1897: done (0 left) 2026-02-21T22:23:48.802 INFO:tasks.rados.rados.0.trial178.stdout:1898: write (excl) oid 49 current snap is 188 2026-02-21T22:23:48.802 INFO:tasks.rados.rados.0.trial178.stdout:1898: seq_num 666 ranges {793462=638545,2106184=763893} 2026-02-21T22:23:48.811 INFO:tasks.rados.rados.0.trial178.stdout:1898: writing trial17818531-49 from 793462 to 1432007 tid 1 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:1898: writing trial17818531-49 from 2106184 to 2870077 tid 2 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:1899: rollback oid 30 current snap is 188 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 30 to 187 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:1898: finishing write tid 1 to trial17818531-49 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:1900: write oid 2 current snap is 188 2026-02-21T22:23:48.821 INFO:tasks.rados.rados.0.trial178.stdout:1900: seq_num 667 ranges {410269=713794,1648313=507551,2917489=292780} 2026-02-21T22:23:48.829 INFO:tasks.rados.rados.0.trial178.stdout:1900: writing trial17818531-2 from 410269 to 1124063 tid 1 2026-02-21T22:23:48.836 INFO:tasks.rados.rados.0.trial178.stdout:1900: writing trial17818531-2 from 1648313 to 2155864 tid 2 2026-02-21T22:23:48.843 INFO:tasks.rados.rados.0.trial178.stdout:1900: writing trial17818531-2 from 2917489 to 3210269 tid 3 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1901: delete oid 10 current snap is 188 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: finishing write tid 2 to trial17818531-49 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: finishing write tid 3 to trial17818531-49 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1899: finishing rollback tid 1 to trial17818531-30 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 745 (ObjNum 129 snap 0 seq_num 99604064) dirty dne 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: finishing write tid 4 to trial17818531-49 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: oid 49 updating version 0 to 775 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: oid 49 version 775 is already newer than 774 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: oid 49 version 775 is already newer than 773 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 775 (ObjNum 666 snap 188 seq_num 666) dirty exists 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1898: left oid 49 (ObjNum 666 snap 188 seq_num 666) 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1900: finishing write tid 1 to trial17818531-2 2026-02-21T22:23:48.844 INFO:tasks.rados.rados.0.trial178.stdout:1900: finishing write tid 2 to trial17818531-2 2026-02-21T22:23:48.845 INFO:tasks.rados.rados.0.trial178.stdout:1900: finishing write tid 3 to trial17818531-2 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: finishing write tid 4 to trial17818531-2 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: finishing write tid 5 to trial17818531-2 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: oid 2 updating version 0 to 772 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: oid 2 updating version 772 to 773 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: oid 2 updating version 773 to 775 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: oid 2 version 775 is already newer than 774 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 775 (ObjNum 667 snap 188 seq_num 667) dirty exists 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: left oid 2 (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1898: done (3 left) 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1899: done (2 left) 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1900: done (1 left) 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1901: done (0 left) 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1902: write oid 34 current snap is 188 2026-02-21T22:23:48.847 INFO:tasks.rados.rados.0.trial178.stdout:1902: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2026-02-21T22:23:48.853 INFO:tasks.rados.rados.0.trial178.stdout:1902: writing trial17818531-34 from 443883 to 908175 tid 1 2026-02-21T22:23:48.860 INFO:tasks.rados.rados.0.trial178.stdout:1902: writing trial17818531-34 from 1365277 to 1876497 tid 2 2026-02-21T22:23:48.869 INFO:tasks.rados.rados.0.trial178.stdout:1902: writing trial17818531-34 from 2572335 to 3243883 tid 3 2026-02-21T22:23:48.869 INFO:tasks.rados.rados.0.trial178.stdout:1903: snap_create 2026-02-21T22:23:48.869 INFO:tasks.rados.rados.0.trial178.stdout:1902: finishing write tid 1 to trial17818531-34 2026-02-21T22:23:48.869 INFO:tasks.rados.rados.0.trial178.stdout:1902: finishing write tid 2 to trial17818531-34 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: finishing write tid 3 to trial17818531-34 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: finishing write tid 4 to trial17818531-34 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: finishing write tid 5 to trial17818531-34 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: oid 34 updating version 0 to 760 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: oid 34 updating version 760 to 761 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: oid 34 version 761 is already newer than 759 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: oid 34 version 761 is already newer than 758 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 761 (ObjNum 669 snap 188 seq_num 669) dirty exists 2026-02-21T22:23:48.871 INFO:tasks.rados.rados.0.trial178.stdout:1902: left oid 34 (ObjNum 669 snap 188 seq_num 669) 2026-02-21T22:23:49.807 INFO:tasks.rados.rados.0.trial178.stdout:1902: done (1 left) 2026-02-21T22:23:49.807 INFO:tasks.rados.rados.0.trial178.stdout:1903: done (0 left) 2026-02-21T22:23:49.808 INFO:tasks.rados.rados.0.trial178.stdout:1904: write oid 8 current snap is 189 2026-02-21T22:23:49.808 INFO:tasks.rados.rados.0.trial178.stdout:1904: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2026-02-21T22:23:49.812 INFO:tasks.rados.rados.0.trial178.stdout:1904: writing trial17818531-8 from 460690 to 1116584 tid 1 2026-02-21T22:23:49.817 INFO:tasks.rados.rados.0.trial178.stdout:1904: writing trial17818531-8 from 1540112 to 2311343 tid 2 2026-02-21T22:23:49.820 INFO:tasks.rados.rados.0.trial178.stdout:1904: writing trial17818531-8 from 2716111 to 3255889 tid 3 2026-02-21T22:23:49.820 INFO:tasks.rados.rados.0.trial178.stdout:1905: write oid 19 current snap is 189 2026-02-21T22:23:49.821 INFO:tasks.rados.rados.0.trial178.stdout:1905: seq_num 671 ranges {477497=731143,1998594=514889,3143534=133963} 2026-02-21T22:23:49.825 INFO:tasks.rados.rados.0.trial178.stdout:1905: writing trial17818531-19 from 477497 to 1208640 tid 1 2026-02-21T22:23:49.830 INFO:tasks.rados.rados.0.trial178.stdout:1905: writing trial17818531-19 from 1998594 to 2513483 tid 2 2026-02-21T22:23:49.833 INFO:tasks.rados.rados.0.trial178.stdout:1905: writing trial17818531-19 from 3143534 to 3277497 tid 3 2026-02-21T22:23:49.833 INFO:tasks.rados.rados.0.trial178.stdout:1906: rollback oid 1 current snap is 189 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 182 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: finishing write tid 1 to trial17818531-8 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: finishing write tid 2 to trial17818531-8 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: finishing write tid 3 to trial17818531-8 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: finishing write tid 4 to trial17818531-8 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: finishing write tid 5 to trial17818531-8 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: oid 8 updating version 0 to 1093 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: oid 8 version 1093 is already newer than 1091 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: oid 8 version 1093 is already newer than 1090 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: oid 8 version 1093 is already newer than 1092 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1093 (ObjNum 670 snap 189 seq_num 670) dirty exists 2026-02-21T22:23:49.834 INFO:tasks.rados.rados.0.trial178.stdout:1904: left oid 8 (ObjNum 670 snap 189 seq_num 670) 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1905: finishing write tid 1 to trial17818531-19 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1905: finishing write tid 2 to trial17818531-19 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1904: done (2 left) 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1907: rollback oid 23 current snap is 189 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 186 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1908: write (excl) oid 18 current snap is 189 2026-02-21T22:23:49.835 INFO:tasks.rados.rados.0.trial178.stdout:1908: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2026-02-21T22:23:49.836 INFO:tasks.rados.rados.0.trial178.stdout:1908: writing trial17818531-18 from 494304 to 900696 tid 1 2026-02-21T22:23:49.842 INFO:tasks.rados.rados.0.trial178.stdout:1908: writing trial17818531-18 from 1540723 to 2315623 tid 2 2026-02-21T22:23:49.846 INFO:tasks.rados.rados.0.trial178.stdout:1908: writing trial17818531-18 from 2770957 to 3294304 tid 3 2026-02-21T22:23:49.846 INFO:tasks.rados.rados.0.trial178.stdout:1909: read oid 50 snap -1 2026-02-21T22:23:49.846 INFO:tasks.rados.rados.0.trial178.stdout:1909: expect (ObjNum 632 snap 179 seq_num 632) 2026-02-21T22:23:49.846 INFO:tasks.rados.rados.0.trial178.stdout:1910: write (excl) oid 35 current snap is 189 2026-02-21T22:23:49.846 INFO:tasks.rados.rados.0.trial178.stdout:1910: seq_num 673 ranges {511111=481641,1599205=518558,2682027=489947} 2026-02-21T22:23:49.849 INFO:tasks.rados.rados.0.trial178.stdout:1910: writing trial17818531-35 from 511111 to 992752 tid 1 2026-02-21T22:23:49.854 INFO:tasks.rados.rados.0.trial178.stdout:1910: writing trial17818531-35 from 1599205 to 2117763 tid 2 2026-02-21T22:23:49.858 INFO:tasks.rados.rados.0.trial178.stdout:1910: writing trial17818531-35 from 2682027 to 3171974 tid 3 2026-02-21T22:23:49.858 INFO:tasks.rados.rados.0.trial178.stdout:1911: write (excl) oid 6 current snap is 189 2026-02-21T22:23:49.858 INFO:tasks.rados.rados.0.trial178.stdout:1911: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2026-02-21T22:23:49.862 INFO:tasks.rados.rados.0.trial178.stdout:1911: writing trial17818531-6 from 527918 to 1084808 tid 1 2026-02-21T22:23:49.867 INFO:tasks.rados.rados.0.trial178.stdout:1911: writing trial17818531-6 from 1657687 to 2436256 tid 2 2026-02-21T22:23:49.871 INFO:tasks.rados.rados.0.trial178.stdout:1911: writing trial17818531-6 from 3225803 to 3327918 tid 3 2026-02-21T22:23:49.871 INFO:tasks.rados.rados.0.trial178.stdout:1912: read oid 24 snap 186 2026-02-21T22:23:49.871 INFO:tasks.rados.rados.0.trial178.stdout:1912: expect (ObjNum 657 snap 185 seq_num 657) 2026-02-21T22:23:49.871 INFO:tasks.rados.rados.0.trial178.stdout:1913: write oid 37 current snap is 189 2026-02-21T22:23:49.871 INFO:tasks.rados.rados.0.trial178.stdout:1913: seq_num 675 ranges {544725=632139,1716169=522227,2736873=512491} 2026-02-21T22:23:49.875 INFO:tasks.rados.rados.0.trial178.stdout:1913: writing trial17818531-37 from 544725 to 1176864 tid 1 2026-02-21T22:23:49.879 INFO:tasks.rados.rados.0.trial178.stdout:1913: writing trial17818531-37 from 1716169 to 2238396 tid 2 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1913: writing trial17818531-37 from 2736873 to 3249364 tid 3 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1914: rollback oid 11 current snap is 189 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 182 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1905: finishing write tid 3 to trial17818531-19 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1905: finishing write tid 4 to trial17818531-19 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1905: finishing write tid 5 to trial17818531-19 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1905: oid 19 updating version 0 to 1096 2026-02-21T22:23:49.884 INFO:tasks.rados.rados.0.trial178.stdout:1905: oid 19 version 1096 is already newer than 1095 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1905: oid 19 updating version 1096 to 1098 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1905: oid 19 version 1098 is already newer than 1097 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1098 (ObjNum 671 snap 189 seq_num 671) dirty exists 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1905: left oid 19 (ObjNum 671 snap 189 seq_num 671) 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1907: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 0 (ObjNum 241 snap 0 seq_num 102025872) dirty dne 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1906: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1906: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1906: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 774 (ObjNum 645 snap 180 seq_num 645) dirty exists 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1908: finishing write tid 1 to trial17818531-18 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1908: finishing write tid 2 to trial17818531-18 2026-02-21T22:23:49.885 INFO:tasks.rados.rados.0.trial178.stdout:1908: finishing write tid 3 to trial17818531-18 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1908: finishing write tid 4 to trial17818531-18 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1905: done (9 left) 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1906: done (8 left) 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1907: done (7 left) 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1915: write (excl) oid 31 current snap is 189 2026-02-21T22:23:49.886 INFO:tasks.rados.rados.0.trial178.stdout:1915: seq_num 676 ranges {561532=707388,2058298=782238} 2026-02-21T22:23:49.888 INFO:tasks.rados.rados.0.trial178.stdout:1915: writing trial17818531-31 from 561532 to 1268920 tid 1 2026-02-21T22:23:49.894 INFO:tasks.rados.rados.0.trial178.stdout:1915: writing trial17818531-31 from 2058298 to 2840536 tid 2 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1916: read oid 43 snap -1 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1916: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1917: read oid 32 snap 188 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1917: expect (ObjNum 656 snap 185 seq_num 656) 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1918: write (excl) oid 42 current snap is 189 2026-02-21T22:23:49.895 INFO:tasks.rados.rados.0.trial178.stdout:1918: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2026-02-21T22:23:49.898 INFO:tasks.rados.rados.0.trial178.stdout:1918: writing trial17818531-42 from 578339 to 1077329 tid 1 2026-02-21T22:23:49.903 INFO:tasks.rados.rados.0.trial178.stdout:1918: writing trial17818531-42 from 1833133 to 2359029 tid 2 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1918: writing trial17818531-42 from 2791719 to 3378339 tid 3 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1919: delete oid 34 current snap is 189 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1908: finishing write tid 5 to trial17818531-18 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1908: oid 18 updating version 0 to 748 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1908: oid 18 updating version 748 to 749 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1908: oid 18 version 749 is already newer than 747 2026-02-21T22:23:49.908 INFO:tasks.rados.rados.0.trial178.stdout:1908: oid 18 version 749 is already newer than 746 2026-02-21T22:23:49.909 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 749 (ObjNum 672 snap 189 seq_num 672) dirty exists 2026-02-21T22:23:49.909 INFO:tasks.rados.rados.0.trial178.stdout:1908: left oid 18 (ObjNum 672 snap 189 seq_num 672) 2026-02-21T22:23:49.909 INFO:tasks.rados.rados.0.trial178.stdout:1909: expect (ObjNum 632 snap 179 seq_num 632) 2026-02-21T22:23:50.000 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 601769 2026-02-21T22:23:50.036 INFO:tasks.rados.rados.0.trial178.stdout:1910: finishing write tid 1 to trial17818531-35 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: finishing write tid 2 to trial17818531-35 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: finishing write tid 3 to trial17818531-35 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: finishing write tid 4 to trial17818531-35 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: finishing write tid 5 to trial17818531-35 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: oid 35 updating version 0 to 988 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: oid 35 version 988 is already newer than 987 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: oid 35 version 988 is already newer than 986 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: oid 35 updating version 988 to 989 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 989 (ObjNum 673 snap 189 seq_num 673) dirty exists 2026-02-21T22:23:50.037 INFO:tasks.rados.rados.0.trial178.stdout:1910: left oid 35 (ObjNum 673 snap 189 seq_num 673) 2026-02-21T22:23:50.038 INFO:tasks.rados.rados.0.trial178.stdout:1911: finishing write tid 1 to trial17818531-6 2026-02-21T22:23:50.038 INFO:tasks.rados.rados.0.trial178.stdout:1912: expect (ObjNum 657 snap 185 seq_num 657) 2026-02-21T22:23:50.038 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:50.103 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: finishing write tid 2 to trial17818531-6 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: finishing write tid 3 to trial17818531-6 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: finishing write tid 4 to trial17818531-6 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: finishing write tid 5 to trial17818531-6 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: oid 6 updating version 0 to 777 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: oid 6 updating version 777 to 780 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: oid 6 version 780 is already newer than 779 2026-02-21T22:23:50.237 INFO:tasks.rados.rados.0.trial178.stdout:1911: oid 6 version 780 is already newer than 778 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 780 (ObjNum 674 snap 189 seq_num 674) dirty exists 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1911: left oid 6 (ObjNum 674 snap 189 seq_num 674) 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1914: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1913: finishing write tid 1 to trial17818531-37 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1914: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 991 (ObjNum 635 snap 180 seq_num 635) dirty exists 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1913: finishing write tid 2 to trial17818531-37 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1913: finishing write tid 3 to trial17818531-37 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1913: finishing write tid 4 to trial17818531-37 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1908: done (11 left) 2026-02-21T22:23:50.238 INFO:tasks.rados.rados.0.trial178.stdout:1909: done (10 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1910: done (9 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1911: done (8 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1912: done (7 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1914: done (6 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1919: done (5 left) 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1920: write (excl) oid 19 current snap is 189 2026-02-21T22:23:50.239 INFO:tasks.rados.rados.0.trial178.stdout:1920: seq_num 679 ranges {611953=649488,1950097=529565,3246565=165388} 2026-02-21T22:23:50.241 INFO:tasks.rados.rados.0.trial178.stdout:1920: writing trial17818531-19 from 611953 to 1261441 tid 1 2026-02-21T22:23:50.246 INFO:tasks.rados.rados.0.trial178.stdout:1920: writing trial17818531-19 from 1950097 to 2479662 tid 2 2026-02-21T22:23:50.250 INFO:tasks.rados.rados.0.trial178.stdout:1920: writing trial17818531-19 from 3246565 to 3411953 tid 3 2026-02-21T22:23:50.250 INFO:tasks.rados.rados.0.trial178.stdout:1921: read oid 2 snap -1 2026-02-21T22:23:50.250 INFO:tasks.rados.rados.0.trial178.stdout:1921: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:23:50.250 INFO:tasks.rados.rados.0.trial178.stdout:1922: read oid 16 snap -1 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1922: expect deleted 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1923: delete oid 28 current snap is 189 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: finishing write tid 5 to trial17818531-37 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: oid 37 updating version 0 to 777 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: oid 37 version 777 is already newer than 775 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: oid 37 updating version 777 to 778 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: oid 37 version 778 is already newer than 776 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 778 (ObjNum 675 snap 189 seq_num 675) dirty exists 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1913: left oid 37 (ObjNum 675 snap 189 seq_num 675) 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1915: finishing write tid 1 to trial17818531-31 2026-02-21T22:23:50.251 INFO:tasks.rados.rados.0.trial178.stdout:1916: expect (ObjNum 498 snap 139 seq_num 498) 2026-02-21T22:23:50.279 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: finishing write tid 2 to trial17818531-31 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: finishing write tid 3 to trial17818531-31 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: finishing write tid 4 to trial17818531-31 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: oid 31 updating version 0 to 778 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: oid 31 version 778 is already newer than 777 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: oid 31 version 778 is already newer than 776 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 778 (ObjNum 676 snap 189 seq_num 676) dirty exists 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1915: left oid 31 (ObjNum 676 snap 189 seq_num 676) 2026-02-21T22:23:50.299 INFO:tasks.rados.rados.0.trial178.stdout:1917: expect (ObjNum 656 snap 185 seq_num 656) 2026-02-21T22:23:50.375 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1476470 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: finishing write tid 1 to trial17818531-42 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: finishing write tid 2 to trial17818531-42 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: finishing write tid 3 to trial17818531-42 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: finishing write tid 4 to trial17818531-42 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: finishing write tid 5 to trial17818531-42 2026-02-21T22:23:50.399 INFO:tasks.rados.rados.0.trial178.stdout:1918: oid 42 updating version 0 to 1102 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1918: oid 42 updating version 1102 to 1103 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1918: oid 42 version 1103 is already newer than 1100 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1918: oid 42 version 1103 is already newer than 1101 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1103 (ObjNum 677 snap 189 seq_num 677) dirty exists 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1918: left oid 42 (ObjNum 677 snap 189 seq_num 677) 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: finishing write tid 1 to trial17818531-19 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: finishing write tid 2 to trial17818531-19 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: finishing write tid 3 to trial17818531-19 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: finishing write tid 4 to trial17818531-19 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: finishing write tid 5 to trial17818531-19 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: oid 19 updating version 0 to 1105 2026-02-21T22:23:50.400 INFO:tasks.rados.rados.0.trial178.stdout:1920: oid 19 version 1105 is already newer than 1104 2026-02-21T22:23:50.401 INFO:tasks.rados.rados.0.trial178.stdout:1920: oid 19 updating version 1105 to 1106 2026-02-21T22:23:50.401 INFO:tasks.rados.rados.0.trial178.stdout:1920: oid 19 updating version 1106 to 1107 2026-02-21T22:23:50.401 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1107 (ObjNum 679 snap 189 seq_num 679) dirty exists 2026-02-21T22:23:50.401 INFO:tasks.rados.rados.0.trial178.stdout:1920: left oid 19 (ObjNum 679 snap 189 seq_num 679) 2026-02-21T22:23:50.401 INFO:tasks.rados.rados.0.trial178.stdout:1921: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:23:50.429 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 128404 2026-02-21T22:23:50.494 INFO:tasks.rados.rados.0.trial178.stdout:1913: done (8 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1915: done (7 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1916: done (6 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1917: done (5 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1918: done (4 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1920: done (3 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1921: done (2 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1922: done (1 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1923: done (0 left) 2026-02-21T22:23:50.495 INFO:tasks.rados.rados.0.trial178.stdout:1924: write oid 32 current snap is 189 2026-02-21T22:23:50.496 INFO:tasks.rados.rados.0.trial178.stdout:1924: seq_num 681 ranges {645567=799986,1950708=533234,3185058=260509} 2026-02-21T22:23:50.504 INFO:tasks.rados.rados.0.trial178.stdout:1924: writing trial17818531-32 from 645567 to 1445553 tid 1 2026-02-21T22:23:50.511 INFO:tasks.rados.rados.0.trial178.stdout:1924: writing trial17818531-32 from 1950708 to 2483942 tid 2 2026-02-21T22:23:50.518 INFO:tasks.rados.rados.0.trial178.stdout:1924: writing trial17818531-32 from 3185058 to 3445567 tid 3 2026-02-21T22:23:50.518 INFO:tasks.rados.rados.0.trial178.stdout:1925: write (excl) oid 30 current snap is 189 2026-02-21T22:23:50.518 INFO:tasks.rados.rados.0.trial178.stdout:1925: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2026-02-21T22:23:50.526 INFO:tasks.rados.rados.0.trial178.stdout:1925: writing trial17818531-30 from 662374 to 1137609 tid 1 2026-02-21T22:23:50.534 INFO:tasks.rados.rados.0.trial178.stdout:1925: writing trial17818531-30 from 1609190 to 2286082 tid 2 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1925: writing trial17818531-30 from 2812481 to 3436582 tid 3 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1926: snap_create 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: finishing write tid 1 to trial17818531-32 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: finishing write tid 2 to trial17818531-32 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: finishing write tid 3 to trial17818531-32 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: finishing write tid 4 to trial17818531-32 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: finishing write tid 5 to trial17818531-32 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: oid 32 updating version 0 to 992 2026-02-21T22:23:50.542 INFO:tasks.rados.rados.0.trial178.stdout:1924: oid 32 updating version 992 to 994 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:1924: oid 32 updating version 994 to 995 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:1924: oid 32 version 995 is already newer than 993 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 995 (ObjNum 681 snap 189 seq_num 681) dirty exists 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:1924: left oid 32 (ObjNum 681 snap 189 seq_num 681) 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:1925: finishing write tid 1 to trial17818531-30 2026-02-21T22:23:50.543 INFO:tasks.rados.rados.0.trial178.stdout:1925: finishing write tid 2 to trial17818531-30 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: finishing write tid 3 to trial17818531-30 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: finishing write tid 4 to trial17818531-30 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: finishing write tid 5 to trial17818531-30 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: oid 30 updating version 0 to 752 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: oid 30 version 752 is already newer than 750 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: oid 30 updating version 752 to 753 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: oid 30 version 753 is already newer than 751 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 753 (ObjNum 682 snap 189 seq_num 682) dirty exists 2026-02-21T22:23:50.544 INFO:tasks.rados.rados.0.trial178.stdout:1925: left oid 30 (ObjNum 682 snap 189 seq_num 682) 2026-02-21T22:23:50.811 INFO:tasks.rados.rados.0.trial178.stdout:1924: done (2 left) 2026-02-21T22:23:50.812 INFO:tasks.rados.rados.0.trial178.stdout:1925: done (1 left) 2026-02-21T22:23:50.812 INFO:tasks.rados.rados.0.trial178.stdout:1926: done (0 left) 2026-02-21T22:23:50.812 INFO:tasks.rados.rados.0.trial178.stdout:1927: delete oid 31 current snap is 190 2026-02-21T22:23:50.815 INFO:tasks.rados.rados.0.trial178.stdout:1927: done (0 left) 2026-02-21T22:23:50.815 INFO:tasks.rados.rados.0.trial178.stdout:1928: read oid 46 snap 189 2026-02-21T22:23:50.815 INFO:tasks.rados.rados.0.trial178.stdout:1928: expect (ObjNum 590 snap 163 seq_num 590) 2026-02-21T22:23:50.815 INFO:tasks.rados.rados.0.trial178.stdout:1929: write oid 1 current snap is 190 2026-02-21T22:23:50.815 INFO:tasks.rados.rados.0.trial178.stdout:1929: seq_num 684 ranges {695988=625733,2009801=680561,3150974=345014} 2026-02-21T22:23:50.823 INFO:tasks.rados.rados.0.trial178.stdout:1929: writing trial17818531-1 from 695988 to 1321721 tid 1 2026-02-21T22:23:50.832 INFO:tasks.rados.rados.0.trial178.stdout:1929: writing trial17818531-1 from 2009801 to 2690362 tid 2 2026-02-21T22:23:50.838 INFO:tasks.rados.rados.0.trial178.stdout:1929: writing trial17818531-1 from 3150974 to 3495988 tid 3 2026-02-21T22:23:50.838 INFO:tasks.rados.rados.0.trial178.stdout:1930: write oid 16 current snap is 190 2026-02-21T22:23:50.838 INFO:tasks.rados.rados.0.trial178.stdout:1930: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2026-02-21T22:23:50.845 INFO:tasks.rados.rados.0.trial178.stdout:1930: writing trial17818531-16 from 712795 to 1130130 tid 1 2026-02-21T22:23:50.854 INFO:tasks.rados.rados.0.trial178.stdout:1930: writing trial17818531-16 from 1784636 to 2325208 tid 2 2026-02-21T22:23:50.861 INFO:tasks.rados.rados.0.trial178.stdout:1930: writing trial17818531-16 from 2894750 to 3352667 tid 3 2026-02-21T22:23:50.861 INFO:tasks.rados.rados.0.trial178.stdout:1931: write oid 23 current snap is 190 2026-02-21T22:23:50.861 INFO:tasks.rados.rados.0.trial178.stdout:1931: seq_num 686 ranges {729602=492584,1843118=684230,3322173=207429} 2026-02-21T22:23:50.865 INFO:tasks.rados.rados.0.trial178.stdout:1931: writing trial17818531-23 from 729602 to 1222186 tid 1 2026-02-21T22:23:50.870 INFO:tasks.rados.rados.0.trial178.stdout:1931: writing trial17818531-23 from 1843118 to 2527348 tid 2 2026-02-21T22:23:50.874 INFO:tasks.rados.rados.0.trial178.stdout:1931: writing trial17818531-23 from 3322173 to 3529602 tid 3 2026-02-21T22:23:50.874 INFO:tasks.rados.rados.0.trial178.stdout:1932: read oid 35 snap 186 2026-02-21T22:23:50.874 INFO:tasks.rados.rados.0.trial178.stdout:1932: expect (ObjNum 598 snap 167 seq_num 598) 2026-02-21T22:23:50.875 INFO:tasks.rados.rados.0.trial178.stdout:1933: delete oid 12 current snap is 190 2026-02-21T22:23:50.875 INFO:tasks.rados.rados.0.trial178.stdout:1928: expect (ObjNum 590 snap 163 seq_num 590) 2026-02-21T22:23:50.946 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:51.001 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 257610 2026-02-21T22:23:51.055 INFO:tasks.rados.rados.0.trial178.stdout:1929: finishing write tid 1 to trial17818531-1 2026-02-21T22:23:51.056 INFO:tasks.rados.rados.0.trial178.stdout:1929: finishing write tid 2 to trial17818531-1 2026-02-21T22:23:51.056 INFO:tasks.rados.rados.0.trial178.stdout:1929: finishing write tid 3 to trial17818531-1 2026-02-21T22:23:51.056 INFO:tasks.rados.rados.0.trial178.stdout:1929: finishing write tid 4 to trial17818531-1 2026-02-21T22:23:51.056 INFO:tasks.rados.rados.0.trial178.stdout:1929: finishing write tid 5 to trial17818531-1 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:1929: oid 1 updating version 0 to 779 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:1929: oid 1 updating version 779 to 782 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:1929: oid 1 version 782 is already newer than 780 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:1929: oid 1 version 782 is already newer than 781 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 782 (ObjNum 684 snap 190 seq_num 684) dirty exists 2026-02-21T22:23:51.057 INFO:tasks.rados.rados.0.trial178.stdout:1929: left oid 1 (ObjNum 684 snap 190 seq_num 684) 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1930: finishing write tid 1 to trial17818531-16 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1930: finishing write tid 2 to trial17818531-16 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1930: finishing write tid 3 to trial17818531-16 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1930: finishing write tid 4 to trial17818531-16 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1928: done (5 left) 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1929: done (4 left) 2026-02-21T22:23:51.058 INFO:tasks.rados.rados.0.trial178.stdout:1933: done (3 left) 2026-02-21T22:23:51.059 INFO:tasks.rados.rados.0.trial178.stdout:1934: copy_from oid 9 from oid 8 current snap is 190 2026-02-21T22:23:51.059 INFO:tasks.rados.rados.0.trial178.stdout:1930: finishing write tid 5 to trial17818531-16 2026-02-21T22:23:51.059 INFO:tasks.rados.rados.0.trial178.stdout:1930: oid 16 updating version 0 to 767 2026-02-21T22:23:51.059 INFO:tasks.rados.rados.0.trial178.stdout:1930: oid 16 version 767 is already newer than 765 2026-02-21T22:23:51.059 INFO:tasks.rados.rados.0.trial178.stdout:1930: oid 16 updating version 767 to 768 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1930: oid 16 version 768 is already newer than 766 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 768 (ObjNum 685 snap 190 seq_num 685) dirty exists 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1930: left oid 16 (ObjNum 685 snap 190 seq_num 685) 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1931: finishing write tid 1 to trial17818531-23 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1931: finishing write tid 2 to trial17818531-23 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1931: finishing write tid 3 to trial17818531-23 2026-02-21T22:23:51.060 INFO:tasks.rados.rados.0.trial178.stdout:1931: finishing write tid 4 to trial17818531-23 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:1931: finishing write tid 5 to trial17818531-23 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:1931: oid 23 updating version 0 to 776 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:1931: oid 23 version 776 is already newer than 775 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:1931: oid 23 version 776 is already newer than 773 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:1931: oid 23 version 776 is already newer than 774 2026-02-21T22:23:51.061 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 776 (ObjNum 686 snap 190 seq_num 686) dirty exists 2026-02-21T22:23:51.062 INFO:tasks.rados.rados.0.trial178.stdout:1931: left oid 23 (ObjNum 686 snap 190 seq_num 686) 2026-02-21T22:23:51.062 INFO:tasks.rados.rados.0.trial178.stdout:1932: expect (ObjNum 598 snap 167 seq_num 598) 2026-02-21T22:23:51.154 INFO:tasks.rados.rados.0.trial178.stdout:1934: finishing copy_from racing read to trial17818531-9 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1934: finishing copy_from to trial17818531-9 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1093 (ObjNum 670 snap 189 seq_num 670) dirty exists 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1930: done (3 left) 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1931: done (2 left) 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1932: done (1 left) 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1934: done (0 left) 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:1935: rollback oid 25 current snap is 190 2026-02-21T22:23:51.155 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 25 to 187 2026-02-21T22:23:51.156 INFO:tasks.rados.rados.0.trial178.stdout:1936: write oid 1 current snap is 190 2026-02-21T22:23:51.156 INFO:tasks.rados.rados.0.trial178.stdout:1936: seq_num 688 ranges {763216=643082,1843729=687899,3260666=302550} 2026-02-21T22:23:51.164 INFO:tasks.rados.rados.0.trial178.stdout:1936: writing trial17818531-1 from 763216 to 1406298 tid 1 2026-02-21T22:23:51.171 INFO:tasks.rados.rados.0.trial178.stdout:1936: writing trial17818531-1 from 1843729 to 2531628 tid 2 2026-02-21T22:23:51.179 INFO:tasks.rados.rados.0.trial178.stdout:1936: writing trial17818531-1 from 3260666 to 3563216 tid 3 2026-02-21T22:23:51.179 INFO:tasks.rados.rados.0.trial178.stdout:1937: write oid 4 current snap is 190 2026-02-21T22:23:51.179 INFO:tasks.rados.rados.0.trial178.stdout:1937: seq_num 689 ranges {780023=718331,1902211=547910,3004442=575581} 2026-02-21T22:23:51.189 INFO:tasks.rados.rados.0.trial178.stdout:1937: writing trial17818531-4 from 780023 to 1498354 tid 1 2026-02-21T22:23:51.196 INFO:tasks.rados.rados.0.trial178.stdout:1937: writing trial17818531-4 from 1902211 to 2450121 tid 2 2026-02-21T22:23:51.202 INFO:tasks.rados.rados.0.trial178.stdout:1937: writing trial17818531-4 from 3004442 to 3580023 tid 3 2026-02-21T22:23:51.203 INFO:tasks.rados.rados.0.trial178.stdout:1938: read oid 35 snap -1 2026-02-21T22:23:51.203 INFO:tasks.rados.rados.0.trial178.stdout:1938: expect (ObjNum 673 snap 189 seq_num 673) 2026-02-21T22:23:51.203 INFO:tasks.rados.rados.0.trial178.stdout:1939: read oid 22 snap -1 2026-02-21T22:23:51.203 INFO:tasks.rados.rados.0.trial178.stdout:1939: expect deleted 2026-02-21T22:23:51.203 INFO:tasks.rados.rados.0.trial178.stdout:1940: snap_remove snap 182 2026-02-21T22:23:51.754 INFO:tasks.rados.rados.0.trial178.stdout:1940: done (5 left) 2026-02-21T22:23:51.755 INFO:tasks.rados.rados.0.trial178.stdout:1941: snap_remove snap 188 2026-02-21T22:23:53.034 INFO:tasks.rados.rados.0.trial178.stdout:1941: done (5 left) 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1942: delete oid 40 current snap is 190 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1935: finishing rollback tid 0 to trial17818531-25 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1935: finishing rollback tid 1 to trial17818531-25 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1935: finishing rollback tid 2 to trial17818531-25 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 782 (ObjNum 558 snap 158 seq_num 558) dirty exists 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: finishing write tid 1 to trial17818531-1 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: finishing write tid 2 to trial17818531-1 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: finishing write tid 3 to trial17818531-1 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: finishing write tid 4 to trial17818531-1 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: finishing write tid 5 to trial17818531-1 2026-02-21T22:23:53.035 INFO:tasks.rados.rados.0.trial178.stdout:1936: oid 1 updating version 0 to 784 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1936: oid 1 version 784 is already newer than 783 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1936: oid 1 updating version 784 to 786 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1936: oid 1 version 786 is already newer than 785 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 786 (ObjNum 688 snap 190 seq_num 688) dirty exists 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1936: left oid 1 (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: finishing write tid 1 to trial17818531-4 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: finishing write tid 2 to trial17818531-4 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: finishing write tid 3 to trial17818531-4 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: finishing write tid 4 to trial17818531-4 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: finishing write tid 5 to trial17818531-4 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: oid 4 updating version 0 to 773 2026-02-21T22:23:53.036 INFO:tasks.rados.rados.0.trial178.stdout:1937: oid 4 version 773 is already newer than 771 2026-02-21T22:23:53.037 INFO:tasks.rados.rados.0.trial178.stdout:1937: oid 4 version 773 is already newer than 772 2026-02-21T22:23:53.037 INFO:tasks.rados.rados.0.trial178.stdout:1937: oid 4 version 773 is already newer than 770 2026-02-21T22:23:53.037 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 773 (ObjNum 689 snap 190 seq_num 689) dirty exists 2026-02-21T22:23:53.037 INFO:tasks.rados.rados.0.trial178.stdout:1937: left oid 4 (ObjNum 689 snap 190 seq_num 689) 2026-02-21T22:23:53.037 INFO:tasks.rados.rados.0.trial178.stdout:1938: expect (ObjNum 673 snap 189 seq_num 673) 2026-02-21T22:23:53.099 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:53.192 INFO:tasks.rados.rados.0.trial178.stdout:1935: done (5 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1936: done (4 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1937: done (3 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1938: done (2 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1939: done (1 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1942: done (0 left) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1943: read oid 13 snap -1 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1943: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1944: read oid 34 snap -1 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1944: expect deleted 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:1945: rollback oid 16 current snap is 190 2026-02-21T22:23:53.193 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 186 2026-02-21T22:23:53.194 INFO:tasks.rados.rados.0.trial178.stdout:1946: write (excl) oid 7 current snap is 190 2026-02-21T22:23:53.194 INFO:tasks.rados.rados.0.trial178.stdout:1946: seq_num 691 ranges {413637=468829,1619175=435226,2542935=758255} 2026-02-21T22:23:53.198 INFO:tasks.rados.rados.0.trial178.stdout:1946: writing trial17818531-7 from 413637 to 882466 tid 1 2026-02-21T22:23:53.206 INFO:tasks.rados.rados.0.trial178.stdout:1946: writing trial17818531-7 from 1619175 to 2054401 tid 2 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:1946: writing trial17818531-7 from 2542935 to 3301190 tid 3 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:1947: delete oid 28 current snap is 190 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:1945: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:1946: finishing write tid 1 to trial17818531-7 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:1943: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:53.215 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:53.335 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3138903 2026-02-21T22:23:53.339 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 564816 2026-02-21T22:23:53.399 INFO:tasks.rados.rados.0.trial178.stdout:1945: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 775 (ObjNum -1409409808 snap 32564 seq_num 304) dirty dne 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: finishing write tid 2 to trial17818531-7 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: finishing write tid 3 to trial17818531-7 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: finishing write tid 4 to trial17818531-7 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: finishing write tid 5 to trial17818531-7 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: oid 7 updating version 0 to 1126 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: oid 7 updating version 1126 to 1127 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: oid 7 version 1127 is already newer than 1124 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: oid 7 version 1127 is already newer than 1125 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:23:53.400 INFO:tasks.rados.rados.0.trial178.stdout:1946: left oid 7 (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1943: done (4 left) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1944: done (3 left) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1945: done (2 left) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1946: done (1 left) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1947: done (0 left) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1948: read oid 15 snap -1 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1948: expect (ObjNum 593 snap 163 seq_num 593) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1949: read oid 30 snap -1 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1949: expect (ObjNum 682 snap 189 seq_num 682) 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1950: copy_from oid 28 from oid 31 current snap is 190 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1951: copy_from oid 9 from oid 13 current snap is 190 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:1952: rollback oid 26 current snap is 190 2026-02-21T22:23:53.401 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 187 2026-02-21T22:23:53.402 INFO:tasks.rados.rados.0.trial178.stdout:1953: snap_remove snap 189 2026-02-21T22:23:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:53.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:53.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:23:53.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:23:53.561 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:53.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1953: done (5 left) 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1954: copy_from oid 25 from oid 18 current snap is 190 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1950: finishing copy_from to trial17818531-28 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1950: got expected ENOENT (src dne) 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1950: finishing copy_from racing read to trial17818531-28 2026-02-21T22:23:54.037 INFO:tasks.rados.rados.0.trial178.stdout:1949: expect (ObjNum 682 snap 189 seq_num 682) 2026-02-21T22:23:54.038 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:54.087 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:54.112 INFO:tasks.rados.rados.0.trial178.stdout:1952: finishing rollback tid 0 to trial17818531-26 2026-02-21T22:23:54.112 INFO:tasks.rados.rados.0.trial178.stdout:1952: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:23:54.112 INFO:tasks.rados.rados.0.trial178.stdout:1949: done (5 left) 2026-02-21T22:23:54.112 INFO:tasks.rados.rados.0.trial178.stdout:1950: done (4 left) 2026-02-21T22:23:54.113 INFO:tasks.rados.rados.0.trial178.stdout:1955: write (excl) oid 31 current snap is 190 2026-02-21T22:23:54.113 INFO:tasks.rados.rados.0.trial178.stdout:1955: seq_num 693 ranges {447251=735680,1736139=438895,2597781=780799} 2026-02-21T22:23:54.120 INFO:tasks.rados.rados.0.trial178.stdout:1955: writing trial17818531-31 from 447251 to 1182931 tid 1 2026-02-21T22:23:54.127 INFO:tasks.rados.rados.0.trial178.stdout:1955: writing trial17818531-31 from 1736139 to 2175034 tid 2 2026-02-21T22:23:54.135 INFO:tasks.rados.rados.0.trial178.stdout:1955: writing trial17818531-31 from 2597781 to 3378580 tid 3 2026-02-21T22:23:54.136 INFO:tasks.rados.rados.0.trial178.stdout:1956: read oid 35 snap -1 2026-02-21T22:23:54.136 INFO:tasks.rados.rados.0.trial178.stdout:1956: expect (ObjNum 673 snap 189 seq_num 673) 2026-02-21T22:23:54.136 INFO:tasks.rados.rados.0.trial178.stdout:1957: snap_remove snap 186 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1957: done (6 left) 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1958: read oid 45 snap 187 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1958: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1959: rollback oid 17 current snap is 190 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1952: finishing rollback tid 2 to trial17818531-26 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 783 (ObjNum 564 snap 158 seq_num 564) dirty exists 2026-02-21T22:23:55.041 INFO:tasks.rados.rados.0.trial178.stdout:1948: expect (ObjNum 593 snap 163 seq_num 593) 2026-02-21T22:23:55.169 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2779689 2026-02-21T22:23:55.189 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:55.268 INFO:tasks.rados.rados.0.trial178.stdout:1951: finishing copy_from racing read to trial17818531-9 2026-02-21T22:23:55.268 INFO:tasks.rados.rados.0.trial178.stdout:1951: finishing copy_from to trial17818531-9 2026-02-21T22:23:55.268 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 518 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:23:55.268 INFO:tasks.rados.rados.0.trial178.stdout:1954: finishing copy_from racing read to trial17818531-25 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1954: finishing copy_from to trial17818531-25 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 749 (ObjNum 672 snap 189 seq_num 672) dirty exists 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: finishing write tid 1 to trial17818531-31 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: finishing write tid 2 to trial17818531-31 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: finishing write tid 3 to trial17818531-31 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: finishing write tid 4 to trial17818531-31 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: finishing write tid 5 to trial17818531-31 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: oid 31 updating version 0 to 785 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: oid 31 version 785 is already newer than 784 2026-02-21T22:23:55.269 INFO:tasks.rados.rados.0.trial178.stdout:1955: oid 31 version 785 is already newer than 783 2026-02-21T22:23:55.270 INFO:tasks.rados.rados.0.trial178.stdout:1955: oid 31 updating version 785 to 786 2026-02-21T22:23:55.270 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 786 (ObjNum 693 snap 190 seq_num 693) dirty exists 2026-02-21T22:23:55.270 INFO:tasks.rados.rados.0.trial178.stdout:1955: left oid 31 (ObjNum 693 snap 190 seq_num 693) 2026-02-21T22:23:55.270 INFO:tasks.rados.rados.0.trial178.stdout:1956: expect (ObjNum 673 snap 189 seq_num 673) 2026-02-21T22:23:55.270 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:55.375 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2399394 2026-02-21T22:23:55.388 INFO:tasks.rados.rados.0.trial178.stdout:1958: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 187 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1948: done (7 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1951: done (6 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1952: done (5 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1954: done (4 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1955: done (3 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1956: done (2 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1958: done (1 left) 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1960: rollback oid 46 current snap is 190 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 187 2026-02-21T22:23:55.549 INFO:tasks.rados.rados.0.trial178.stdout:1961: delete oid 33 current snap is 190 2026-02-21T22:23:55.550 INFO:tasks.rados.rados.0.trial178.stdout:1961: done (2 left) 2026-02-21T22:23:55.550 INFO:tasks.rados.rados.0.trial178.stdout:1962: write (excl) oid 43 current snap is 190 2026-02-21T22:23:55.550 INFO:tasks.rados.rados.0.trial178.stdout:1962: seq_num 695 ranges {480865=486178,1453103=442564,2652627=519696} 2026-02-21T22:23:55.557 INFO:tasks.rados.rados.0.trial178.stdout:1962: writing trial17818531-43 from 480865 to 967043 tid 1 2026-02-21T22:23:55.564 INFO:tasks.rados.rados.0.trial178.stdout:1962: writing trial17818531-43 from 1453103 to 1895667 tid 2 2026-02-21T22:23:55.573 INFO:tasks.rados.rados.0.trial178.stdout:1962: writing trial17818531-43 from 2652627 to 3172323 tid 3 2026-02-21T22:23:55.573 INFO:tasks.rados.rados.0.trial178.stdout:1963: read oid 41 snap -1 2026-02-21T22:23:55.573 INFO:tasks.rados.rados.0.trial178.stdout:1963: expect deleted 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1964: snap_create 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1960: finishing rollback tid 0 to trial17818531-46 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1960: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1959: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1960: finishing rollback tid 2 to trial17818531-46 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 779 (ObjNum 590 snap 163 seq_num 590) dirty exists 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1959: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1959: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 730 (ObjNum 648 snap 182 seq_num 648) dirty exists 2026-02-21T22:23:55.574 INFO:tasks.rados.rados.0.trial178.stdout:1962: finishing write tid 1 to trial17818531-43 2026-02-21T22:23:55.575 INFO:tasks.rados.rados.0.trial178.stdout:1962: finishing write tid 2 to trial17818531-43 2026-02-21T22:23:55.575 INFO:tasks.rados.rados.0.trial178.stdout:1962: finishing write tid 3 to trial17818531-43 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: finishing write tid 4 to trial17818531-43 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: finishing write tid 5 to trial17818531-43 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: oid 43 updating version 0 to 365 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: oid 43 updating version 365 to 366 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: oid 43 updating version 366 to 368 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: oid 43 version 368 is already newer than 367 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 368 (ObjNum 695 snap 190 seq_num 695) dirty exists 2026-02-21T22:23:55.576 INFO:tasks.rados.rados.0.trial178.stdout:1962: left oid 43 (ObjNum 695 snap 190 seq_num 695) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1959: done (4 left) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1960: done (3 left) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1962: done (2 left) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1963: done (1 left) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1964: done (0 left) 2026-02-21T22:23:56.046 INFO:tasks.rados.rados.0.trial178.stdout:1965: snap_create 2026-02-21T22:23:56.754 INFO:tasks.rados.rados.0.trial178.stdout:1965: done (0 left) 2026-02-21T22:23:56.754 INFO:tasks.rados.rados.0.trial178.stdout:1966: read oid 3 snap -1 2026-02-21T22:23:56.754 INFO:tasks.rados.rados.0.trial178.stdout:1966: expect deleted 2026-02-21T22:23:56.754 INFO:tasks.rados.rados.0.trial178.stdout:1967: snap_remove snap 190 2026-02-21T22:23:57.755 INFO:tasks.rados.rados.0.trial178.stdout:1967: done (1 left) 2026-02-21T22:23:57.755 INFO:tasks.rados.rados.0.trial178.stdout:1968: write oid 18 current snap is 192 2026-02-21T22:23:57.755 INFO:tasks.rados.rados.0.trial178.stdout:1968: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2026-02-21T22:23:57.762 INFO:tasks.rados.rados.0.trial178.stdout:1968: writing trial17818531-18 from 497672 to 1059099 tid 1 2026-02-21T22:23:57.772 INFO:tasks.rados.rados.0.trial178.stdout:1968: writing trial17818531-18 from 1795232 to 2497807 tid 2 2026-02-21T22:23:57.780 INFO:tasks.rados.rados.0.trial178.stdout:1968: writing trial17818531-18 from 2963697 to 3694665 tid 3 2026-02-21T22:23:57.780 INFO:tasks.rados.rados.0.trial178.stdout:1969: write (excl) oid 42 current snap is 192 2026-02-21T22:23:57.781 INFO:tasks.rados.rados.0.trial178.stdout:1969: seq_num 697 ranges {514479=636676,1853714=446233,2991120=542240} 2026-02-21T22:23:57.788 INFO:tasks.rados.rados.0.trial178.stdout:1969: writing trial17818531-42 from 514479 to 1151155 tid 1 2026-02-21T22:23:57.796 INFO:tasks.rados.rados.0.trial178.stdout:1969: writing trial17818531-42 from 1853714 to 2299947 tid 2 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1969: writing trial17818531-42 from 2991120 to 3533360 tid 3 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1970: read oid 9 snap -1 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1970: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1971: snap_create 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: finishing write tid 1 to trial17818531-18 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: finishing write tid 2 to trial17818531-18 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: finishing write tid 3 to trial17818531-18 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: finishing write tid 4 to trial17818531-18 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: finishing write tid 5 to trial17818531-18 2026-02-21T22:23:57.803 INFO:tasks.rados.rados.0.trial178.stdout:1968: oid 18 updating version 0 to 757 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1968: oid 18 version 757 is already newer than 754 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1968: oid 18 version 757 is already newer than 756 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1968: oid 18 version 757 is already newer than 755 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 757 (ObjNum 696 snap 192 seq_num 696) dirty exists 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1968: left oid 18 (ObjNum 696 snap 192 seq_num 696) 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1969: finishing write tid 1 to trial17818531-42 2026-02-21T22:23:57.804 INFO:tasks.rados.rados.0.trial178.stdout:1969: finishing write tid 2 to trial17818531-42 2026-02-21T22:23:57.805 INFO:tasks.rados.rados.0.trial178.stdout:1969: finishing write tid 3 to trial17818531-42 2026-02-21T22:23:57.811 INFO:tasks.rados.rados.0.trial178.stdout:1969: finishing write tid 4 to trial17818531-42 2026-02-21T22:23:57.811 INFO:tasks.rados.rados.0.trial178.stdout:1969: finishing write tid 5 to trial17818531-42 2026-02-21T22:23:57.811 INFO:tasks.rados.rados.0.trial178.stdout:1969: oid 42 updating version 0 to 1136 2026-02-21T22:23:57.811 INFO:tasks.rados.rados.0.trial178.stdout:1969: oid 42 updating version 1136 to 1137 2026-02-21T22:23:57.811 INFO:tasks.rados.rados.0.trial178.stdout:1969: oid 42 version 1137 is already newer than 1135 2026-02-21T22:23:57.812 INFO:tasks.rados.rados.0.trial178.stdout:1969: oid 42 updating version 1137 to 1138 2026-02-21T22:23:57.812 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1138 (ObjNum 697 snap 192 seq_num 697) dirty exists 2026-02-21T22:23:57.812 INFO:tasks.rados.rados.0.trial178.stdout:1969: left oid 42 (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:23:57.813 INFO:tasks.rados.rados.0.trial178.stdout:1970: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:23:57.868 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2451883 2026-02-21T22:23:57.958 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:23:58.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:23:58.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1966: done (4 left) 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1968: done (3 left) 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1969: done (2 left) 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1970: done (1 left) 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1971: done (0 left) 2026-02-21T22:23:58.759 INFO:tasks.rados.rados.0.trial178.stdout:1972: rollback oid 20 current snap is 193 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 191 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:1973: read oid 1 snap -1 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:1973: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:1974: rollback oid 41 current snap is 193 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 191 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:1975: write (excl) oid 36 current snap is 193 2026-02-21T22:23:58.760 INFO:tasks.rados.rados.0.trial178.stdout:1975: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2026-02-21T22:23:58.764 INFO:tasks.rados.rados.0.trial178.stdout:1975: writing trial17818531-36 from 531286 to 1243211 tid 1 2026-02-21T22:23:58.769 INFO:tasks.rados.rados.0.trial178.stdout:1975: writing trial17818531-36 from 1912196 to 2618440 tid 2 2026-02-21T22:23:58.774 INFO:tasks.rados.rados.0.trial178.stdout:1975: writing trial17818531-36 from 3018543 to 3731286 tid 3 2026-02-21T22:23:58.774 INFO:tasks.rados.rados.0.trial178.stdout:1976: write oid 19 current snap is 193 2026-02-21T22:23:58.774 INFO:tasks.rados.rados.0.trial178.stdout:1976: seq_num 699 ranges {548093=787174,1970678=449902,3045966=681137} 2026-02-21T22:23:58.779 INFO:tasks.rados.rados.0.trial178.stdout:1976: writing trial17818531-19 from 548093 to 1335267 tid 1 2026-02-21T22:23:58.783 INFO:tasks.rados.rados.0.trial178.stdout:1976: writing trial17818531-19 from 1970678 to 2420580 tid 2 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1976: writing trial17818531-19 from 3045966 to 3727103 tid 3 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1977: copy_from oid 48 from oid 7 current snap is 193 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1974: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 0 (ObjNum 33 snap 0 seq_num 5) dirty dne 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1972: finishing rollback tid 0 to trial17818531-20 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1972: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:23:58.789 INFO:tasks.rados.rados.0.trial178.stdout:1972: finishing rollback tid 2 to trial17818531-20 2026-02-21T22:23:58.790 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 733 (ObjNum 575 snap 158 seq_num 575) dirty exists 2026-02-21T22:23:58.790 INFO:tasks.rados.rados.0.trial178.stdout:1973: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:23:58.900 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3342231 2026-02-21T22:23:58.963 INFO:tasks.rados.rados.0.trial178.stdout:1975: finishing write tid 1 to trial17818531-36 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1975: finishing write tid 2 to trial17818531-36 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1975: finishing write tid 3 to trial17818531-36 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1975: finishing write tid 4 to trial17818531-36 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1972: done (5 left) 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1973: done (4 left) 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1974: done (3 left) 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1978: rollback oid 18 current snap is 193 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 192 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1975: finishing write tid 5 to trial17818531-36 2026-02-21T22:23:58.964 INFO:tasks.rados.rados.0.trial178.stdout:1975: oid 36 updating version 0 to 379 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1975: oid 36 version 379 is already newer than 376 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1975: oid 36 version 379 is already newer than 377 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1975: oid 36 version 379 is already newer than 378 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 379 (ObjNum 698 snap 193 seq_num 698) dirty exists 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1975: left oid 36 (ObjNum 698 snap 193 seq_num 698) 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: finishing write tid 1 to trial17818531-19 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: finishing write tid 2 to trial17818531-19 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: finishing write tid 3 to trial17818531-19 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: finishing write tid 4 to trial17818531-19 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: finishing write tid 5 to trial17818531-19 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: oid 19 updating version 0 to 1142 2026-02-21T22:23:58.965 INFO:tasks.rados.rados.0.trial178.stdout:1976: oid 19 updating version 1142 to 1143 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1976: oid 19 version 1143 is already newer than 1141 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1976: oid 19 version 1143 is already newer than 1140 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1143 (ObjNum 699 snap 193 seq_num 699) dirty exists 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1976: left oid 19 (ObjNum 699 snap 193 seq_num 699) 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1977: finishing copy_from racing read to trial17818531-48 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1977: finishing copy_from to trial17818531-48 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1975: done (3 left) 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1976: done (2 left) 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1977: done (1 left) 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1979: write (excl) oid 24 current snap is 193 2026-02-21T22:23:58.966 INFO:tasks.rados.rados.0.trial178.stdout:1979: seq_num 700 ranges {564900=578776,1629160=709913,3073389=492409} 2026-02-21T22:23:58.969 INFO:tasks.rados.rados.0.trial178.stdout:1979: writing trial17818531-24 from 564900 to 1143676 tid 1 2026-02-21T22:23:58.977 INFO:tasks.rados.rados.0.trial178.stdout:1979: writing trial17818531-24 from 1629160 to 2339073 tid 2 2026-02-21T22:23:58.985 INFO:tasks.rados.rados.0.trial178.stdout:1979: writing trial17818531-24 from 3073389 to 3565798 tid 3 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1980: rollback oid 4 current snap is 193 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 187 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1978: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1978: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1978: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 760 (ObjNum 696 snap 192 seq_num 696) dirty exists 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1979: finishing write tid 1 to trial17818531-24 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1978: done (2 left) 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1981: delete oid 27 current snap is 193 2026-02-21T22:23:58.986 INFO:tasks.rados.rados.0.trial178.stdout:1979: finishing write tid 2 to trial17818531-24 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: finishing write tid 3 to trial17818531-24 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1980: finishing rollback tid 0 to trial17818531-4 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1980: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 777 (ObjNum 49 snap 0 seq_num 1634300532) dirty dne 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: finishing write tid 4 to trial17818531-24 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: finishing write tid 5 to trial17818531-24 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: oid 24 updating version 0 to 1145 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: oid 24 updating version 1145 to 1147 2026-02-21T22:23:58.988 INFO:tasks.rados.rados.0.trial178.stdout:1979: oid 24 version 1147 is already newer than 1146 2026-02-21T22:23:58.989 INFO:tasks.rados.rados.0.trial178.stdout:1979: oid 24 updating version 1147 to 1148 2026-02-21T22:23:58.989 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1148 (ObjNum 700 snap 193 seq_num 700) dirty exists 2026-02-21T22:23:58.989 INFO:tasks.rados.rados.0.trial178.stdout:1979: left oid 24 (ObjNum 700 snap 193 seq_num 700) 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1979: done (2 left) 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1980: done (1 left) 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1981: done (0 left) 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1982: copy_from oid 24 from oid 18 current snap is 193 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1983: read oid 36 snap -1 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1983: expect (ObjNum 698 snap 193 seq_num 698) 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1984: read oid 12 snap 191 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1984: expect deleted 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1985: write (excl) oid 41 current snap is 193 2026-02-21T22:23:58.990 INFO:tasks.rados.rados.0.trial178.stdout:1985: seq_num 702 ranges {598514=729274,1746124=597229,3128235=514953} 2026-02-21T22:23:58.998 INFO:tasks.rados.rados.0.trial178.stdout:1985: writing trial17818531-41 from 598514 to 1327788 tid 1 2026-02-21T22:23:59.005 INFO:tasks.rados.rados.0.trial178.stdout:1985: writing trial17818531-41 from 1746124 to 2343353 tid 2 2026-02-21T22:23:59.014 INFO:tasks.rados.rados.0.trial178.stdout:1985: writing trial17818531-41 from 3128235 to 3643188 tid 3 2026-02-21T22:23:59.014 INFO:tasks.rados.rados.0.trial178.stdout:1986: read oid 28 snap -1 2026-02-21T22:23:59.014 INFO:tasks.rados.rados.0.trial178.stdout:1986: expect deleted 2026-02-21T22:23:59.014 INFO:tasks.rados.rados.0.trial178.stdout:1987: snap_create 2026-02-21T22:23:59.014 INFO:tasks.rados.rados.0.trial178.stdout:1983: expect (ObjNum 698 snap 193 seq_num 698) 2026-02-21T22:23:59.185 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 574626 2026-02-21T22:23:59.254 INFO:tasks.rados.rados.0.trial178.stdout:1985: finishing write tid 1 to trial17818531-41 2026-02-21T22:23:59.254 INFO:tasks.rados.rados.0.trial178.stdout:1985: finishing write tid 2 to trial17818531-41 2026-02-21T22:23:59.254 INFO:tasks.rados.rados.0.trial178.stdout:1982: finishing copy_from racing read to trial17818531-24 2026-02-21T22:23:59.254 INFO:tasks.rados.rados.0.trial178.stdout:1982: finishing copy_from to trial17818531-24 2026-02-21T22:23:59.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 760 (ObjNum 696 snap 192 seq_num 696) dirty exists 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: finishing write tid 3 to trial17818531-41 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: finishing write tid 4 to trial17818531-41 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: finishing write tid 5 to trial17818531-41 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: oid 41 updating version 0 to 735 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: oid 41 updating version 735 to 737 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: oid 41 version 737 is already newer than 734 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: oid 41 version 737 is already newer than 736 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 737 (ObjNum 702 snap 193 seq_num 702) dirty exists 2026-02-21T22:23:59.255 INFO:tasks.rados.rados.0.trial178.stdout:1985: left oid 41 (ObjNum 702 snap 193 seq_num 702) 2026-02-21T22:23:59.762 INFO:tasks.rados.rados.0.trial178.stdout:1982: done (5 left) 2026-02-21T22:23:59.762 INFO:tasks.rados.rados.0.trial178.stdout:1983: done (4 left) 2026-02-21T22:23:59.762 INFO:tasks.rados.rados.0.trial178.stdout:1984: done (3 left) 2026-02-21T22:23:59.762 INFO:tasks.rados.rados.0.trial178.stdout:1985: done (2 left) 2026-02-21T22:23:59.763 INFO:tasks.rados.rados.0.trial178.stdout:1986: done (1 left) 2026-02-21T22:23:59.763 INFO:tasks.rados.rados.0.trial178.stdout:1987: done (0 left) 2026-02-21T22:23:59.763 INFO:tasks.rados.rados.0.trial178.stdout:1988: delete oid 31 current snap is 194 2026-02-21T22:23:59.764 INFO:tasks.rados.rados.0.trial178.stdout:1988: done (0 left) 2026-02-21T22:23:59.764 INFO:tasks.rados.rados.0.trial178.stdout:1989: write (excl) oid 36 current snap is 194 2026-02-21T22:23:59.765 INFO:tasks.rados.rados.0.trial178.stdout:1989: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2026-02-21T22:23:59.771 INFO:tasks.rados.rados.0.trial178.stdout:1989: writing trial17818531-36 from 632128 to 1111900 tid 1 2026-02-21T22:23:59.780 INFO:tasks.rados.rados.0.trial178.stdout:1989: writing trial17818531-36 from 1746735 to 2347633 tid 2 2026-02-21T22:23:59.789 INFO:tasks.rados.rados.0.trial178.stdout:1989: writing trial17818531-36 from 3066728 to 3720578 tid 3 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:1990: rollback oid 30 current snap is 194 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 30 to 193 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:1989: finishing write tid 1 to trial17818531-36 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:1989: finishing write tid 2 to trial17818531-36 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:1991: write oid 25 current snap is 194 2026-02-21T22:23:59.790 INFO:tasks.rados.rados.0.trial178.stdout:1991: seq_num 705 ranges {648935=555021,1805217=460909,2694151=465122,3817442=31493} 2026-02-21T22:23:59.797 INFO:tasks.rados.rados.0.trial178.stdout:1991: writing trial17818531-25 from 648935 to 1203956 tid 1 2026-02-21T22:23:59.804 INFO:tasks.rados.rados.0.trial178.stdout:1991: writing trial17818531-25 from 1805217 to 2266126 tid 2 2026-02-21T22:23:59.810 INFO:tasks.rados.rados.0.trial178.stdout:1991: writing trial17818531-25 from 2694151 to 3159273 tid 3 2026-02-21T22:23:59.812 INFO:tasks.rados.rados.0.trial178.stdout:1991: writing trial17818531-25 from 3817442 to 3848935 tid 4 2026-02-21T22:23:59.812 INFO:tasks.rados.rados.0.trial178.stdout:1992: read oid 7 snap -1 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1992: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1993: read oid 31 snap -1 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1993: expect deleted 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1994: delete oid 4 current snap is 194 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: finishing write tid 3 to trial17818531-36 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: finishing write tid 4 to trial17818531-36 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1990: finishing rollback tid 0 to trial17818531-30 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: finishing write tid 5 to trial17818531-36 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: oid 36 updating version 0 to 380 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: oid 36 updating version 380 to 382 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: oid 36 updating version 382 to 383 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: oid 36 version 383 is already newer than 381 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 383 (ObjNum 704 snap 194 seq_num 704) dirty exists 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1989: left oid 36 (ObjNum 704 snap 194 seq_num 704) 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1990: finishing rollback tid 1 to trial17818531-30 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1990: finishing rollback tid 2 to trial17818531-30 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 763 (ObjNum 682 snap 189 seq_num 682) dirty exists 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 1 to trial17818531-25 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 2 to trial17818531-25 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 3 to trial17818531-25 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 4 to trial17818531-25 2026-02-21T22:23:59.813 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 5 to trial17818531-25 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: finishing write tid 6 to trial17818531-25 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: oid 25 updating version 0 to 790 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: oid 25 updating version 790 to 791 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: oid 25 version 791 is already newer than 788 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: oid 25 version 791 is already newer than 789 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: oid 25 updating version 791 to 792 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 792 (ObjNum 705 snap 194 seq_num 705) dirty exists 2026-02-21T22:23:59.814 INFO:tasks.rados.rados.0.trial178.stdout:1991: left oid 25 (ObjNum 705 snap 194 seq_num 705) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1989: done (5 left) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1990: done (4 left) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1991: done (3 left) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1993: done (2 left) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1994: done (1 left) 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1995: rollback oid 28 current snap is 194 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 193 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1996: copy_from oid 35 from oid 29 current snap is 194 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1997: read oid 4 snap -1 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1997: expect deleted 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1998: write (excl) oid 50 current snap is 194 2026-02-21T22:23:59.815 INFO:tasks.rados.rados.0.trial178.stdout:1998: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2026-02-21T22:23:59.820 INFO:tasks.rados.rados.0.trial178.stdout:1998: writing trial17818531-50 from 682549 to 1388068 tid 1 2026-02-21T22:23:59.824 INFO:tasks.rados.rados.0.trial178.stdout:1998: writing trial17818531-50 from 1922181 to 2386759 tid 2 2026-02-21T22:23:59.829 INFO:tasks.rados.rados.0.trial178.stdout:1998: writing trial17818531-50 from 3148997 to 3636663 tid 3 2026-02-21T22:23:59.829 INFO:tasks.rados.rados.0.trial178.stdout:1999: snap_remove snap 192 2026-02-21T22:24:00.764 INFO:tasks.rados.rados.0.trial178.stdout:1999: done (5 left) 2026-02-21T22:24:00.764 INFO:tasks.rados.rados.0.trial178.stdout:2000: write oid 20 current snap is 194 2026-02-21T22:24:00.765 INFO:tasks.rados.rados.0.trial178.stdout:2000: seq_num 708 ranges {699356=497121,1980663=608236,3176420=415291} 2026-02-21T22:24:00.769 INFO:tasks.rados.rados.0.trial178.stdout:2000: writing trial17818531-20 from 699356 to 1196477 tid 1 2026-02-21T22:24:00.774 INFO:tasks.rados.rados.0.trial178.stdout:2000: writing trial17818531-20 from 1980663 to 2588899 tid 2 2026-02-21T22:24:00.778 INFO:tasks.rados.rados.0.trial178.stdout:2000: writing trial17818531-20 from 3176420 to 3591711 tid 3 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:2001: rollback oid 40 current snap is 194 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 193 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:2002: read oid 47 snap -1 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:2002: expect (ObjNum 653 snap 183 seq_num 653) 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:2003: write oid 46 current snap is 194 2026-02-21T22:24:00.779 INFO:tasks.rados.rados.0.trial178.stdout:2003: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2026-02-21T22:24:00.783 INFO:tasks.rados.rados.0.trial178.stdout:2003: writing trial17818531-46 from 716163 to 1288533 tid 1 2026-02-21T22:24:00.787 INFO:tasks.rados.rados.0.trial178.stdout:2003: writing trial17818531-46 from 2039145 to 2507392 tid 2 2026-02-21T22:24:00.793 INFO:tasks.rados.rados.0.trial178.stdout:2003: writing trial17818531-46 from 3203843 to 3830406 tid 3 2026-02-21T22:24:00.793 INFO:tasks.rados.rados.0.trial178.stdout:2004: write (excl) oid 41 current snap is 194 2026-02-21T22:24:00.793 INFO:tasks.rados.rados.0.trial178.stdout:2004: seq_num 710 ranges {732970=647619,2097627=611905,3231266=437835} 2026-02-21T22:24:00.798 INFO:tasks.rados.rados.0.trial178.stdout:2004: writing trial17818531-41 from 732970 to 1380589 tid 1 2026-02-21T22:24:00.803 INFO:tasks.rados.rados.0.trial178.stdout:2004: writing trial17818531-41 from 2097627 to 2709532 tid 2 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2004: writing trial17818531-41 from 3231266 to 3669101 tid 3 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2005: read oid 26 snap -1 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2005: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2006: snap_create 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:1995: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:24:00.807 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 434 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:24:00.808 INFO:tasks.rados.rados.0.trial178.stdout:1992: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:24:00.825 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2623831 2026-02-21T22:24:00.861 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:00.889 INFO:tasks.rados.rados.0.trial178.stdout:1998: finishing write tid 1 to trial17818531-50 2026-02-21T22:24:00.889 INFO:tasks.rados.rados.0.trial178.stdout:1998: finishing write tid 2 to trial17818531-50 2026-02-21T22:24:00.889 INFO:tasks.rados.rados.0.trial178.stdout:1996: finishing copy_from racing read to trial17818531-35 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1996: finishing copy_from to trial17818531-35 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 757 (ObjNum 592 snap 163 seq_num 592) dirty exists 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: finishing write tid 3 to trial17818531-50 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: finishing write tid 4 to trial17818531-50 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: finishing write tid 5 to trial17818531-50 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: oid 50 updating version 0 to 764 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: oid 50 updating version 764 to 765 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: oid 50 updating version 765 to 767 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: oid 50 version 767 is already newer than 766 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 767 (ObjNum 707 snap 194 seq_num 707) dirty exists 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:1998: left oid 50 (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:00.890 INFO:tasks.rados.rados.0.trial178.stdout:2000: finishing write tid 1 to trial17818531-20 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: finishing write tid 2 to trial17818531-20 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: finishing write tid 3 to trial17818531-20 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: finishing write tid 4 to trial17818531-20 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: finishing write tid 5 to trial17818531-20 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: oid 20 updating version 0 to 738 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: oid 20 updating version 738 to 741 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: oid 20 version 741 is already newer than 740 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: oid 20 version 741 is already newer than 739 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 741 (ObjNum 708 snap 194 seq_num 708) dirty exists 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2000: left oid 20 (ObjNum 708 snap 194 seq_num 708) 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2001: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 763 (ObjNum 33 snap 0 seq_num 99653264) dirty dne 2026-02-21T22:24:00.891 INFO:tasks.rados.rados.0.trial178.stdout:2002: expect (ObjNum 653 snap 183 seq_num 653) 2026-02-21T22:24:00.933 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 633460 2026-02-21T22:24:00.969 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: finishing write tid 1 to trial17818531-46 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: finishing write tid 2 to trial17818531-46 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: finishing write tid 3 to trial17818531-46 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: finishing write tid 4 to trial17818531-46 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: finishing write tid 5 to trial17818531-46 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: oid 46 updating version 0 to 782 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: oid 46 updating version 782 to 783 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: oid 46 version 783 is already newer than 780 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: oid 46 version 783 is already newer than 781 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 783 (ObjNum 709 snap 194 seq_num 709) dirty exists 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2003: left oid 46 (ObjNum 709 snap 194 seq_num 709) 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2004: finishing write tid 1 to trial17818531-41 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2004: finishing write tid 2 to trial17818531-41 2026-02-21T22:24:01.012 INFO:tasks.rados.rados.0.trial178.stdout:2004: finishing write tid 3 to trial17818531-41 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: finishing write tid 4 to trial17818531-41 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: finishing write tid 5 to trial17818531-41 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: oid 41 updating version 0 to 743 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: oid 41 updating version 743 to 745 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: oid 41 version 745 is already newer than 742 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: oid 41 version 745 is already newer than 744 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 745 (ObjNum 710 snap 194 seq_num 710) dirty exists 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2004: left oid 41 (ObjNum 710 snap 194 seq_num 710) 2026-02-21T22:24:01.013 INFO:tasks.rados.rados.0.trial178.stdout:2005: expect (ObjNum 564 snap 158 seq_num 564) 2026-02-21T22:24:01.065 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:01.756 INFO:tasks.rados.rados.0.trial178.stdout:1992: done (11 left) 2026-02-21T22:24:01.756 INFO:tasks.rados.rados.0.trial178.stdout:1995: done (10 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:1996: done (9 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:1997: done (8 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:1998: done (7 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2000: done (6 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2001: done (5 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2002: done (4 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2003: done (3 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2004: done (2 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2005: done (1 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2006: done (0 left) 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2007: rollback oid 28 current snap is 195 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 193 2026-02-21T22:24:01.757 INFO:tasks.rados.rados.0.trial178.stdout:2008: snap_remove snap 187 2026-02-21T22:24:03.007 INFO:tasks.rados.rados.0.trial178.stdout:2008: done (1 left) 2026-02-21T22:24:03.007 INFO:tasks.rados.rados.0.trial178.stdout:2009: delete oid 19 current snap is 195 2026-02-21T22:24:03.007 INFO:tasks.rados.rados.0.trial178.stdout:2007: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:24:03.007 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 434 (ObjNum 0 snap 0 seq_num 304) dirty dne 2026-02-21T22:24:03.009 INFO:tasks.rados.rados.0.trial178.stdout:2007: done (1 left) 2026-02-21T22:24:03.009 INFO:tasks.rados.rados.0.trial178.stdout:2009: done (0 left) 2026-02-21T22:24:03.009 INFO:tasks.rados.rados.0.trial178.stdout:2010: delete oid 5 current snap is 195 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:2010: done (0 left) 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:2011: rollback oid 31 current snap is 195 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 194 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:2012: copy_from oid 6 from oid 7 current snap is 195 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:2013: write oid 30 current snap is 195 2026-02-21T22:24:03.011 INFO:tasks.rados.rados.0.trial178.stdout:2013: seq_num 713 ranges {783391=473366,1756720=475585,2797182=788004} 2026-02-21T22:24:03.020 INFO:tasks.rados.rados.0.trial178.stdout:2013: writing trial17818531-30 from 783391 to 1256757 tid 1 2026-02-21T22:24:03.026 INFO:tasks.rados.rados.0.trial178.stdout:2013: writing trial17818531-30 from 1756720 to 2232305 tid 2 2026-02-21T22:24:03.035 INFO:tasks.rados.rados.0.trial178.stdout:2013: writing trial17818531-30 from 2797182 to 3585186 tid 3 2026-02-21T22:24:03.035 INFO:tasks.rados.rados.0.trial178.stdout:2014: copy_from oid 46 from oid 11 current snap is 195 2026-02-21T22:24:03.035 INFO:tasks.rados.rados.0.trial178.stdout:2011: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:24:03.035 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 787 (ObjNum 129 snap 0 seq_num 99722848) dirty dne 2026-02-21T22:24:03.035 INFO:tasks.rados.rados.0.trial178.stdout:2013: finishing write tid 1 to trial17818531-30 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2012: finishing copy_from racing read to trial17818531-6 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2012: finishing copy_from to trial17818531-6 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2013: finishing write tid 2 to trial17818531-30 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2011: done (3 left) 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2012: done (2 left) 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2015: write oid 35 current snap is 195 2026-02-21T22:24:03.036 INFO:tasks.rados.rados.0.trial178.stdout:2015: seq_num 714 ranges {400198=548615,1415202=584996} 2026-02-21T22:24:03.042 INFO:tasks.rados.rados.0.trial178.stdout:2015: writing trial17818531-35 from 400198 to 948813 tid 1 2026-02-21T22:24:03.049 INFO:tasks.rados.rados.0.trial178.stdout:2015: writing trial17818531-35 from 1415202 to 2000198 tid 2 2026-02-21T22:24:03.049 INFO:tasks.rados.rados.0.trial178.stdout:2016: read oid 41 snap -1 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2016: expect (ObjNum 710 snap 194 seq_num 710) 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2017: snap_create 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: finishing write tid 3 to trial17818531-30 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: finishing write tid 4 to trial17818531-30 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: finishing write tid 5 to trial17818531-30 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: oid 30 updating version 0 to 765 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: oid 30 updating version 765 to 767 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: oid 30 version 767 is already newer than 766 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: oid 30 version 767 is already newer than 764 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 767 (ObjNum 713 snap 195 seq_num 713) dirty exists 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2013: left oid 30 (ObjNum 713 snap 195 seq_num 713) 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2015: finishing write tid 1 to trial17818531-35 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2014: finishing copy_from racing read to trial17818531-46 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:2014: finishing copy_from to trial17818531-46 2026-02-21T22:24:03.050 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 991 (ObjNum 635 snap 180 seq_num 635) dirty exists 2026-02-21T22:24:03.059 INFO:tasks.rados.rados.0.trial178.stdout:2015: finishing write tid 2 to trial17818531-35 2026-02-21T22:24:03.059 INFO:tasks.rados.rados.0.trial178.stdout:2015: finishing write tid 3 to trial17818531-35 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:2015: finishing write tid 4 to trial17818531-35 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:2015: oid 35 updating version 0 to 997 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:2015: oid 35 version 997 is already newer than 996 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:2015: oid 35 updating version 997 to 998 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 998 (ObjNum 714 snap 195 seq_num 714) dirty exists 2026-02-21T22:24:03.060 INFO:tasks.rados.rados.0.trial178.stdout:2015: left oid 35 (ObjNum 714 snap 195 seq_num 714) 2026-02-21T22:24:03.062 INFO:tasks.rados.rados.0.trial178.stdout:2016: expect (ObjNum 710 snap 194 seq_num 710) 2026-02-21T22:24:03.062 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:03.124 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:03.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:03.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2013: done (4 left) 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2014: done (3 left) 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2015: done (2 left) 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2016: done (1 left) 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2017: done (0 left) 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2018: write (excl) oid 37 current snap is 196 2026-02-21T22:24:03.782 INFO:tasks.rados.rados.0.trial178.stdout:2018: seq_num 715 ranges {417005=740217,1590037=426968} 2026-02-21T22:24:03.790 INFO:tasks.rados.rados.0.trial178.stdout:2018: writing trial17818531-37 from 417005 to 1157222 tid 1 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2018: writing trial17818531-37 from 1590037 to 2017005 tid 2 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2019: rollback oid 13 current snap is 196 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 194 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2018: finishing write tid 1 to trial17818531-37 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2020: read oid 8 snap -1 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2020: expect (ObjNum 670 snap 189 seq_num 670) 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2021: read oid 31 snap -1 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2021: expect deleted 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2022: write oid 34 current snap is 196 2026-02-21T22:24:03.796 INFO:tasks.rados.rados.0.trial178.stdout:2022: seq_num 716 ranges {433812=415466,1648519=385293} 2026-02-21T22:24:03.801 INFO:tasks.rados.rados.0.trial178.stdout:2022: writing trial17818531-34 from 433812 to 849278 tid 1 2026-02-21T22:24:03.809 INFO:tasks.rados.rados.0.trial178.stdout:2022: writing trial17818531-34 from 1648519 to 2033812 tid 2 2026-02-21T22:24:03.809 INFO:tasks.rados.rados.0.trial178.stdout:2023: snap_remove snap 193 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2023: done (5 left) 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2024: read oid 25 snap 194 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2024: expect (ObjNum 705 snap 194 seq_num 705) 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2025: rollback oid 7 current snap is 196 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 7 to 195 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: finishing write tid 2 to trial17818531-37 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2019: finishing rollback tid 0 to trial17818531-13 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: finishing write tid 3 to trial17818531-37 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: finishing write tid 4 to trial17818531-37 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: oid 37 updating version 0 to 1130 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: oid 37 version 1130 is already newer than 1129 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: oid 37 version 1130 is already newer than 1128 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1130 (ObjNum 715 snap 196 seq_num 715) dirty exists 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2018: left oid 37 (ObjNum 715 snap 196 seq_num 715) 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2019: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2019: finishing rollback tid 2 to trial17818531-13 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1001 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2022: finishing write tid 1 to trial17818531-34 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:2020: expect (ObjNum 670 snap 189 seq_num 670) 2026-02-21T22:24:04.787 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:04.887 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: finishing write tid 2 to trial17818531-34 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: finishing write tid 3 to trial17818531-34 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: finishing write tid 4 to trial17818531-34 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: oid 34 updating version 0 to 765 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: oid 34 version 765 is already newer than 764 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:2022: oid 34 version 765 is already newer than 763 2026-02-21T22:24:04.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 765 (ObjNum 716 snap 196 seq_num 716) dirty exists 2026-02-21T22:24:04.931 INFO:tasks.rados.rados.0.trial178.stdout:2022: left oid 34 (ObjNum 716 snap 196 seq_num 716) 2026-02-21T22:24:04.931 INFO:tasks.rados.rados.0.trial178.stdout:2024: expect (ObjNum 705 snap 194 seq_num 705) 2026-02-21T22:24:05.034 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3447712 2026-02-21T22:24:05.041 INFO:tasks.rados.rados.0.trial178.stdout:2025: finishing rollback tid 0 to trial17818531-7 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2025: finishing rollback tid 1 to trial17818531-7 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2025: finishing rollback tid 2 to trial17818531-7 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1169 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2018: done (6 left) 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2019: done (5 left) 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2020: done (4 left) 2026-02-21T22:24:05.042 INFO:tasks.rados.rados.0.trial178.stdout:2021: done (3 left) 2026-02-21T22:24:05.043 INFO:tasks.rados.rados.0.trial178.stdout:2022: done (2 left) 2026-02-21T22:24:05.043 INFO:tasks.rados.rados.0.trial178.stdout:2024: done (1 left) 2026-02-21T22:24:05.043 INFO:tasks.rados.rados.0.trial178.stdout:2025: done (0 left) 2026-02-21T22:24:05.043 INFO:tasks.rados.rados.0.trial178.stdout:2026: snap_remove snap 194 2026-02-21T22:24:05.790 INFO:tasks.rados.rados.0.trial178.stdout:2026: done (0 left) 2026-02-21T22:24:05.790 INFO:tasks.rados.rados.0.trial178.stdout:2027: read oid 35 snap -1 2026-02-21T22:24:05.790 INFO:tasks.rados.rados.0.trial178.stdout:2027: expect (ObjNum 714 snap 195 seq_num 714) 2026-02-21T22:24:05.791 INFO:tasks.rados.rados.0.trial178.stdout:2028: delete oid 43 current snap is 196 2026-02-21T22:24:05.795 INFO:tasks.rados.rados.0.trial178.stdout:2028: done (1 left) 2026-02-21T22:24:05.795 INFO:tasks.rados.rados.0.trial178.stdout:2029: snap_create 2026-02-21T22:24:05.797 INFO:tasks.rados.rados.0.trial178.stdout:2027: expect (ObjNum 714 snap 195 seq_num 714) 2026-02-21T22:24:05.832 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 139654 2026-02-21T22:24:06.756 INFO:tasks.rados.rados.0.trial178.stdout:2027: done (1 left) 2026-02-21T22:24:06.756 INFO:tasks.rados.rados.0.trial178.stdout:2029: done (0 left) 2026-02-21T22:24:06.756 INFO:tasks.rados.rados.0.trial178.stdout:2030: snap_create 2026-02-21T22:24:07.757 INFO:tasks.rados.rados.0.trial178.stdout:2030: done (0 left) 2026-02-21T22:24:07.757 INFO:tasks.rados.rados.0.trial178.stdout:2031: snap_remove snap 191 2026-02-21T22:24:08.562 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:08.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:08.760 INFO:tasks.rados.rados.0.trial178.stdout:2031: done (0 left) 2026-02-21T22:24:08.760 INFO:tasks.rados.rados.0.trial178.stdout:2032: snap_create 2026-02-21T22:24:09.761 INFO:tasks.rados.rados.0.trial178.stdout:2032: done (0 left) 2026-02-21T22:24:09.761 INFO:tasks.rados.rados.0.trial178.stdout:2033: delete oid 43 current snap is 199 2026-02-21T22:24:09.762 INFO:tasks.rados.rados.0.trial178.stdout:2033: done (0 left) 2026-02-21T22:24:09.762 INFO:tasks.rados.rados.0.trial178.stdout:2034: copy_from oid 2 from oid 28 current snap is 199 2026-02-21T22:24:09.762 INFO:tasks.rados.rados.0.trial178.stdout:2035: copy_from oid 30 from oid 43 current snap is 199 2026-02-21T22:24:09.762 INFO:tasks.rados.rados.0.trial178.stdout:2036: delete oid 41 current snap is 199 2026-02-21T22:24:09.763 INFO:tasks.rados.rados.0.trial178.stdout:2035: finishing copy_from to trial17818531-30 2026-02-21T22:24:09.763 INFO:tasks.rados.rados.0.trial178.stdout:2035: got expected ENOENT (src dne) 2026-02-21T22:24:09.763 INFO:tasks.rados.rados.0.trial178.stdout:2035: finishing copy_from racing read to trial17818531-30 2026-02-21T22:24:09.766 INFO:tasks.rados.rados.0.trial178.stdout:2035: done (2 left) 2026-02-21T22:24:09.766 INFO:tasks.rados.rados.0.trial178.stdout:2036: done (1 left) 2026-02-21T22:24:09.766 INFO:tasks.rados.rados.0.trial178.stdout:2037: read oid 41 snap -1 2026-02-21T22:24:09.766 INFO:tasks.rados.rados.0.trial178.stdout:2037: expect deleted 2026-02-21T22:24:09.767 INFO:tasks.rados.rados.0.trial178.stdout:2038: snap_create 2026-02-21T22:24:09.767 INFO:tasks.rados.rados.0.trial178.stdout:2034: finishing copy_from to trial17818531-2 2026-02-21T22:24:09.767 INFO:tasks.rados.rados.0.trial178.stdout:2034: got expected ENOENT (src dne) 2026-02-21T22:24:09.767 INFO:tasks.rados.rados.0.trial178.stdout:2034: finishing copy_from racing read to trial17818531-2 2026-02-21T22:24:10.767 INFO:tasks.rados.rados.0.trial178.stdout:2034: done (2 left) 2026-02-21T22:24:10.767 INFO:tasks.rados.rados.0.trial178.stdout:2037: done (1 left) 2026-02-21T22:24:10.767 INFO:tasks.rados.rados.0.trial178.stdout:2038: done (0 left) 2026-02-21T22:24:10.768 INFO:tasks.rados.rados.0.trial178.stdout:2039: rollback oid 3 current snap is 200 2026-02-21T22:24:10.768 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 197 2026-02-21T22:24:10.768 INFO:tasks.rados.rados.0.trial178.stdout:2040: write (excl) oid 10 current snap is 200 2026-02-21T22:24:10.768 INFO:tasks.rados.rados.0.trial178.stdout:2040: seq_num 720 ranges {501040=716462,1766094=334946} 2026-02-21T22:24:10.775 INFO:tasks.rados.rados.0.trial178.stdout:2040: writing trial17818531-10 from 501040 to 1217502 tid 1 2026-02-21T22:24:10.781 INFO:tasks.rados.rados.0.trial178.stdout:2040: writing trial17818531-10 from 1766094 to 2101040 tid 2 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2041: read oid 4 snap -1 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2041: expect deleted 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2042: copy_from oid 29 from oid 1 current snap is 200 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2039: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2040: finishing write tid 1 to trial17818531-10 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2039: done (3 left) 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2043: read oid 23 snap -1 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2043: expect (ObjNum 686 snap 190 seq_num 686) 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2044: copy_from oid 31 from oid 50 current snap is 200 2026-02-21T22:24:10.782 INFO:tasks.rados.rados.0.trial178.stdout:2045: rollback oid 49 current snap is 200 2026-02-21T22:24:10.783 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 198 2026-02-21T22:24:10.783 INFO:tasks.rados.rados.0.trial178.stdout:2046: snap_remove snap 196 2026-02-21T22:24:11.759 INFO:tasks.rados.rados.0.trial178.stdout:2046: done (6 left) 2026-02-21T22:24:11.759 INFO:tasks.rados.rados.0.trial178.stdout:2047: rollback oid 18 current snap is 200 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 195 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: finishing write tid 2 to trial17818531-10 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: finishing write tid 3 to trial17818531-10 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: finishing write tid 4 to trial17818531-10 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: oid 10 updating version 0 to 1130 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: oid 10 version 1130 is already newer than 1128 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: oid 10 version 1130 is already newer than 1129 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1130 (ObjNum 720 snap 200 seq_num 720) dirty exists 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2040: left oid 10 (ObjNum 720 snap 200 seq_num 720) 2026-02-21T22:24:11.760 INFO:tasks.rados.rados.0.trial178.stdout:2043: expect (ObjNum 686 snap 190 seq_num 686) 2026-02-21T22:24:11.798 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3191876 2026-02-21T22:24:11.817 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2192818 2026-02-21T22:24:11.827 INFO:tasks.rados.rados.0.trial178.stdout:2045: finishing rollback tid 0 to trial17818531-49 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2045: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2045: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 795 (ObjNum 666 snap 188 seq_num 666) dirty exists 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2042: finishing copy_from racing read to trial17818531-29 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2042: finishing copy_from to trial17818531-29 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 786 (ObjNum 688 snap 190 seq_num 688) dirty exists 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2044: finishing copy_from racing read to trial17818531-31 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2044: finishing copy_from to trial17818531-31 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 767 (ObjNum 707 snap 194 seq_num 707) dirty exists 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2047: finishing rollback tid 0 to trial17818531-18 2026-02-21T22:24:11.828 INFO:tasks.rados.rados.0.trial178.stdout:2047: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2047: finishing rollback tid 2 to trial17818531-18 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 770 (ObjNum 696 snap 192 seq_num 696) dirty exists 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2040: done (6 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2041: done (5 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2042: done (4 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2043: done (3 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2044: done (2 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2045: done (1 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2047: done (0 left) 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2048: read oid 22 snap -1 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2048: expect deleted 2026-02-21T22:24:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2049: snap_create 2026-02-21T22:24:12.772 INFO:tasks.rados.rados.0.trial178.stdout:2048: done (1 left) 2026-02-21T22:24:12.772 INFO:tasks.rados.rados.0.trial178.stdout:2049: done (0 left) 2026-02-21T22:24:12.772 INFO:tasks.rados.rados.0.trial178.stdout:2050: write oid 18 current snap is 201 2026-02-21T22:24:12.773 INFO:tasks.rados.rados.0.trial178.stdout:2050: seq_num 721 ranges {517847=791711,2108223=9624} 2026-02-21T22:24:12.781 INFO:tasks.rados.rados.0.trial178.stdout:2050: writing trial17818531-18 from 517847 to 1309558 tid 1 2026-02-21T22:24:12.788 INFO:tasks.rados.rados.0.trial178.stdout:2050: writing trial17818531-18 from 2108223 to 2117847 tid 2 2026-02-21T22:24:12.788 INFO:tasks.rados.rados.0.trial178.stdout:2051: read oid 23 snap -1 2026-02-21T22:24:12.788 INFO:tasks.rados.rados.0.trial178.stdout:2051: expect (ObjNum 686 snap 190 seq_num 686) 2026-02-21T22:24:12.788 INFO:tasks.rados.rados.0.trial178.stdout:2052: write oid 36 current snap is 201 2026-02-21T22:24:12.788 INFO:tasks.rados.rados.0.trial178.stdout:2052: seq_num 722 ranges {534654=466960,1766705=367949} 2026-02-21T22:24:12.794 INFO:tasks.rados.rados.0.trial178.stdout:2052: writing trial17818531-36 from 534654 to 1001614 tid 1 2026-02-21T22:24:12.802 INFO:tasks.rados.rados.0.trial178.stdout:2052: writing trial17818531-36 from 1766705 to 2134654 tid 2 2026-02-21T22:24:12.802 INFO:tasks.rados.rados.0.trial178.stdout:2053: snap_create 2026-02-21T22:24:12.802 INFO:tasks.rados.rados.0.trial178.stdout:2050: finishing write tid 1 to trial17818531-18 2026-02-21T22:24:12.802 INFO:tasks.rados.rados.0.trial178.stdout:2050: finishing write tid 2 to trial17818531-18 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: finishing write tid 3 to trial17818531-18 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: finishing write tid 4 to trial17818531-18 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: oid 18 updating version 0 to 773 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: oid 18 version 773 is already newer than 771 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: oid 18 version 773 is already newer than 772 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 773 (ObjNum 721 snap 201 seq_num 721) dirty exists 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2050: left oid 18 (ObjNum 721 snap 201 seq_num 721) 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:2051: expect (ObjNum 686 snap 190 seq_num 686) 2026-02-21T22:24:12.803 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:12.850 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1337029 2026-02-21T22:24:12.868 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 223069 2026-02-21T22:24:12.893 INFO:tasks.rados.rados.0.trial178.stdout:2052: finishing write tid 1 to trial17818531-36 2026-02-21T22:24:12.893 INFO:tasks.rados.rados.0.trial178.stdout:2052: finishing write tid 2 to trial17818531-36 2026-02-21T22:24:12.893 INFO:tasks.rados.rados.0.trial178.stdout:2052: finishing write tid 3 to trial17818531-36 2026-02-21T22:24:12.893 INFO:tasks.rados.rados.0.trial178.stdout:2052: finishing write tid 4 to trial17818531-36 2026-02-21T22:24:12.894 INFO:tasks.rados.rados.0.trial178.stdout:2052: oid 36 updating version 0 to 385 2026-02-21T22:24:12.894 INFO:tasks.rados.rados.0.trial178.stdout:2052: oid 36 updating version 385 to 386 2026-02-21T22:24:12.894 INFO:tasks.rados.rados.0.trial178.stdout:2052: oid 36 version 386 is already newer than 384 2026-02-21T22:24:12.894 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 386 (ObjNum 722 snap 201 seq_num 722) dirty exists 2026-02-21T22:24:12.894 INFO:tasks.rados.rados.0.trial178.stdout:2052: left oid 36 (ObjNum 722 snap 201 seq_num 722) 2026-02-21T22:24:13.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:13.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:13.777 INFO:tasks.rados.rados.0.trial178.stdout:2050: done (3 left) 2026-02-21T22:24:13.777 INFO:tasks.rados.rados.0.trial178.stdout:2051: done (2 left) 2026-02-21T22:24:13.777 INFO:tasks.rados.rados.0.trial178.stdout:2052: done (1 left) 2026-02-21T22:24:13.777 INFO:tasks.rados.rados.0.trial178.stdout:2053: done (0 left) 2026-02-21T22:24:13.777 INFO:tasks.rados.rados.0.trial178.stdout:2054: read oid 31 snap -1 2026-02-21T22:24:13.778 INFO:tasks.rados.rados.0.trial178.stdout:2054: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:13.778 INFO:tasks.rados.rados.0.trial178.stdout:2055: write (excl) oid 41 current snap is 202 2026-02-21T22:24:13.778 INFO:tasks.rados.rados.0.trial178.stdout:2055: seq_num 723 ranges {551461=658562,1941540=209921} 2026-02-21T22:24:13.784 INFO:tasks.rados.rados.0.trial178.stdout:2055: writing trial17818531-41 from 551461 to 1210023 tid 1 2026-02-21T22:24:13.791 INFO:tasks.rados.rados.0.trial178.stdout:2055: writing trial17818531-41 from 1941540 to 2151461 tid 2 2026-02-21T22:24:13.791 INFO:tasks.rados.rados.0.trial178.stdout:2056: snap_create 2026-02-21T22:24:13.791 INFO:tasks.rados.rados.0.trial178.stdout:2055: finishing write tid 1 to trial17818531-41 2026-02-21T22:24:13.791 INFO:tasks.rados.rados.0.trial178.stdout:2054: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:13.884 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3126589 2026-02-21T22:24:13.895 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: finishing write tid 2 to trial17818531-41 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: finishing write tid 3 to trial17818531-41 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: finishing write tid 4 to trial17818531-41 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: oid 41 updating version 0 to 748 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: oid 41 updating version 748 to 749 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: oid 41 version 749 is already newer than 747 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 749 (ObjNum 723 snap 202 seq_num 723) dirty exists 2026-02-21T22:24:13.953 INFO:tasks.rados.rados.0.trial178.stdout:2055: left oid 41 (ObjNum 723 snap 202 seq_num 723) 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2054: done (2 left) 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2055: done (1 left) 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2056: done (0 left) 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2057: copy_from oid 49 from oid 4 current snap is 203 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2058: copy_from oid 22 from oid 18 current snap is 203 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2059: copy_from oid 35 from oid 1 current snap is 203 2026-02-21T22:24:14.778 INFO:tasks.rados.rados.0.trial178.stdout:2060: snap_create 2026-02-21T22:24:14.781 INFO:tasks.rados.rados.0.trial178.stdout:2057: finishing copy_from to trial17818531-49 2026-02-21T22:24:14.781 INFO:tasks.rados.rados.0.trial178.stdout:2057: got expected ENOENT (src dne) 2026-02-21T22:24:14.781 INFO:tasks.rados.rados.0.trial178.stdout:2057: finishing copy_from racing read to trial17818531-49 2026-02-21T22:24:14.788 INFO:tasks.rados.rados.0.trial178.stdout:2058: finishing copy_from racing read to trial17818531-22 2026-02-21T22:24:14.789 INFO:tasks.rados.rados.0.trial178.stdout:2058: finishing copy_from to trial17818531-22 2026-02-21T22:24:14.789 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 773 (ObjNum 721 snap 201 seq_num 721) dirty exists 2026-02-21T22:24:14.794 INFO:tasks.rados.rados.0.trial178.stdout:2059: finishing copy_from racing read to trial17818531-35 2026-02-21T22:24:14.796 INFO:tasks.rados.rados.0.trial178.stdout:2059: finishing copy_from to trial17818531-35 2026-02-21T22:24:14.796 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 786 (ObjNum 688 snap 190 seq_num 688) dirty exists 2026-02-21T22:24:15.779 INFO:tasks.rados.rados.0.trial178.stdout:2057: done (3 left) 2026-02-21T22:24:15.779 INFO:tasks.rados.rados.0.trial178.stdout:2058: done (2 left) 2026-02-21T22:24:15.779 INFO:tasks.rados.rados.0.trial178.stdout:2059: done (1 left) 2026-02-21T22:24:15.779 INFO:tasks.rados.rados.0.trial178.stdout:2060: done (0 left) 2026-02-21T22:24:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2061: rollback oid 32 current snap is 204 2026-02-21T22:24:15.780 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 203 2026-02-21T22:24:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2062: write oid 20 current snap is 204 2026-02-21T22:24:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2062: seq_num 724 ranges {568268=733811,2000022=168246} 2026-02-21T22:24:15.788 INFO:tasks.rados.rados.0.trial178.stdout:2062: writing trial17818531-20 from 568268 to 1302079 tid 1 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2062: writing trial17818531-20 from 2000022 to 2168268 tid 2 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2063: read oid 41 snap -1 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2063: expect (ObjNum 723 snap 202 seq_num 723) 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2064: snap_create 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2061: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2061: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2061: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1004 (ObjNum 681 snap 189 seq_num 681) dirty exists 2026-02-21T22:24:15.794 INFO:tasks.rados.rados.0.trial178.stdout:2062: finishing write tid 1 to trial17818531-20 2026-02-21T22:24:15.799 INFO:tasks.rados.rados.0.trial178.stdout:2062: finishing write tid 2 to trial17818531-20 2026-02-21T22:24:15.799 INFO:tasks.rados.rados.0.trial178.stdout:2062: finishing write tid 3 to trial17818531-20 2026-02-21T22:24:15.799 INFO:tasks.rados.rados.0.trial178.stdout:2062: finishing write tid 4 to trial17818531-20 2026-02-21T22:24:15.799 INFO:tasks.rados.rados.0.trial178.stdout:2062: oid 20 updating version 0 to 750 2026-02-21T22:24:15.799 INFO:tasks.rados.rados.0.trial178.stdout:2062: oid 20 updating version 750 to 752 2026-02-21T22:24:15.800 INFO:tasks.rados.rados.0.trial178.stdout:2062: oid 20 version 752 is already newer than 751 2026-02-21T22:24:15.800 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 752 (ObjNum 724 snap 204 seq_num 724) dirty exists 2026-02-21T22:24:15.800 INFO:tasks.rados.rados.0.trial178.stdout:2062: left oid 20 (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:24:15.800 INFO:tasks.rados.rados.0.trial178.stdout:2063: expect (ObjNum 723 snap 202 seq_num 723) 2026-02-21T22:24:15.800 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:16.758 INFO:tasks.rados.rados.0.trial178.stdout:2061: done (3 left) 2026-02-21T22:24:16.759 INFO:tasks.rados.rados.0.trial178.stdout:2062: done (2 left) 2026-02-21T22:24:16.759 INFO:tasks.rados.rados.0.trial178.stdout:2063: done (1 left) 2026-02-21T22:24:16.759 INFO:tasks.rados.rados.0.trial178.stdout:2064: done (0 left) 2026-02-21T22:24:16.759 INFO:tasks.rados.rados.0.trial178.stdout:2065: snap_remove snap 201 2026-02-21T22:24:17.759 INFO:tasks.rados.rados.0.trial178.stdout:2065: done (0 left) 2026-02-21T22:24:17.759 INFO:tasks.rados.rados.0.trial178.stdout:2066: snap_create 2026-02-21T22:24:18.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:18.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:18.760 INFO:tasks.rados.rados.0.trial178.stdout:2066: done (0 left) 2026-02-21T22:24:18.760 INFO:tasks.rados.rados.0.trial178.stdout:2067: snap_create 2026-02-21T22:24:19.941 INFO:tasks.rados.rados.0.trial178.stdout:2067: done (0 left) 2026-02-21T22:24:19.941 INFO:tasks.rados.rados.0.trial178.stdout:2068: write oid 34 current snap is 207 2026-02-21T22:24:19.942 INFO:tasks.rados.rados.0.trial178.stdout:2068: seq_num 725 ranges {585075=409060,1542151=642924} 2026-02-21T22:24:19.948 INFO:tasks.rados.rados.0.trial178.stdout:2068: writing trial17818531-34 from 585075 to 994135 tid 1 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2068: writing trial17818531-34 from 1542151 to 2185075 tid 2 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2069: copy_from oid 49 from oid 27 current snap is 207 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2068: finishing write tid 1 to trial17818531-34 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2070: read oid 41 snap -1 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2070: expect (ObjNum 723 snap 202 seq_num 723) 2026-02-21T22:24:19.956 INFO:tasks.rados.rados.0.trial178.stdout:2071: snap_remove snap 203 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2071: done (3 left) 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2072: delete oid 35 current snap is 207 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2069: finishing copy_from to trial17818531-49 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2069: got expected ENOENT (src dne) 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2069: finishing copy_from racing read to trial17818531-49 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2068: finishing write tid 2 to trial17818531-34 2026-02-21T22:24:20.767 INFO:tasks.rados.rados.0.trial178.stdout:2068: finishing write tid 3 to trial17818531-34 2026-02-21T22:24:20.768 INFO:tasks.rados.rados.0.trial178.stdout:2070: expect (ObjNum 723 snap 202 seq_num 723) 2026-02-21T22:24:20.768 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:2068: finishing write tid 4 to trial17818531-34 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:2068: oid 34 updating version 0 to 787 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:2068: oid 34 updating version 787 to 789 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:2068: oid 34 version 789 is already newer than 788 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 789 (ObjNum 725 snap 207 seq_num 725) dirty exists 2026-02-21T22:24:20.831 INFO:tasks.rados.rados.0.trial178.stdout:2068: left oid 34 (ObjNum 725 snap 207 seq_num 725) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2068: done (3 left) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2069: done (2 left) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2070: done (1 left) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2072: done (0 left) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2073: read oid 39 snap -1 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2073: expect (ObjNum 625 snap 178 seq_num 625) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2074: read oid 25 snap -1 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2074: expect (ObjNum 705 snap 194 seq_num 705) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2075: read oid 2 snap -1 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2075: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2076: read oid 13 snap -1 2026-02-21T22:24:20.832 INFO:tasks.rados.rados.0.trial178.stdout:2076: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:24:20.833 INFO:tasks.rados.rados.0.trial178.stdout:2077: rollback oid 14 current snap is 207 2026-02-21T22:24:20.833 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 195 2026-02-21T22:24:20.833 INFO:tasks.rados.rados.0.trial178.stdout:2078: delete oid 3 current snap is 207 2026-02-21T22:24:20.834 INFO:tasks.rados.rados.0.trial178.stdout:2076: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:24:20.931 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1984332 2026-02-21T22:24:20.995 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2305505 2026-02-21T22:24:21.021 INFO:tasks.rados.rados.0.trial178.stdout:2073: expect (ObjNum 625 snap 178 seq_num 625) 2026-02-21T22:24:21.037 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:21.055 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:2077: finishing rollback tid 0 to trial17818531-14 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:2077: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:2077: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1097 (ObjNum 598 snap 167 seq_num 598) dirty exists 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:2074: expect (ObjNum 705 snap 194 seq_num 705) 2026-02-21T22:24:21.074 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:21.195 INFO:tasks.rados.rados.0.trial178.stdout:2075: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:21.195 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:21.253 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 362552 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2073: done (5 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2074: done (4 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2075: done (3 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2076: done (2 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2077: done (1 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2078: done (0 left) 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2079: rollback oid 15 current snap is 207 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 198 2026-02-21T22:24:21.284 INFO:tasks.rados.rados.0.trial178.stdout:2080: delete oid 33 current snap is 207 2026-02-21T22:24:21.286 INFO:tasks.rados.rados.0.trial178.stdout:2079: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:24:21.287 INFO:tasks.rados.rados.0.trial178.stdout:2079: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:24:21.287 INFO:tasks.rados.rados.0.trial178.stdout:2079: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:24:21.287 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 755 (ObjNum 593 snap 163 seq_num 593) dirty exists 2026-02-21T22:24:21.296 INFO:tasks.rados.rados.0.trial178.stdout:2079: done (1 left) 2026-02-21T22:24:21.296 INFO:tasks.rados.rados.0.trial178.stdout:2080: done (0 left) 2026-02-21T22:24:21.296 INFO:tasks.rados.rados.0.trial178.stdout:2081: snap_create 2026-02-21T22:24:21.771 INFO:tasks.rados.rados.0.trial178.stdout:2081: done (0 left) 2026-02-21T22:24:21.771 INFO:tasks.rados.rados.0.trial178.stdout:2082: copy_from oid 30 from oid 6 current snap is 208 2026-02-21T22:24:21.771 INFO:tasks.rados.rados.0.trial178.stdout:2083: write (excl) oid 5 current snap is 208 2026-02-21T22:24:21.772 INFO:tasks.rados.rados.0.trial178.stdout:2083: seq_num 729 ranges {652303=710056,2059726=192577} 2026-02-21T22:24:21.780 INFO:tasks.rados.rados.0.trial178.stdout:2083: writing trial17818531-5 from 652303 to 1362359 tid 1 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2083: writing trial17818531-5 from 2059726 to 2252303 tid 2 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2084: read oid 44 snap -1 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2084: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2085: read oid 4 snap 207 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2085: expect deleted 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2086: write (excl) oid 12 current snap is 208 2026-02-21T22:24:21.787 INFO:tasks.rados.rados.0.trial178.stdout:2086: seq_num 730 ranges {669110=501658,1834561=434549} 2026-02-21T22:24:21.794 INFO:tasks.rados.rados.0.trial178.stdout:2086: writing trial17818531-12 from 669110 to 1170768 tid 1 2026-02-21T22:24:21.802 INFO:tasks.rados.rados.0.trial178.stdout:2086: writing trial17818531-12 from 1834561 to 2269110 tid 2 2026-02-21T22:24:21.802 INFO:tasks.rados.rados.0.trial178.stdout:2087: rollback oid 26 current snap is 208 2026-02-21T22:24:21.802 INFO:tasks.rados.rados.0.trial178.stdout:2082: finishing copy_from racing read to trial17818531-30 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2082: finishing copy_from to trial17818531-30 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: finishing write tid 1 to trial17818531-5 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: finishing write tid 2 to trial17818531-5 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: finishing write tid 3 to trial17818531-5 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: finishing write tid 4 to trial17818531-5 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: oid 5 updating version 0 to 1130 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: oid 5 version 1130 is already newer than 1129 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: oid 5 version 1130 is already newer than 1128 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1130 (ObjNum 729 snap 208 seq_num 729) dirty exists 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2083: left oid 5 (ObjNum 729 snap 208 seq_num 729) 2026-02-21T22:24:21.803 INFO:tasks.rados.rados.0.trial178.stdout:2084: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:24:21.882 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 720155 2026-02-21T22:24:21.928 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:21.986 INFO:tasks.rados.rados.0.trial178.stdout:2086: finishing write tid 1 to trial17818531-12 2026-02-21T22:24:21.986 INFO:tasks.rados.rados.0.trial178.stdout:2086: finishing write tid 2 to trial17818531-12 2026-02-21T22:24:21.986 INFO:tasks.rados.rados.0.trial178.stdout:2086: finishing write tid 3 to trial17818531-12 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: finishing write tid 4 to trial17818531-12 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: oid 12 updating version 0 to 779 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: oid 12 updating version 779 to 780 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: oid 12 version 780 is already newer than 778 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 780 (ObjNum 730 snap 208 seq_num 730) dirty exists 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: left oid 12 (ObjNum 730 snap 208 seq_num 730) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 202 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2082: done (5 left) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2083: done (4 left) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2084: done (3 left) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2085: done (2 left) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2086: done (1 left) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2088: copy_from oid 6 from oid 40 current snap is 208 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2089: read oid 10 snap -1 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2089: expect (ObjNum 720 snap 200 seq_num 720) 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2090: write (excl) oid 11 current snap is 208 2026-02-21T22:24:21.987 INFO:tasks.rados.rados.0.trial178.stdout:2090: seq_num 731 ranges {685917=576907,1893043=392874} 2026-02-21T22:24:21.995 INFO:tasks.rados.rados.0.trial178.stdout:2090: writing trial17818531-11 from 685917 to 1262824 tid 1 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2090: writing trial17818531-11 from 1893043 to 2285917 tid 2 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2091: read oid 8 snap -1 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2091: expect (ObjNum 670 snap 189 seq_num 670) 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2092: read oid 28 snap -1 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2092: expect deleted 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2093: read oid 44 snap -1 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2093: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2094: copy_from oid 23 from oid 29 current snap is 208 2026-02-21T22:24:22.002 INFO:tasks.rados.rados.0.trial178.stdout:2087: finishing rollback tid 0 to trial17818531-26 2026-02-21T22:24:22.003 INFO:tasks.rados.rados.0.trial178.stdout:2088: finishing copy_from to trial17818531-6 2026-02-21T22:24:22.003 INFO:tasks.rados.rados.0.trial178.stdout:2088: got expected ENOENT (src dne) 2026-02-21T22:24:22.003 INFO:tasks.rados.rados.0.trial178.stdout:2088: finishing copy_from racing read to trial17818531-6 2026-02-21T22:24:22.003 INFO:tasks.rados.rados.0.trial178.stdout:2089: expect (ObjNum 720 snap 200 seq_num 720) 2026-02-21T22:24:22.040 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1768754 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2087: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2087: finishing rollback tid 2 to trial17818531-26 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1133 (ObjNum 564 snap 158 seq_num 564) dirty exists 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: finishing write tid 1 to trial17818531-11 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: finishing write tid 2 to trial17818531-11 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: finishing write tid 3 to trial17818531-11 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: finishing write tid 4 to trial17818531-11 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: oid 11 updating version 0 to 1007 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: oid 11 version 1007 is already newer than 1006 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: oid 11 updating version 1007 to 1008 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1008 (ObjNum 731 snap 208 seq_num 731) dirty exists 2026-02-21T22:24:22.042 INFO:tasks.rados.rados.0.trial178.stdout:2090: left oid 11 (ObjNum 731 snap 208 seq_num 731) 2026-02-21T22:24:22.043 INFO:tasks.rados.rados.0.trial178.stdout:2091: expect (ObjNum 670 snap 189 seq_num 670) 2026-02-21T22:24:22.129 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2093: expect (ObjNum 588 snap 162 seq_num 588) 2026-02-21T22:24:22.173 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:22.315 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1865250 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2094: finishing copy_from racing read to trial17818531-23 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2094: finishing copy_from to trial17818531-23 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 786 (ObjNum 688 snap 190 seq_num 688) dirty exists 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2087: done (7 left) 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2088: done (6 left) 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2089: done (5 left) 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2090: done (4 left) 2026-02-21T22:24:22.342 INFO:tasks.rados.rados.0.trial178.stdout:2091: done (3 left) 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2092: done (2 left) 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2093: done (1 left) 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2094: done (0 left) 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2095: delete oid 3 current snap is 208 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2095: done (0 left) 2026-02-21T22:24:22.343 INFO:tasks.rados.rados.0.trial178.stdout:2096: snap_remove snap 202 2026-02-21T22:24:22.773 INFO:tasks.rados.rados.0.trial178.stdout:2096: done (0 left) 2026-02-21T22:24:22.773 INFO:tasks.rados.rados.0.trial178.stdout:2097: delete oid 15 current snap is 208 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2097: done (0 left) 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2098: copy_from oid 35 from oid 30 current snap is 208 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2099: read oid 50 snap -1 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2099: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2100: read oid 3 snap 200 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2100: expect deleted 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2101: write oid 37 current snap is 208 2026-02-21T22:24:22.780 INFO:tasks.rados.rados.0.trial178.stdout:2101: seq_num 734 ranges {736338=402654,1552136=539580} 2026-02-21T22:24:22.784 INFO:tasks.rados.rados.0.trial178.stdout:2101: writing trial17818531-37 from 736338 to 1138992 tid 1 2026-02-21T22:24:22.790 INFO:tasks.rados.rados.0.trial178.stdout:2101: writing trial17818531-37 from 1552136 to 2091716 tid 2 2026-02-21T22:24:22.791 INFO:tasks.rados.rados.0.trial178.stdout:2102: read oid 17 snap 195 2026-02-21T22:24:22.791 INFO:tasks.rados.rados.0.trial178.stdout:2102: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:24:22.791 INFO:tasks.rados.rados.0.trial178.stdout:2103: delete oid 26 current snap is 208 2026-02-21T22:24:22.791 INFO:tasks.rados.rados.0.trial178.stdout:2099: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:22.915 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1699748 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: finishing write tid 1 to trial17818531-37 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: finishing write tid 2 to trial17818531-37 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2099: done (5 left) 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2100: done (4 left) 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2103: done (3 left) 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2104: copy_from oid 36 from oid 28 current snap is 208 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: finishing write tid 3 to trial17818531-37 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: finishing write tid 4 to trial17818531-37 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: oid 37 updating version 0 to 1133 2026-02-21T22:24:22.959 INFO:tasks.rados.rados.0.trial178.stdout:2101: oid 37 version 1133 is already newer than 1131 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:2101: oid 37 version 1133 is already newer than 1132 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1133 (ObjNum 734 snap 208 seq_num 734) dirty exists 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:2101: left oid 37 (ObjNum 734 snap 208 seq_num 734) 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:2098: finishing copy_from to trial17818531-35 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:2098: finishing copy_from racing read to trial17818531-35 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:2102: expect (ObjNum 648 snap 182 seq_num 648) 2026-02-21T22:24:22.960 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 78314 2026-02-21T22:24:22.980 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:23.002 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2104: finishing copy_from to trial17818531-36 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2104: got expected ENOENT (src dne) 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2098: done (3 left) 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2101: done (2 left) 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2102: done (1 left) 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2105: snap_create 2026-02-21T22:24:23.025 INFO:tasks.rados.rados.0.trial178.stdout:2104: finishing copy_from racing read to trial17818531-36 2026-02-21T22:24:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:24:23] 2026-02-21T22:24:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:24:23.528 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:24:23, mode upmap, max misplaced 0.050000 2026-02-21T22:24:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:24:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:24:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['unique_pool_0', '.mgr', 'rbd'] 2026-02-21T22:24:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:24:23.529 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:24:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:24:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:23.546 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:24:23.547 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:24:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:24:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:24:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:24:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "431", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:24:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 431, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 214, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:24:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0003600932933666088 of space, bias 1.0, pg target 0.10802798800998265 quantized to 32 (current 16) 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:24:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:24:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:24:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:24:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:24:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:24:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:23.779 INFO:tasks.rados.rados.0.trial178.stdout:2104: done (1 left) 2026-02-21T22:24:23.779 INFO:tasks.rados.rados.0.trial178.stdout:2105: done (0 left) 2026-02-21T22:24:23.780 INFO:tasks.rados.rados.0.trial178.stdout:2106: rollback oid 39 current snap is 209 2026-02-21T22:24:23.780 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 195 2026-02-21T22:24:23.780 INFO:tasks.rados.rados.0.trial178.stdout:2107: write oid 23 current snap is 209 2026-02-21T22:24:23.780 INFO:tasks.rados.rados.0.trial178.stdout:2107: seq_num 736 ranges {769952=553152,2069100=300852} 2026-02-21T22:24:23.785 INFO:tasks.rados.rados.0.trial178.stdout:2107: writing trial17818531-23 from 769952 to 1323104 tid 1 2026-02-21T22:24:23.791 INFO:tasks.rados.rados.0.trial178.stdout:2107: writing trial17818531-23 from 2069100 to 2369952 tid 2 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2108: copy_from oid 25 from oid 30 current snap is 209 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2106: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2106: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2106: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1100 (ObjNum 625 snap 178 seq_num 625) dirty exists 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2107: finishing write tid 1 to trial17818531-23 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2106: done (2 left) 2026-02-21T22:24:23.792 INFO:tasks.rados.rados.0.trial178.stdout:2109: delete oid 11 current snap is 209 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2109: done (2 left) 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2110: read oid 16 snap -1 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2110: expect deleted 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2111: read oid 50 snap -1 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2111: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2112: write (excl) oid 36 current snap is 209 2026-02-21T22:24:23.794 INFO:tasks.rados.rados.0.trial178.stdout:2112: seq_num 738 ranges {403566=420003,1386064=546918} 2026-02-21T22:24:23.797 INFO:tasks.rados.rados.0.trial178.stdout:2112: writing trial17818531-36 from 403566 to 823569 tid 1 2026-02-21T22:24:23.801 INFO:tasks.rados.rados.0.trial178.stdout:2112: writing trial17818531-36 from 1386064 to 1932982 tid 2 2026-02-21T22:24:23.801 INFO:tasks.rados.rados.0.trial178.stdout:2113: write oid 33 current snap is 209 2026-02-21T22:24:23.801 INFO:tasks.rados.rados.0.trial178.stdout:2113: seq_num 739 ranges {420373=495252,1444546=690576} 2026-02-21T22:24:23.804 INFO:tasks.rados.rados.0.trial178.stdout:2113: writing trial17818531-33 from 420373 to 915625 tid 1 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2113: writing trial17818531-33 from 1444546 to 2135122 tid 2 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2114: rollback oid 1 current snap is 209 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 205 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: finishing write tid 2 to trial17818531-23 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: finishing write tid 3 to trial17818531-23 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: finishing write tid 4 to trial17818531-23 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: oid 23 updating version 0 to 994 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: oid 23 version 994 is already newer than 992 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: oid 23 version 994 is already newer than 993 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 994 (ObjNum 736 snap 209 seq_num 736) dirty exists 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2107: left oid 23 (ObjNum 736 snap 209 seq_num 736) 2026-02-21T22:24:23.813 INFO:tasks.rados.rados.0.trial178.stdout:2111: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:24.001 INFO:tasks.rados.rados.0.trial178.stdout:2112: finishing write tid 1 to trial17818531-36 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: finishing write tid 2 to trial17818531-36 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: finishing write tid 3 to trial17818531-36 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: finishing write tid 4 to trial17818531-36 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: oid 36 updating version 0 to 389 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: oid 36 version 389 is already newer than 388 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: oid 36 version 389 is already newer than 387 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 389 (ObjNum 738 snap 209 seq_num 738) dirty exists 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2112: left oid 36 (ObjNum 738 snap 209 seq_num 738) 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2108: finishing copy_from racing read to trial17818531-25 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:2108: finishing copy_from to trial17818531-25 2026-02-21T22:24:24.002 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2113: finishing write tid 1 to trial17818531-33 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2113: finishing write tid 2 to trial17818531-33 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2107: done (6 left) 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2108: done (5 left) 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2110: done (4 left) 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2111: done (3 left) 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2112: done (2 left) 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2115: delete oid 14 current snap is 209 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2113: finishing write tid 3 to trial17818531-33 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2114: finishing rollback tid 0 to trial17818531-1 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2113: finishing write tid 4 to trial17818531-33 2026-02-21T22:24:24.003 INFO:tasks.rados.rados.0.trial178.stdout:2113: oid 33 updating version 0 to 441 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:2113: oid 33 updating version 441 to 443 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:2113: oid 33 version 443 is already newer than 442 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 443 (ObjNum 739 snap 209 seq_num 739) dirty exists 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:2113: left oid 33 (ObjNum 739 snap 209 seq_num 739) 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:2114: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:2114: finishing rollback tid 2 to trial17818531-1 2026-02-21T22:24:24.004 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1136 (ObjNum 688 snap 190 seq_num 688) dirty exists 2026-02-21T22:24:24.005 INFO:tasks.rados.rados.0.trial178.stdout:2113: done (2 left) 2026-02-21T22:24:24.005 INFO:tasks.rados.rados.0.trial178.stdout:2114: done (1 left) 2026-02-21T22:24:24.005 INFO:tasks.rados.rados.0.trial178.stdout:2115: done (0 left) 2026-02-21T22:24:24.005 INFO:tasks.rados.rados.0.trial178.stdout:2116: write (excl) oid 41 current snap is 209 2026-02-21T22:24:24.005 INFO:tasks.rados.rados.0.trial178.stdout:2116: seq_num 741 ranges {453987=645750,1845157=608830} 2026-02-21T22:24:24.010 INFO:tasks.rados.rados.0.trial178.stdout:2116: writing trial17818531-41 from 453987 to 1099737 tid 1 2026-02-21T22:24:24.018 INFO:tasks.rados.rados.0.trial178.stdout:2116: writing trial17818531-41 from 1845157 to 2453987 tid 2 2026-02-21T22:24:24.018 INFO:tasks.rados.rados.0.trial178.stdout:2117: copy_from oid 25 from oid 13 current snap is 209 2026-02-21T22:24:24.018 INFO:tasks.rados.rados.0.trial178.stdout:2116: finishing write tid 1 to trial17818531-41 2026-02-21T22:24:24.018 INFO:tasks.rados.rados.0.trial178.stdout:2118: write oid 8 current snap is 209 2026-02-21T22:24:24.018 INFO:tasks.rados.rados.0.trial178.stdout:2118: seq_num 742 ranges {470794=720999,1903639=554256} 2026-02-21T22:24:24.025 INFO:tasks.rados.rados.0.trial178.stdout:2118: writing trial17818531-8 from 470794 to 1191793 tid 1 2026-02-21T22:24:24.033 INFO:tasks.rados.rados.0.trial178.stdout:2118: writing trial17818531-8 from 1903639 to 2457895 tid 2 2026-02-21T22:24:24.033 INFO:tasks.rados.rados.0.trial178.stdout:2119: snap_remove snap 199 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2119: done (3 left) 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2120: rollback oid 48 current snap is 209 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 200 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: finishing write tid 2 to trial17818531-41 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: finishing write tid 3 to trial17818531-41 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: finishing write tid 4 to trial17818531-41 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: oid 41 updating version 0 to 759 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: oid 41 version 759 is already newer than 758 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: oid 41 version 759 is already newer than 757 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 759 (ObjNum 741 snap 209 seq_num 741) dirty exists 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: left oid 41 (ObjNum 741 snap 209 seq_num 741) 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2117: finishing copy_from racing read to trial17818531-25 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2116: done (3 left) 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2121: write (excl) oid 12 current snap is 209 2026-02-21T22:24:24.784 INFO:tasks.rados.rados.0.trial178.stdout:2121: seq_num 743 ranges {487601=796248,1962121=525480} 2026-02-21T22:24:24.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: writing trial17818531-12 from 487601 to 1283849 tid 1 2026-02-21T22:24:24.800 INFO:tasks.rados.rados.0.trial178.stdout:2121: writing trial17818531-12 from 1962121 to 2487601 tid 2 2026-02-21T22:24:24.800 INFO:tasks.rados.rados.0.trial178.stdout:2122: snap_remove snap 205 2026-02-21T22:24:25.789 INFO:tasks.rados.rados.0.trial178.stdout:2122: done (4 left) 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2123: copy_from oid 44 from oid 18 current snap is 209 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2117: finishing copy_from to trial17818531-25 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1001 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: finishing write tid 1 to trial17818531-8 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: finishing write tid 2 to trial17818531-8 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: finishing write tid 3 to trial17818531-8 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: finishing write tid 4 to trial17818531-8 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: oid 8 updating version 0 to 1185 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: oid 8 version 1185 is already newer than 1184 2026-02-21T22:24:25.790 INFO:tasks.rados.rados.0.trial178.stdout:2118: oid 8 version 1185 is already newer than 1183 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1185 (ObjNum 742 snap 209 seq_num 742) dirty exists 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2118: left oid 8 (ObjNum 742 snap 209 seq_num 742) 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2120: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2120: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2120: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1139 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2121: finishing write tid 1 to trial17818531-12 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2121: finishing write tid 2 to trial17818531-12 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2117: done (4 left) 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2118: done (3 left) 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2120: done (2 left) 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2124: delete oid 27 current snap is 209 2026-02-21T22:24:25.791 INFO:tasks.rados.rados.0.trial178.stdout:2121: finishing write tid 3 to trial17818531-12 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: finishing write tid 4 to trial17818531-12 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: oid 12 updating version 0 to 782 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: oid 12 updating version 782 to 783 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: oid 12 version 783 is already newer than 781 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 783 (ObjNum 743 snap 209 seq_num 743) dirty exists 2026-02-21T22:24:25.792 INFO:tasks.rados.rados.0.trial178.stdout:2121: left oid 12 (ObjNum 743 snap 209 seq_num 743) 2026-02-21T22:24:25.793 INFO:tasks.rados.rados.0.trial178.stdout:2121: done (2 left) 2026-02-21T22:24:25.793 INFO:tasks.rados.rados.0.trial178.stdout:2124: done (1 left) 2026-02-21T22:24:25.793 INFO:tasks.rados.rados.0.trial178.stdout:2125: write oid 15 current snap is 209 2026-02-21T22:24:25.793 INFO:tasks.rados.rados.0.trial178.stdout:2125: seq_num 745 ranges {521215=546746,1562732=701583} 2026-02-21T22:24:25.799 INFO:tasks.rados.rados.0.trial178.stdout:2125: writing trial17818531-15 from 521215 to 1067961 tid 1 2026-02-21T22:24:25.807 INFO:tasks.rados.rados.0.trial178.stdout:2125: writing trial17818531-15 from 1562732 to 2264315 tid 2 2026-02-21T22:24:25.807 INFO:tasks.rados.rados.0.trial178.stdout:2126: write oid 14 current snap is 209 2026-02-21T22:24:25.807 INFO:tasks.rados.rados.0.trial178.stdout:2126: seq_num 746 ranges {538022=738348,1737567=561594} 2026-02-21T22:24:25.815 INFO:tasks.rados.rados.0.trial178.stdout:2126: writing trial17818531-14 from 538022 to 1276370 tid 1 2026-02-21T22:24:25.823 INFO:tasks.rados.rados.0.trial178.stdout:2126: writing trial17818531-14 from 1737567 to 2299161 tid 2 2026-02-21T22:24:25.823 INFO:tasks.rados.rados.0.trial178.stdout:2127: copy_from oid 36 from oid 21 current snap is 209 2026-02-21T22:24:25.823 INFO:tasks.rados.rados.0.trial178.stdout:2123: finishing copy_from racing read to trial17818531-44 2026-02-21T22:24:25.823 INFO:tasks.rados.rados.0.trial178.stdout:2123: finishing copy_from to trial17818531-44 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 773 (ObjNum 721 snap 201 seq_num 721) dirty exists 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:2125: finishing write tid 1 to trial17818531-15 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:2125: finishing write tid 2 to trial17818531-15 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:2125: finishing write tid 3 to trial17818531-15 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:2123: done (3 left) 2026-02-21T22:24:25.824 INFO:tasks.rados.rados.0.trial178.stdout:2128: snap_remove snap 195 2026-02-21T22:24:26.760 INFO:tasks.rados.rados.0.trial178.stdout:2128: done (3 left) 2026-02-21T22:24:26.760 INFO:tasks.rados.rados.0.trial178.stdout:2129: snap_remove snap 197 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2129: done (3 left) 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2130: delete oid 32 current snap is 209 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2125: finishing write tid 4 to trial17818531-15 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2125: oid 15 updating version 0 to 764 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2125: oid 15 updating version 764 to 765 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2125: oid 15 version 765 is already newer than 763 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 765 (ObjNum 745 snap 209 seq_num 745) dirty exists 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2125: left oid 15 (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: finishing write tid 1 to trial17818531-14 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: finishing write tid 2 to trial17818531-14 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: finishing write tid 3 to trial17818531-14 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: finishing write tid 4 to trial17818531-14 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: oid 14 updating version 0 to 1104 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: oid 14 version 1104 is already newer than 1102 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: oid 14 version 1104 is already newer than 1103 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1104 (ObjNum 746 snap 209 seq_num 746) dirty exists 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2126: left oid 14 (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2127: finishing copy_from racing read to trial17818531-36 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:2127: finishing copy_from to trial17818531-36 2026-02-21T22:24:27.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1081 (ObjNum 615 snap 173 seq_num 615) dirty exists 2026-02-21T22:24:27.766 INFO:tasks.rados.rados.0.trial178.stdout:2125: done (3 left) 2026-02-21T22:24:27.766 INFO:tasks.rados.rados.0.trial178.stdout:2126: done (2 left) 2026-02-21T22:24:27.766 INFO:tasks.rados.rados.0.trial178.stdout:2127: done (1 left) 2026-02-21T22:24:27.766 INFO:tasks.rados.rados.0.trial178.stdout:2130: done (0 left) 2026-02-21T22:24:27.766 INFO:tasks.rados.rados.0.trial178.stdout:2131: snap_create 2026-02-21T22:24:28.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:28.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:28.762 INFO:tasks.rados.rados.0.trial178.stdout:2131: done (0 left) 2026-02-21T22:24:28.762 INFO:tasks.rados.rados.0.trial178.stdout:2132: snap_create 2026-02-21T22:24:29.763 INFO:tasks.rados.rados.0.trial178.stdout:2132: done (0 left) 2026-02-21T22:24:29.763 INFO:tasks.rados.rados.0.trial178.stdout:2133: snap_create 2026-02-21T22:24:30.764 INFO:tasks.rados.rados.0.trial178.stdout:2133: done (0 left) 2026-02-21T22:24:30.764 INFO:tasks.rados.rados.0.trial178.stdout:2134: write (excl) oid 43 current snap is 212 2026-02-21T22:24:30.764 INFO:tasks.rados.rados.0.trial178.stdout:2134: seq_num 748 ranges {571636=488846,1854531=565263} 2026-02-21T22:24:30.771 INFO:tasks.rados.rados.0.trial178.stdout:2134: writing trial17818531-43 from 571636 to 1060482 tid 1 2026-02-21T22:24:30.780 INFO:tasks.rados.rados.0.trial178.stdout:2134: writing trial17818531-43 from 1854531 to 2419794 tid 2 2026-02-21T22:24:30.780 INFO:tasks.rados.rados.0.trial178.stdout:2135: delete oid 30 current snap is 212 2026-02-21T22:24:30.781 INFO:tasks.rados.rados.0.trial178.stdout:2134: finishing write tid 1 to trial17818531-43 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: finishing write tid 2 to trial17818531-43 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: finishing write tid 3 to trial17818531-43 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: finishing write tid 4 to trial17818531-43 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: oid 43 updating version 0 to 371 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: oid 43 version 371 is already newer than 370 2026-02-21T22:24:30.783 INFO:tasks.rados.rados.0.trial178.stdout:2134: oid 43 updating version 371 to 372 2026-02-21T22:24:30.784 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 372 (ObjNum 748 snap 212 seq_num 748) dirty exists 2026-02-21T22:24:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2134: left oid 43 (ObjNum 748 snap 212 seq_num 748) 2026-02-21T22:24:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2134: done (1 left) 2026-02-21T22:24:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2135: done (0 left) 2026-02-21T22:24:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2136: delete oid 46 current snap is 212 2026-02-21T22:24:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2136: done (0 left) 2026-02-21T22:24:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2137: copy_from oid 49 from oid 15 current snap is 212 2026-02-21T22:24:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2138: snap_remove snap 204 2026-02-21T22:24:31.760 INFO:tasks.rados.rados.0.trial178.stdout:2138: done (1 left) 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:2139: rollback oid 32 current snap is 212 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 208 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:2137: finishing copy_from racing read to trial17818531-49 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:2137: finishing copy_from to trial17818531-49 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 765 (ObjNum 745 snap 209 seq_num 745) dirty exists 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:2137: done (1 left) 2026-02-21T22:24:31.761 INFO:tasks.rados.rados.0.trial178.stdout:2140: snap_remove snap 198 2026-02-21T22:24:32.763 INFO:tasks.rados.rados.0.trial178.stdout:2140: done (1 left) 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2141: read oid 2 snap -1 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2141: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2142: rollback oid 21 current snap is 212 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2139: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2139: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1131 (ObjNum 681 snap 189 seq_num 681) dirty exists 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 208 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2139: done (2 left) 2026-02-21T22:24:32.764 INFO:tasks.rados.rados.0.trial178.stdout:2143: snap_remove snap 211 2026-02-21T22:24:33.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:33.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:33.767 INFO:tasks.rados.rados.0.trial178.stdout:2143: done (2 left) 2026-02-21T22:24:33.767 INFO:tasks.rados.rados.0.trial178.stdout:2144: delete oid 23 current snap is 212 2026-02-21T22:24:33.768 INFO:tasks.rados.rados.0.trial178.stdout:2142: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:24:33.768 INFO:tasks.rados.rados.0.trial178.stdout:2141: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2142: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2141: done (2 left) 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2144: done (1 left) 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2145: read oid 12 snap -1 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2145: expect (ObjNum 743 snap 209 seq_num 743) 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2146: read oid 20 snap -1 2026-02-21T22:24:33.883 INFO:tasks.rados.rados.0.trial178.stdout:2146: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:24:33.884 INFO:tasks.rados.rados.0.trial178.stdout:2147: write oid 45 current snap is 212 2026-02-21T22:24:33.884 INFO:tasks.rados.rados.0.trial178.stdout:2142: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:24:33.884 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1201 (ObjNum 615 snap 173 seq_num 615) dirty exists 2026-02-21T22:24:33.884 INFO:tasks.rados.rados.0.trial178.stdout:2147: seq_num 752 ranges {638864=789842,1972106=456248} 2026-02-21T22:24:33.887 INFO:tasks.rados.rados.0.trial178.stdout:2147: writing trial17818531-45 from 638864 to 1428706 tid 1 2026-02-21T22:24:33.891 INFO:tasks.rados.rados.0.trial178.stdout:2147: writing trial17818531-45 from 1972106 to 2428354 tid 2 2026-02-21T22:24:33.892 INFO:tasks.rados.rados.0.trial178.stdout:2142: done (3 left) 2026-02-21T22:24:33.892 INFO:tasks.rados.rados.0.trial178.stdout:2148: write (excl) oid 31 current snap is 212 2026-02-21T22:24:33.892 INFO:tasks.rados.rados.0.trial178.stdout:2148: seq_num 753 ranges {655671=581444,2030588=625083} 2026-02-21T22:24:33.896 INFO:tasks.rados.rados.0.trial178.stdout:2148: writing trial17818531-31 from 655671 to 1237115 tid 1 2026-02-21T22:24:33.902 INFO:tasks.rados.rados.0.trial178.stdout:2148: writing trial17818531-31 from 2030588 to 2655671 tid 2 2026-02-21T22:24:33.902 INFO:tasks.rados.rados.0.trial178.stdout:2149: snap_create 2026-02-21T22:24:33.903 INFO:tasks.rados.rados.0.trial178.stdout:2145: expect (ObjNum 743 snap 209 seq_num 743) 2026-02-21T22:24:33.933 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:33.965 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:33.990 INFO:tasks.rados.rados.0.trial178.stdout:2146: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:24:34.114 INFO:tasks.rados.rados.0.trial178.stdout:2147: finishing write tid 1 to trial17818531-45 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: finishing write tid 2 to trial17818531-45 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: finishing write tid 3 to trial17818531-45 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: finishing write tid 4 to trial17818531-45 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: oid 45 updating version 0 to 1142 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: oid 45 version 1142 is already newer than 1141 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: oid 45 version 1142 is already newer than 1140 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1142 (ObjNum 752 snap 212 seq_num 752) dirty exists 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2147: left oid 45 (ObjNum 752 snap 212 seq_num 752) 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: finishing write tid 1 to trial17818531-31 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: finishing write tid 2 to trial17818531-31 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: finishing write tid 3 to trial17818531-31 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: finishing write tid 4 to trial17818531-31 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: oid 31 updating version 0 to 1130 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: oid 31 version 1130 is already newer than 1128 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: oid 31 version 1130 is already newer than 1129 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1130 (ObjNum 753 snap 212 seq_num 753) dirty exists 2026-02-21T22:24:34.115 INFO:tasks.rados.rados.0.trial178.stdout:2148: left oid 31 (ObjNum 753 snap 212 seq_num 753) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2145: done (4 left) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2146: done (3 left) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2147: done (2 left) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2148: done (1 left) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2149: done (0 left) 2026-02-21T22:24:34.768 INFO:tasks.rados.rados.0.trial178.stdout:2150: snap_remove snap 208 2026-02-21T22:24:35.769 INFO:tasks.rados.rados.0.trial178.stdout:2150: done (0 left) 2026-02-21T22:24:35.769 INFO:tasks.rados.rados.0.trial178.stdout:2151: snap_remove snap 206 2026-02-21T22:24:36.761 INFO:tasks.rados.rados.0.trial178.stdout:2151: done (0 left) 2026-02-21T22:24:36.761 INFO:tasks.rados.rados.0.trial178.stdout:2152: snap_remove snap 200 2026-02-21T22:24:37.762 INFO:tasks.rados.rados.0.trial178.stdout:2152: done (0 left) 2026-02-21T22:24:37.762 INFO:tasks.rados.rados.0.trial178.stdout:2153: read oid 5 snap -1 2026-02-21T22:24:37.762 INFO:tasks.rados.rados.0.trial178.stdout:2153: expect (ObjNum 729 snap 208 seq_num 729) 2026-02-21T22:24:37.763 INFO:tasks.rados.rados.0.trial178.stdout:2154: snap_remove snap 210 2026-02-21T22:24:38.564 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:38.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:38.763 INFO:tasks.rados.rados.0.trial178.stdout:2154: done (1 left) 2026-02-21T22:24:38.764 INFO:tasks.rados.rados.0.trial178.stdout:2155: rollback oid 12 current snap is 213 2026-02-21T22:24:38.764 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 209 2026-02-21T22:24:38.764 INFO:tasks.rados.rados.0.trial178.stdout:2153: expect (ObjNum 729 snap 208 seq_num 729) 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2155: finishing rollback tid 0 to trial17818531-12 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2155: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2155: finishing rollback tid 2 to trial17818531-12 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 786 (ObjNum 743 snap 209 seq_num 743) dirty exists 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2153: done (1 left) 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2155: done (0 left) 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2156: write (excl) oid 17 current snap is 213 2026-02-21T22:24:38.826 INFO:tasks.rados.rados.0.trial178.stdout:2156: seq_num 754 ranges {672478=656693,2089070=459917} 2026-02-21T22:24:38.830 INFO:tasks.rados.rados.0.trial178.stdout:2156: writing trial17818531-17 from 672478 to 1329171 tid 1 2026-02-21T22:24:38.835 INFO:tasks.rados.rados.0.trial178.stdout:2156: writing trial17818531-17 from 2089070 to 2548987 tid 2 2026-02-21T22:24:38.835 INFO:tasks.rados.rados.0.trial178.stdout:2157: snap_remove snap 209 2026-02-21T22:24:39.768 INFO:tasks.rados.rados.0.trial178.stdout:2157: done (1 left) 2026-02-21T22:24:39.768 INFO:tasks.rados.rados.0.trial178.stdout:2158: snap_create 2026-02-21T22:24:39.768 INFO:tasks.rados.rados.0.trial178.stdout:2156: finishing write tid 1 to trial17818531-17 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: finishing write tid 2 to trial17818531-17 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: finishing write tid 3 to trial17818531-17 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: finishing write tid 4 to trial17818531-17 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: oid 17 updating version 0 to 800 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: oid 17 version 800 is already newer than 799 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: oid 17 updating version 800 to 801 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 801 (ObjNum 754 snap 213 seq_num 754) dirty exists 2026-02-21T22:24:39.769 INFO:tasks.rados.rados.0.trial178.stdout:2156: left oid 17 (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:24:40.771 INFO:tasks.rados.rados.0.trial178.stdout:2156: done (1 left) 2026-02-21T22:24:40.772 INFO:tasks.rados.rados.0.trial178.stdout:2158: done (0 left) 2026-02-21T22:24:40.772 INFO:tasks.rados.rados.0.trial178.stdout:2159: write (excl) oid 27 current snap is 214 2026-02-21T22:24:40.772 INFO:tasks.rados.rados.0.trial178.stdout:2159: seq_num 755 ranges {689285=731942,2147552=541733} 2026-02-21T22:24:40.777 INFO:tasks.rados.rados.0.trial178.stdout:2159: writing trial17818531-27 from 689285 to 1421227 tid 1 2026-02-21T22:24:40.782 INFO:tasks.rados.rados.0.trial178.stdout:2159: writing trial17818531-27 from 2147552 to 2689285 tid 2 2026-02-21T22:24:40.782 INFO:tasks.rados.rados.0.trial178.stdout:2160: write (excl) oid 31 current snap is 214 2026-02-21T22:24:40.782 INFO:tasks.rados.rados.0.trial178.stdout:2160: seq_num 756 ranges {706092=407191,1806034=463586} 2026-02-21T22:24:40.786 INFO:tasks.rados.rados.0.trial178.stdout:2160: writing trial17818531-31 from 706092 to 1113283 tid 1 2026-02-21T22:24:40.791 INFO:tasks.rados.rados.0.trial178.stdout:2160: writing trial17818531-31 from 1806034 to 2269620 tid 2 2026-02-21T22:24:40.791 INFO:tasks.rados.rados.0.trial178.stdout:2161: write (excl) oid 34 current snap is 214 2026-02-21T22:24:40.791 INFO:tasks.rados.rados.0.trial178.stdout:2161: seq_num 757 ranges {722899=482440,1748163=723597} 2026-02-21T22:24:40.795 INFO:tasks.rados.rados.0.trial178.stdout:2161: writing trial17818531-34 from 722899 to 1205339 tid 1 2026-02-21T22:24:40.801 INFO:tasks.rados.rados.0.trial178.stdout:2161: writing trial17818531-34 from 1748163 to 2471760 tid 2 2026-02-21T22:24:40.801 INFO:tasks.rados.rados.0.trial178.stdout:2162: write (excl) oid 8 current snap is 214 2026-02-21T22:24:40.801 INFO:tasks.rados.rados.0.trial178.stdout:2162: seq_num 758 ranges {739706=557689,1806645=467255} 2026-02-21T22:24:40.806 INFO:tasks.rados.rados.0.trial178.stdout:2162: writing trial17818531-8 from 739706 to 1297395 tid 1 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2162: writing trial17818531-8 from 1806645 to 2273900 tid 2 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2163: rollback oid 11 current snap is 214 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 207 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: finishing write tid 1 to trial17818531-27 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: finishing write tid 2 to trial17818531-27 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: finishing write tid 3 to trial17818531-27 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: finishing write tid 4 to trial17818531-27 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: oid 27 updating version 0 to 1107 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: oid 27 version 1107 is already newer than 1105 2026-02-21T22:24:40.811 INFO:tasks.rados.rados.0.trial178.stdout:2159: oid 27 version 1107 is already newer than 1106 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1107 (ObjNum 755 snap 214 seq_num 755) dirty exists 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2159: left oid 27 (ObjNum 755 snap 214 seq_num 755) 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: finishing write tid 1 to trial17818531-31 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: finishing write tid 2 to trial17818531-31 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: finishing write tid 3 to trial17818531-31 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: finishing write tid 4 to trial17818531-31 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: oid 31 updating version 0 to 1133 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: oid 31 version 1133 is already newer than 1132 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: oid 31 version 1133 is already newer than 1131 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1133 (ObjNum 756 snap 214 seq_num 756) dirty exists 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2160: left oid 31 (ObjNum 756 snap 214 seq_num 756) 2026-02-21T22:24:40.812 INFO:tasks.rados.rados.0.trial178.stdout:2161: finishing write tid 1 to trial17818531-34 2026-02-21T22:24:40.813 INFO:tasks.rados.rados.0.trial178.stdout:2159: done (4 left) 2026-02-21T22:24:40.813 INFO:tasks.rados.rados.0.trial178.stdout:2160: done (3 left) 2026-02-21T22:24:40.813 INFO:tasks.rados.rados.0.trial178.stdout:2164: write (excl) oid 3 current snap is 214 2026-02-21T22:24:40.813 INFO:tasks.rados.rados.0.trial178.stdout:2164: seq_num 759 ranges {756513=632938,1865127=727266} 2026-02-21T22:24:40.816 INFO:tasks.rados.rados.0.trial178.stdout:2164: writing trial17818531-3 from 756513 to 1389451 tid 1 2026-02-21T22:24:40.820 INFO:tasks.rados.rados.0.trial178.stdout:2164: writing trial17818531-3 from 1865127 to 2592393 tid 2 2026-02-21T22:24:40.821 INFO:tasks.rados.rados.0.trial178.stdout:2165: snap_remove snap 213 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2165: done (4 left) 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2166: delete oid 4 current snap is 214 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2161: finishing write tid 2 to trial17818531-34 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2161: finishing write tid 3 to trial17818531-34 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2161: finishing write tid 4 to trial17818531-34 2026-02-21T22:24:41.763 INFO:tasks.rados.rados.0.trial178.stdout:2161: oid 34 updating version 0 to 792 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2161: oid 34 version 792 is already newer than 790 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2161: oid 34 version 792 is already newer than 791 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 792 (ObjNum 757 snap 214 seq_num 757) dirty exists 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2161: left oid 34 (ObjNum 757 snap 214 seq_num 757) 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2162: finishing write tid 1 to trial17818531-8 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2163: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2163: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1133 (ObjNum 635 snap 180 seq_num 635) dirty exists 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2162: finishing write tid 2 to trial17818531-8 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2162: finishing write tid 3 to trial17818531-8 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2162: finishing write tid 4 to trial17818531-8 2026-02-21T22:24:41.764 INFO:tasks.rados.rados.0.trial178.stdout:2162: oid 8 updating version 0 to 1225 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2162: oid 8 version 1225 is already newer than 1223 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2162: oid 8 version 1225 is already newer than 1224 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1225 (ObjNum 758 snap 214 seq_num 758) dirty exists 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2162: left oid 8 (ObjNum 758 snap 214 seq_num 758) 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: finishing write tid 1 to trial17818531-3 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: finishing write tid 2 to trial17818531-3 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: finishing write tid 3 to trial17818531-3 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: finishing write tid 4 to trial17818531-3 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: oid 3 updating version 0 to 1135 2026-02-21T22:24:41.765 INFO:tasks.rados.rados.0.trial178.stdout:2164: oid 3 version 1135 is already newer than 1134 2026-02-21T22:24:41.766 INFO:tasks.rados.rados.0.trial178.stdout:2164: oid 3 updating version 1135 to 1136 2026-02-21T22:24:41.766 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1136 (ObjNum 759 snap 214 seq_num 759) dirty exists 2026-02-21T22:24:41.766 INFO:tasks.rados.rados.0.trial178.stdout:2164: left oid 3 (ObjNum 759 snap 214 seq_num 759) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2161: done (4 left) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2162: done (3 left) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2163: done (2 left) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2164: done (1 left) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2166: done (0 left) 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2167: rollback oid 37 current snap is 214 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 212 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2168: write oid 9 current snap is 214 2026-02-21T22:24:41.767 INFO:tasks.rados.rados.0.trial178.stdout:2168: seq_num 761 ranges {790127=499789,1698444=730935} 2026-02-21T22:24:41.771 INFO:tasks.rados.rados.0.trial178.stdout:2168: writing trial17818531-9 from 790127 to 1289916 tid 1 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2168: writing trial17818531-9 from 1698444 to 2429379 tid 2 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2169: read oid 36 snap 207 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2169: expect (ObjNum 722 snap 201 seq_num 722) 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2170: read oid 27 snap -1 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2170: expect (ObjNum 755 snap 214 seq_num 755) 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2171: delete oid 12 current snap is 214 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2167: finishing rollback tid 0 to trial17818531-37 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2167: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:24:41.776 INFO:tasks.rados.rados.0.trial178.stdout:2167: finishing rollback tid 2 to trial17818531-37 2026-02-21T22:24:41.777 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1145 (ObjNum 734 snap 208 seq_num 734) dirty exists 2026-02-21T22:24:41.777 INFO:tasks.rados.rados.0.trial178.stdout:2168: finishing write tid 1 to trial17818531-9 2026-02-21T22:24:41.777 INFO:tasks.rados.rados.0.trial178.stdout:2167: done (4 left) 2026-02-21T22:24:41.777 INFO:tasks.rados.rados.0.trial178.stdout:2171: done (3 left) 2026-02-21T22:24:41.777 INFO:tasks.rados.rados.0.trial178.stdout:2172: delete oid 45 current snap is 214 2026-02-21T22:24:41.778 INFO:tasks.rados.rados.0.trial178.stdout:2172: done (3 left) 2026-02-21T22:24:41.778 INFO:tasks.rados.rados.0.trial178.stdout:2173: delete oid 34 current snap is 214 2026-02-21T22:24:41.779 INFO:tasks.rados.rados.0.trial178.stdout:2169: expect (ObjNum 722 snap 201 seq_num 722) 2026-02-21T22:24:41.780 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:41.898 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:41.949 INFO:tasks.rados.rados.0.trial178.stdout:2168: finishing write tid 2 to trial17818531-9 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: finishing write tid 3 to trial17818531-9 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: finishing write tid 4 to trial17818531-9 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: oid 9 updating version 0 to 1108 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: oid 9 updating version 1108 to 1109 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: oid 9 updating version 1109 to 1110 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1110 (ObjNum 761 snap 214 seq_num 761) dirty exists 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2168: left oid 9 (ObjNum 761 snap 214 seq_num 761) 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:2170: expect (ObjNum 755 snap 214 seq_num 755) 2026-02-21T22:24:41.950 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:41.983 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2168: done (3 left) 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2169: done (2 left) 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2170: done (1 left) 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2173: done (0 left) 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2174: write (excl) oid 19 current snap is 214 2026-02-21T22:24:42.004 INFO:tasks.rados.rados.0.trial178.stdout:2174: seq_num 765 ranges {457355=400785,1416019=621920,2725060=132295} 2026-02-21T22:24:42.009 INFO:tasks.rados.rados.0.trial178.stdout:2174: writing trial17818531-19 from 457355 to 858140 tid 1 2026-02-21T22:24:42.018 INFO:tasks.rados.rados.0.trial178.stdout:2174: writing trial17818531-19 from 1416019 to 2037939 tid 2 2026-02-21T22:24:42.024 INFO:tasks.rados.rados.0.trial178.stdout:2174: writing trial17818531-19 from 2725060 to 2857355 tid 3 2026-02-21T22:24:42.024 INFO:tasks.rados.rados.0.trial178.stdout:2175: snap_create 2026-02-21T22:24:42.024 INFO:tasks.rados.rados.0.trial178.stdout:2174: finishing write tid 1 to trial17818531-19 2026-02-21T22:24:42.024 INFO:tasks.rados.rados.0.trial178.stdout:2174: finishing write tid 2 to trial17818531-19 2026-02-21T22:24:42.025 INFO:tasks.rados.rados.0.trial178.stdout:2174: finishing write tid 3 to trial17818531-19 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: finishing write tid 4 to trial17818531-19 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: finishing write tid 5 to trial17818531-19 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: oid 19 updating version 0 to 1228 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: oid 19 updating version 1228 to 1230 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: oid 19 version 1230 is already newer than 1229 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: oid 19 updating version 1230 to 1231 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1231 (ObjNum 765 snap 214 seq_num 765) dirty exists 2026-02-21T22:24:42.026 INFO:tasks.rados.rados.0.trial178.stdout:2174: left oid 19 (ObjNum 765 snap 214 seq_num 765) 2026-02-21T22:24:42.775 INFO:tasks.rados.rados.0.trial178.stdout:2174: done (1 left) 2026-02-21T22:24:42.775 INFO:tasks.rados.rados.0.trial178.stdout:2175: done (0 left) 2026-02-21T22:24:42.775 INFO:tasks.rados.rados.0.trial178.stdout:2176: write (excl) oid 4 current snap is 215 2026-02-21T22:24:42.775 INFO:tasks.rados.rados.0.trial178.stdout:2176: seq_num 766 ranges {474162=476034,1358148=481931,2352483=465303} 2026-02-21T22:24:42.781 INFO:tasks.rados.rados.0.trial178.stdout:2176: writing trial17818531-4 from 474162 to 950196 tid 1 2026-02-21T22:24:42.787 INFO:tasks.rados.rados.0.trial178.stdout:2176: writing trial17818531-4 from 1358148 to 1840079 tid 2 2026-02-21T22:24:42.794 INFO:tasks.rados.rados.0.trial178.stdout:2176: writing trial17818531-4 from 2352483 to 2817786 tid 3 2026-02-21T22:24:42.794 INFO:tasks.rados.rados.0.trial178.stdout:2177: snap_create 2026-02-21T22:24:42.794 INFO:tasks.rados.rados.0.trial178.stdout:2176: finishing write tid 1 to trial17818531-4 2026-02-21T22:24:42.794 INFO:tasks.rados.rados.0.trial178.stdout:2176: finishing write tid 2 to trial17818531-4 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: finishing write tid 3 to trial17818531-4 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: finishing write tid 4 to trial17818531-4 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: finishing write tid 5 to trial17818531-4 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: oid 4 updating version 0 to 790 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: oid 4 version 790 is already newer than 788 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: oid 4 version 790 is already newer than 789 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: oid 4 updating version 790 to 791 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 791 (ObjNum 766 snap 215 seq_num 766) dirty exists 2026-02-21T22:24:42.796 INFO:tasks.rados.rados.0.trial178.stdout:2176: left oid 4 (ObjNum 766 snap 215 seq_num 766) 2026-02-21T22:24:43.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:43.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:43.781 INFO:tasks.rados.rados.0.trial178.stdout:2176: done (1 left) 2026-02-21T22:24:43.781 INFO:tasks.rados.rados.0.trial178.stdout:2177: done (0 left) 2026-02-21T22:24:43.781 INFO:tasks.rados.rados.0.trial178.stdout:2178: delete oid 38 current snap is 216 2026-02-21T22:24:43.784 INFO:tasks.rados.rados.0.trial178.stdout:2178: done (0 left) 2026-02-21T22:24:43.784 INFO:tasks.rados.rados.0.trial178.stdout:2179: snap_create 2026-02-21T22:24:44.782 INFO:tasks.rados.rados.0.trial178.stdout:2179: done (0 left) 2026-02-21T22:24:44.782 INFO:tasks.rados.rados.0.trial178.stdout:2180: copy_from oid 22 from oid 39 current snap is 217 2026-02-21T22:24:44.782 INFO:tasks.rados.rados.0.trial178.stdout:2181: delete oid 49 current snap is 217 2026-02-21T22:24:44.787 INFO:tasks.rados.rados.0.trial178.stdout:2181: done (1 left) 2026-02-21T22:24:44.787 INFO:tasks.rados.rados.0.trial178.stdout:2182: write oid 41 current snap is 217 2026-02-21T22:24:44.787 INFO:tasks.rados.rados.0.trial178.stdout:2182: seq_num 769 ranges {524583=418134,1649947=629258,2834752=89831} 2026-02-21T22:24:44.793 INFO:tasks.rados.rados.0.trial178.stdout:2182: writing trial17818531-41 from 524583 to 942717 tid 1 2026-02-21T22:24:44.802 INFO:tasks.rados.rados.0.trial178.stdout:2182: writing trial17818531-41 from 1649947 to 2279205 tid 2 2026-02-21T22:24:44.806 INFO:tasks.rados.rados.0.trial178.stdout:2182: writing trial17818531-41 from 2834752 to 2924583 tid 3 2026-02-21T22:24:44.806 INFO:tasks.rados.rados.0.trial178.stdout:2183: write (excl) oid 4 current snap is 217 2026-02-21T22:24:44.806 INFO:tasks.rados.rados.0.trial178.stdout:2183: seq_num 770 ranges {541390=493383,1592076=489269,2862175=79215} 2026-02-21T22:24:44.813 INFO:tasks.rados.rados.0.trial178.stdout:2183: writing trial17818531-4 from 541390 to 1034773 tid 1 2026-02-21T22:24:44.820 INFO:tasks.rados.rados.0.trial178.stdout:2183: writing trial17818531-4 from 1592076 to 2081345 tid 2 2026-02-21T22:24:44.826 INFO:tasks.rados.rados.0.trial178.stdout:2183: writing trial17818531-4 from 2862175 to 2941390 tid 3 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2184: delete oid 23 current snap is 217 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2180: finishing copy_from racing read to trial17818531-22 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2180: finishing copy_from to trial17818531-22 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1100 (ObjNum 625 snap 178 seq_num 625) dirty exists 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: finishing write tid 1 to trial17818531-41 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: finishing write tid 2 to trial17818531-41 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: finishing write tid 3 to trial17818531-41 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: finishing write tid 4 to trial17818531-41 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: finishing write tid 5 to trial17818531-41 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: oid 41 updating version 0 to 807 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: oid 41 updating version 807 to 808 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: oid 41 version 808 is already newer than 805 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:2182: oid 41 version 808 is already newer than 806 2026-02-21T22:24:44.827 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 808 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2182: left oid 41 (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2183: finishing write tid 1 to trial17818531-4 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2183: finishing write tid 2 to trial17818531-4 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2180: done (3 left) 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2182: done (2 left) 2026-02-21T22:24:44.828 INFO:tasks.rados.rados.0.trial178.stdout:2184: done (1 left) 2026-02-21T22:24:44.829 INFO:tasks.rados.rados.0.trial178.stdout:2185: write (excl) oid 33 current snap is 217 2026-02-21T22:24:44.829 INFO:tasks.rados.rados.0.trial178.stdout:2185: seq_num 772 ranges {575004=643881,1709040=492938,2917021=57983} 2026-02-21T22:24:44.836 INFO:tasks.rados.rados.0.trial178.stdout:2185: writing trial17818531-33 from 575004 to 1218885 tid 1 2026-02-21T22:24:44.843 INFO:tasks.rados.rados.0.trial178.stdout:2185: writing trial17818531-33 from 1709040 to 2201978 tid 2 2026-02-21T22:24:44.848 INFO:tasks.rados.rados.0.trial178.stdout:2185: writing trial17818531-33 from 2917021 to 2975004 tid 3 2026-02-21T22:24:44.848 INFO:tasks.rados.rados.0.trial178.stdout:2186: snap_remove snap 216 2026-02-21T22:24:45.787 INFO:tasks.rados.rados.0.trial178.stdout:2186: done (2 left) 2026-02-21T22:24:45.787 INFO:tasks.rados.rados.0.trial178.stdout:2187: read oid 21 snap -1 2026-02-21T22:24:45.787 INFO:tasks.rados.rados.0.trial178.stdout:2187: expect (ObjNum 615 snap 173 seq_num 615) 2026-02-21T22:24:45.787 INFO:tasks.rados.rados.0.trial178.stdout:2188: copy_from oid 25 from oid 17 current snap is 217 2026-02-21T22:24:45.787 INFO:tasks.rados.rados.0.trial178.stdout:2183: finishing write tid 3 to trial17818531-4 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: finishing write tid 4 to trial17818531-4 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: finishing write tid 5 to trial17818531-4 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: oid 4 updating version 0 to 793 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: oid 4 updating version 793 to 795 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: oid 4 version 795 is already newer than 794 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: oid 4 version 795 is already newer than 792 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 795 (ObjNum 770 snap 217 seq_num 770) dirty exists 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: left oid 4 (ObjNum 770 snap 217 seq_num 770) 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: finishing write tid 1 to trial17818531-33 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2183: done (3 left) 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2189: delete oid 29 current snap is 217 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: finishing write tid 2 to trial17818531-33 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: finishing write tid 3 to trial17818531-33 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: finishing write tid 4 to trial17818531-33 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: finishing write tid 5 to trial17818531-33 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: oid 33 updating version 0 to 447 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: oid 33 updating version 447 to 448 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: oid 33 updating version 448 to 449 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: oid 33 updating version 449 to 450 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 450 (ObjNum 772 snap 217 seq_num 772) dirty exists 2026-02-21T22:24:45.788 INFO:tasks.rados.rados.0.trial178.stdout:2185: left oid 33 (ObjNum 772 snap 217 seq_num 772) 2026-02-21T22:24:45.794 INFO:tasks.rados.rados.0.trial178.stdout:2185: done (3 left) 2026-02-21T22:24:45.794 INFO:tasks.rados.rados.0.trial178.stdout:2189: done (2 left) 2026-02-21T22:24:45.794 INFO:tasks.rados.rados.0.trial178.stdout:2190: snap_create 2026-02-21T22:24:45.796 INFO:tasks.rados.rados.0.trial178.stdout:2187: expect (ObjNum 615 snap 173 seq_num 615) 2026-02-21T22:24:45.796 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:45.846 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:45.906 INFO:tasks.rados.rados.0.trial178.stdout:2188: finishing copy_from racing read to trial17818531-25 2026-02-21T22:24:45.906 INFO:tasks.rados.rados.0.trial178.stdout:2188: finishing copy_from to trial17818531-25 2026-02-21T22:24:45.906 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 801 (ObjNum 754 snap 213 seq_num 754) dirty exists 2026-02-21T22:24:46.763 INFO:tasks.rados.rados.0.trial178.stdout:2187: done (2 left) 2026-02-21T22:24:46.763 INFO:tasks.rados.rados.0.trial178.stdout:2188: done (1 left) 2026-02-21T22:24:46.763 INFO:tasks.rados.rados.0.trial178.stdout:2190: done (0 left) 2026-02-21T22:24:46.763 INFO:tasks.rados.rados.0.trial178.stdout:2191: delete oid 48 current snap is 218 2026-02-21T22:24:46.767 INFO:tasks.rados.rados.0.trial178.stdout:2191: done (0 left) 2026-02-21T22:24:46.767 INFO:tasks.rados.rados.0.trial178.stdout:2192: read oid 2 snap -1 2026-02-21T22:24:46.767 INFO:tasks.rados.rados.0.trial178.stdout:2192: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:46.767 INFO:tasks.rados.rados.0.trial178.stdout:2193: snap_remove snap 215 2026-02-21T22:24:47.766 INFO:tasks.rados.rados.0.trial178.stdout:2193: done (1 left) 2026-02-21T22:24:47.766 INFO:tasks.rados.rados.0.trial178.stdout:2194: read oid 26 snap -1 2026-02-21T22:24:47.766 INFO:tasks.rados.rados.0.trial178.stdout:2194: expect deleted 2026-02-21T22:24:47.766 INFO:tasks.rados.rados.0.trial178.stdout:2195: snap_remove snap 207 2026-02-21T22:24:48.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:48.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:48.768 INFO:tasks.rados.rados.0.trial178.stdout:2195: done (2 left) 2026-02-21T22:24:48.768 INFO:tasks.rados.rados.0.trial178.stdout:2196: delete oid 37 current snap is 218 2026-02-21T22:24:48.768 INFO:tasks.rados.rados.0.trial178.stdout:2192: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:48.844 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2192: done (2 left) 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2194: done (1 left) 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2196: done (0 left) 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2197: copy_from oid 40 from oid 48 current snap is 218 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2198: write oid 27 current snap is 218 2026-02-21T22:24:48.874 INFO:tasks.rados.rados.0.trial178.stdout:2198: seq_num 776 ranges {642232=661230,1942968=783923} 2026-02-21T22:24:48.883 INFO:tasks.rados.rados.0.trial178.stdout:2198: writing trial17818531-27 from 642232 to 1303462 tid 1 2026-02-21T22:24:48.893 INFO:tasks.rados.rados.0.trial178.stdout:2198: writing trial17818531-27 from 1942968 to 2726891 tid 2 2026-02-21T22:24:48.893 INFO:tasks.rados.rados.0.trial178.stdout:2199: read oid 50 snap 212 2026-02-21T22:24:48.893 INFO:tasks.rados.rados.0.trial178.stdout:2199: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:48.893 INFO:tasks.rados.rados.0.trial178.stdout:2200: snap_remove snap 214 2026-02-21T22:24:49.773 INFO:tasks.rados.rados.0.trial178.stdout:2197: finishing copy_from to trial17818531-40 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2197: got expected ENOENT (src dne) 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2197: finishing copy_from racing read to trial17818531-40 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: finishing write tid 1 to trial17818531-27 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2197: done (3 left) 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2200: done (2 left) 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2201: rollback oid 17 current snap is 218 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 212 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: finishing write tid 2 to trial17818531-27 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: finishing write tid 3 to trial17818531-27 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: finishing write tid 4 to trial17818531-27 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: oid 27 updating version 0 to 1111 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: oid 27 updating version 1111 to 1113 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: oid 27 version 1113 is already newer than 1112 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1113 (ObjNum 776 snap 218 seq_num 776) dirty exists 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2198: left oid 27 (ObjNum 776 snap 218 seq_num 776) 2026-02-21T22:24:49.774 INFO:tasks.rados.rados.0.trial178.stdout:2199: expect (ObjNum 707 snap 194 seq_num 707) 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2201: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2198: done (2 left) 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2199: done (1 left) 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2202: read oid 45 snap -1 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2202: expect deleted 2026-02-21T22:24:49.970 INFO:tasks.rados.rados.0.trial178.stdout:2203: rollback oid 27 current snap is 218 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 212 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2201: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2201: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 822 (ObjNum 648 snap 182 seq_num 648) dirty exists 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2201: done (2 left) 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2204: read oid 1 snap -1 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2204: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2205: rollback oid 5 current snap is 218 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 212 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2206: delete oid 34 current snap is 218 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2203: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2202: done (4 left) 2026-02-21T22:24:49.971 INFO:tasks.rados.rados.0.trial178.stdout:2206: done (3 left) 2026-02-21T22:24:49.972 INFO:tasks.rados.rados.0.trial178.stdout:2207: read oid 42 snap 212 2026-02-21T22:24:49.972 INFO:tasks.rados.rados.0.trial178.stdout:2207: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:24:49.972 INFO:tasks.rados.rados.0.trial178.stdout:2208: snap_remove snap 217 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2208: done (4 left) 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2209: read oid 26 snap 212 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2209: expect deleted 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2210: read oid 21 snap -1 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2210: expect (ObjNum 615 snap 173 seq_num 615) 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2211: write oid 4 current snap is 218 2026-02-21T22:24:50.777 INFO:tasks.rados.rados.0.trial178.stdout:2203: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1115 (ObjNum 4113 snap 0 seq_num 4294967295) dirty dne 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:2205: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:2205: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:2205: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1134 (ObjNum 729 snap 208 seq_num 729) dirty exists 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:2204: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:24:50.778 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:50.917 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:50.978 INFO:tasks.rados.rados.0.trial178.stdout:2207: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:24:51.066 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 406928 2026-02-21T22:24:51.112 INFO:tasks.rados.rados.0.trial178.stdout:2210: expect (ObjNum 615 snap 173 seq_num 615) 2026-02-21T22:24:51.142 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:51.199 INFO:tasks.rados.rados.0.trial178.stdout:2211: seq_num 778 ranges {675846=411728,1543579=787592,2848853=226993} 2026-02-21T22:24:51.203 INFO:tasks.rados.rados.0.trial178.stdout:2211: writing trial17818531-4 from 675846 to 1087574 tid 1 2026-02-21T22:24:51.208 INFO:tasks.rados.rados.0.trial178.stdout:2211: writing trial17818531-4 from 1543579 to 2331171 tid 2 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2211: writing trial17818531-4 from 2848853 to 3075846 tid 3 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2203: done (6 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2204: done (5 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2205: done (4 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2207: done (3 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2209: done (2 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2210: done (1 left) 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2212: write oid 28 current snap is 218 2026-02-21T22:24:51.213 INFO:tasks.rados.rados.0.trial178.stdout:2212: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2026-02-21T22:24:51.217 INFO:tasks.rados.rados.0.trial178.stdout:2212: writing trial17818531-28 from 692653 to 1179630 tid 1 2026-02-21T22:24:51.221 INFO:tasks.rados.rados.0.trial178.stdout:2212: writing trial17818531-28 from 1602061 to 2249664 tid 2 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2212: writing trial17818531-28 from 2992629 to 3092653 tid 3 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2213: copy_from oid 29 from oid 50 current snap is 218 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: finishing write tid 1 to trial17818531-4 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: finishing write tid 2 to trial17818531-4 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: finishing write tid 3 to trial17818531-4 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: finishing write tid 4 to trial17818531-4 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: finishing write tid 5 to trial17818531-4 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: oid 4 updating version 0 to 799 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: oid 4 version 799 is already newer than 797 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: oid 4 version 799 is already newer than 798 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: oid 4 version 799 is already newer than 796 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 799 (ObjNum 778 snap 218 seq_num 778) dirty exists 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: left oid 4 (ObjNum 778 snap 218 seq_num 778) 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2212: finishing write tid 1 to trial17818531-28 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2212: finishing write tid 2 to trial17818531-28 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2211: done (2 left) 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2214: copy_from oid 36 from oid 25 current snap is 218 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2215: write (excl) oid 45 current snap is 218 2026-02-21T22:24:51.225 INFO:tasks.rados.rados.0.trial178.stdout:2215: seq_num 780 ranges {709460=562226,2060543=791261} 2026-02-21T22:24:51.233 INFO:tasks.rados.rados.0.trial178.stdout:2215: writing trial17818531-45 from 709460 to 1271686 tid 1 2026-02-21T22:24:51.239 INFO:tasks.rados.rados.0.trial178.stdout:2215: writing trial17818531-45 from 2060543 to 2851804 tid 2 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2216: rollback oid 42 current snap is 218 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 212 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: finishing write tid 3 to trial17818531-28 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: finishing write tid 4 to trial17818531-28 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: finishing write tid 5 to trial17818531-28 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: oid 28 updating version 0 to 464 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: oid 28 updating version 464 to 465 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: oid 28 version 465 is already newer than 463 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: oid 28 updating version 465 to 466 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 466 (ObjNum 779 snap 218 seq_num 779) dirty exists 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: left oid 28 (ObjNum 779 snap 218 seq_num 779) 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2213: finishing copy_from racing read to trial17818531-29 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2213: finishing copy_from to trial17818531-29 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 767 (ObjNum 707 snap 194 seq_num 707) dirty exists 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2215: finishing write tid 1 to trial17818531-45 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2214: finishing copy_from racing read to trial17818531-36 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2212: done (4 left) 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2213: done (3 left) 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2217: copy_from oid 43 from oid 10 current snap is 218 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2214: finishing copy_from to trial17818531-36 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 801 (ObjNum 754 snap 213 seq_num 754) dirty exists 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2214: done (3 left) 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2218: read oid 5 snap 212 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2218: expect (ObjNum 729 snap 208 seq_num 729) 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2219: copy_from oid 37 from oid 8 current snap is 218 2026-02-21T22:24:51.240 INFO:tasks.rados.rados.0.trial178.stdout:2220: snap_create 2026-02-21T22:24:51.241 INFO:tasks.rados.rados.0.trial178.stdout:2218: expect (ObjNum 729 snap 208 seq_num 729) 2026-02-21T22:24:51.291 INFO:tasks.rados.rados.0.trial178.stdout:2215: finishing write tid 2 to trial17818531-45 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: finishing write tid 3 to trial17818531-45 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: finishing write tid 4 to trial17818531-45 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: oid 45 updating version 0 to 1150 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: oid 45 updating version 1150 to 1151 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: oid 45 version 1151 is already newer than 1149 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1151 (ObjNum 780 snap 218 seq_num 780) dirty exists 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2215: left oid 45 (ObjNum 780 snap 218 seq_num 780) 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2216: finishing rollback tid 0 to trial17818531-42 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2216: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:2216: finishing rollback tid 2 to trial17818531-42 2026-02-21T22:24:51.292 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1239 (ObjNum 697 snap 192 seq_num 697) dirty exists 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:2217: finishing copy_from racing read to trial17818531-43 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:2217: finishing copy_from to trial17818531-43 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1130 (ObjNum 720 snap 200 seq_num 720) dirty exists 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:2219: finishing copy_from racing read to trial17818531-37 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:2219: finishing copy_from to trial17818531-37 2026-02-21T22:24:51.293 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1225 (ObjNum 758 snap 214 seq_num 758) dirty exists 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2215: done (5 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2216: done (4 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2217: done (3 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2218: done (2 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2219: done (1 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2220: done (0 left) 2026-02-21T22:24:51.781 INFO:tasks.rados.rados.0.trial178.stdout:2221: write (excl) oid 32 current snap is 219 2026-02-21T22:24:51.782 INFO:tasks.rados.rados.0.trial178.stdout:2221: seq_num 781 ranges {726267=637475,2119025=651272} 2026-02-21T22:24:51.786 INFO:tasks.rados.rados.0.trial178.stdout:2221: writing trial17818531-32 from 726267 to 1363742 tid 1 2026-02-21T22:24:51.792 INFO:tasks.rados.rados.0.trial178.stdout:2221: writing trial17818531-32 from 2119025 to 2770297 tid 2 2026-02-21T22:24:51.792 INFO:tasks.rados.rados.0.trial178.stdout:2222: write oid 21 current snap is 219 2026-02-21T22:24:51.792 INFO:tasks.rados.rados.0.trial178.stdout:2221: finishing write tid 1 to trial17818531-32 2026-02-21T22:24:51.792 INFO:tasks.rados.rados.0.trial178.stdout:2222: seq_num 782 ranges {743074=712724,2061154=794930} 2026-02-21T22:24:51.797 INFO:tasks.rados.rados.0.trial178.stdout:2222: writing trial17818531-21 from 743074 to 1455798 tid 1 2026-02-21T22:24:51.803 INFO:tasks.rados.rados.0.trial178.stdout:2222: writing trial17818531-21 from 2061154 to 2856084 tid 2 2026-02-21T22:24:51.803 INFO:tasks.rados.rados.0.trial178.stdout:2223: snap_remove snap 218 2026-02-21T22:24:52.784 INFO:tasks.rados.rados.0.trial178.stdout:2223: done (2 left) 2026-02-21T22:24:52.784 INFO:tasks.rados.rados.0.trial178.stdout:2224: read oid 9 snap -1 2026-02-21T22:24:52.784 INFO:tasks.rados.rados.0.trial178.stdout:2224: expect (ObjNum 761 snap 214 seq_num 761) 2026-02-21T22:24:52.784 INFO:tasks.rados.rados.0.trial178.stdout:2225: write oid 37 current snap is 219 2026-02-21T22:24:52.784 INFO:tasks.rados.rados.0.trial178.stdout:2225: seq_num 783 ranges {759881=787973,2119636=654941} 2026-02-21T22:24:52.794 INFO:tasks.rados.rados.0.trial178.stdout:2225: writing trial17818531-37 from 759881 to 1547854 tid 1 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2225: writing trial17818531-37 from 2119636 to 2774577 tid 2 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2226: read oid 40 snap -1 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2226: expect deleted 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2227: write (excl) oid 31 current snap is 219 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: finishing write tid 2 to trial17818531-32 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: finishing write tid 3 to trial17818531-32 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: finishing write tid 4 to trial17818531-32 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: oid 32 updating version 0 to 1139 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: oid 32 version 1139 is already newer than 1138 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: oid 32 version 1139 is already newer than 1137 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1139 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2221: left oid 32 (ObjNum 781 snap 219 seq_num 781) 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2222: finishing write tid 1 to trial17818531-21 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2222: finishing write tid 2 to trial17818531-21 2026-02-21T22:24:52.803 INFO:tasks.rados.rados.0.trial178.stdout:2227: seq_num 784 ranges {776688=579575,1894471=798599} 2026-02-21T22:24:52.812 INFO:tasks.rados.rados.0.trial178.stdout:2227: writing trial17818531-31 from 776688 to 1356263 tid 1 2026-02-21T22:24:52.821 INFO:tasks.rados.rados.0.trial178.stdout:2227: writing trial17818531-31 from 1894471 to 2693070 tid 2 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2221: done (5 left) 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2228: read oid 4 snap -1 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2228: expect (ObjNum 778 snap 218 seq_num 778) 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2229: delete oid 14 current snap is 219 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2222: finishing write tid 3 to trial17818531-21 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2222: finishing write tid 4 to trial17818531-21 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2222: oid 21 updating version 0 to 1242 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2222: oid 21 version 1242 is already newer than 1241 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:2222: oid 21 updating version 1242 to 1243 2026-02-21T22:24:52.822 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1243 (ObjNum 782 snap 219 seq_num 782) dirty exists 2026-02-21T22:24:52.823 INFO:tasks.rados.rados.0.trial178.stdout:2222: left oid 21 (ObjNum 782 snap 219 seq_num 782) 2026-02-21T22:24:52.823 INFO:tasks.rados.rados.0.trial178.stdout:2224: expect (ObjNum 761 snap 214 seq_num 761) 2026-02-21T22:24:52.934 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: finishing write tid 1 to trial17818531-37 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: finishing write tid 2 to trial17818531-37 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: finishing write tid 3 to trial17818531-37 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: finishing write tid 4 to trial17818531-37 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: oid 37 updating version 0 to 1226 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: oid 37 updating version 1226 to 1227 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: oid 37 updating version 1227 to 1228 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1228 (ObjNum 783 snap 219 seq_num 783) dirty exists 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2225: left oid 37 (ObjNum 783 snap 219 seq_num 783) 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2227: finishing write tid 1 to trial17818531-31 2026-02-21T22:24:52.982 INFO:tasks.rados.rados.0.trial178.stdout:2227: finishing write tid 2 to trial17818531-31 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2227: finishing write tid 3 to trial17818531-31 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2222: done (6 left) 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2224: done (5 left) 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2225: done (4 left) 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2226: done (3 left) 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2229: done (2 left) 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2230: write (excl) oid 46 current snap is 219 2026-02-21T22:24:52.983 INFO:tasks.rados.rados.0.trial178.stdout:2230: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2026-02-21T22:24:52.989 INFO:tasks.rados.rados.0.trial178.stdout:2230: writing trial17818531-46 from 410302 to 1140375 tid 1 2026-02-21T22:24:52.996 INFO:tasks.rados.rados.0.trial178.stdout:2230: writing trial17818531-46 from 1895082 to 2297350 tid 2 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2230: writing trial17818531-46 from 2951884 to 3210302 tid 3 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2231: copy_from oid 11 from oid 41 current snap is 219 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2227: finishing write tid 4 to trial17818531-31 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2227: oid 31 updating version 0 to 1136 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2227: oid 31 version 1136 is already newer than 1135 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:2227: oid 31 updating version 1136 to 1137 2026-02-21T22:24:53.003 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1137 (ObjNum 784 snap 219 seq_num 784) dirty exists 2026-02-21T22:24:53.004 INFO:tasks.rados.rados.0.trial178.stdout:2227: left oid 31 (ObjNum 784 snap 219 seq_num 784) 2026-02-21T22:24:53.004 INFO:tasks.rados.rados.0.trial178.stdout:2228: expect (ObjNum 778 snap 218 seq_num 778) 2026-02-21T22:24:53.004 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 253052 2026-02-21T22:24:53.074 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 970436 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: finishing write tid 1 to trial17818531-46 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: finishing write tid 2 to trial17818531-46 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: finishing write tid 3 to trial17818531-46 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: finishing write tid 4 to trial17818531-46 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: finishing write tid 5 to trial17818531-46 2026-02-21T22:24:53.096 INFO:tasks.rados.rados.0.trial178.stdout:2230: oid 46 updating version 0 to 997 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2230: oid 46 updating version 997 to 998 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2230: oid 46 updating version 998 to 999 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2230: oid 46 updating version 999 to 1000 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1000 (ObjNum 786 snap 219 seq_num 786) dirty exists 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2230: left oid 46 (ObjNum 786 snap 219 seq_num 786) 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2231: finishing copy_from racing read to trial17818531-11 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2231: finishing copy_from to trial17818531-11 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 808 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2227: done (3 left) 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2228: done (2 left) 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2230: done (1 left) 2026-02-21T22:24:53.097 INFO:tasks.rados.rados.0.trial178.stdout:2231: done (0 left) 2026-02-21T22:24:53.098 INFO:tasks.rados.rados.0.trial178.stdout:2232: write oid 35 current snap is 219 2026-02-21T22:24:53.098 INFO:tasks.rados.rados.0.trial178.stdout:2232: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2026-02-21T22:24:53.101 INFO:tasks.rados.rados.0.trial178.stdout:2232: writing trial17818531-35 from 427109 to 832431 tid 1 2026-02-21T22:24:53.109 INFO:tasks.rados.rados.0.trial178.stdout:2232: writing trial17818531-35 from 1553564 to 2099490 tid 2 2026-02-21T22:24:53.116 INFO:tasks.rados.rados.0.trial178.stdout:2232: writing trial17818531-35 from 2579307 to 3227109 tid 3 2026-02-21T22:24:53.117 INFO:tasks.rados.rados.0.trial178.stdout:2233: delete oid 16 current snap is 219 2026-02-21T22:24:53.117 INFO:tasks.rados.rados.0.trial178.stdout:2232: finishing write tid 1 to trial17818531-35 2026-02-21T22:24:53.117 INFO:tasks.rados.rados.0.trial178.stdout:2232: finishing write tid 2 to trial17818531-35 2026-02-21T22:24:53.118 INFO:tasks.rados.rados.0.trial178.stdout:2233: done (1 left) 2026-02-21T22:24:53.118 INFO:tasks.rados.rados.0.trial178.stdout:2234: rollback oid 40 current snap is 219 2026-02-21T22:24:53.118 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 212 2026-02-21T22:24:53.118 INFO:tasks.rados.rados.0.trial178.stdout:2235: delete oid 48 current snap is 219 2026-02-21T22:24:53.119 INFO:tasks.rados.rados.0.trial178.stdout:2235: done (2 left) 2026-02-21T22:24:53.119 INFO:tasks.rados.rados.0.trial178.stdout:2236: snap_create 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: finishing write tid 3 to trial17818531-35 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2234: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 0 (ObjNum 33 snap 0 seq_num 1397807077) dirty dne 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: finishing write tid 4 to trial17818531-35 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: finishing write tid 5 to trial17818531-35 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: oid 35 updating version 0 to 1141 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: oid 35 updating version 1141 to 1142 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: oid 35 updating version 1142 to 1143 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: oid 35 version 1143 is already newer than 1140 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1143 (ObjNum 787 snap 219 seq_num 787) dirty exists 2026-02-21T22:24:53.120 INFO:tasks.rados.rados.0.trial178.stdout:2232: left oid 35 (ObjNum 787 snap 219 seq_num 787) 2026-02-21T22:24:53.535 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:53.547 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:53.547 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:24:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:24:53.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:53.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:53.789 INFO:tasks.rados.rados.0.trial178.stdout:2232: done (2 left) 2026-02-21T22:24:53.789 INFO:tasks.rados.rados.0.trial178.stdout:2234: done (1 left) 2026-02-21T22:24:53.789 INFO:tasks.rados.rados.0.trial178.stdout:2236: done (0 left) 2026-02-21T22:24:53.789 INFO:tasks.rados.rados.0.trial178.stdout:2237: write oid 28 current snap is 220 2026-02-21T22:24:53.789 INFO:tasks.rados.rados.0.trial178.stdout:2237: seq_num 790 ranges {477530=631069,1612657=409606,2545223=517596} 2026-02-21T22:24:53.796 INFO:tasks.rados.rados.0.trial178.stdout:2237: writing trial17818531-28 from 477530 to 1108599 tid 1 2026-02-21T22:24:53.802 INFO:tasks.rados.rados.0.trial178.stdout:2237: writing trial17818531-28 from 1612657 to 2022263 tid 2 2026-02-21T22:24:53.809 INFO:tasks.rados.rados.0.trial178.stdout:2237: writing trial17818531-28 from 2545223 to 3062819 tid 3 2026-02-21T22:24:53.809 INFO:tasks.rados.rados.0.trial178.stdout:2238: read oid 12 snap -1 2026-02-21T22:24:53.810 INFO:tasks.rados.rados.0.trial178.stdout:2238: expect deleted 2026-02-21T22:24:53.810 INFO:tasks.rados.rados.0.trial178.stdout:2239: write oid 5 current snap is 220 2026-02-21T22:24:53.810 INFO:tasks.rados.rados.0.trial178.stdout:2239: seq_num 791 ranges {494337=422671,1387492=553264,2688999=445221} 2026-02-21T22:24:53.815 INFO:tasks.rados.rados.0.trial178.stdout:2239: writing trial17818531-5 from 494337 to 917008 tid 1 2026-02-21T22:24:53.822 INFO:tasks.rados.rados.0.trial178.stdout:2239: writing trial17818531-5 from 1387492 to 1940756 tid 2 2026-02-21T22:24:53.831 INFO:tasks.rados.rados.0.trial178.stdout:2239: writing trial17818531-5 from 2688999 to 3134220 tid 3 2026-02-21T22:24:53.831 INFO:tasks.rados.rados.0.trial178.stdout:2240: copy_from oid 34 from oid 16 current snap is 220 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: finishing write tid 1 to trial17818531-28 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: finishing write tid 2 to trial17818531-28 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: finishing write tid 3 to trial17818531-28 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: finishing write tid 4 to trial17818531-28 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: finishing write tid 5 to trial17818531-28 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: oid 28 updating version 0 to 470 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: oid 28 version 470 is already newer than 469 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: oid 28 version 470 is already newer than 468 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: oid 28 updating version 470 to 471 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 471 (ObjNum 790 snap 220 seq_num 790) dirty exists 2026-02-21T22:24:53.832 INFO:tasks.rados.rados.0.trial178.stdout:2237: left oid 28 (ObjNum 790 snap 220 seq_num 790) 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2239: finishing write tid 1 to trial17818531-5 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2239: finishing write tid 2 to trial17818531-5 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2237: done (3 left) 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2238: done (2 left) 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2241: rollback oid 23 current snap is 220 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 212 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2242: read oid 45 snap -1 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2242: expect (ObjNum 780 snap 218 seq_num 780) 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2243: snap_create 2026-02-21T22:24:53.833 INFO:tasks.rados.rados.0.trial178.stdout:2241: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:24:53.834 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 0 (ObjNum 65 snap 0 seq_num 1634300532) dirty dne 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: finishing write tid 3 to trial17818531-5 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: finishing write tid 4 to trial17818531-5 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: finishing write tid 5 to trial17818531-5 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: oid 5 updating version 0 to 1138 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: oid 5 version 1138 is already newer than 1136 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: oid 5 version 1138 is already newer than 1137 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: oid 5 version 1138 is already newer than 1135 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1138 (ObjNum 791 snap 220 seq_num 791) dirty exists 2026-02-21T22:24:53.835 INFO:tasks.rados.rados.0.trial178.stdout:2239: left oid 5 (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:24:53.836 INFO:tasks.rados.rados.0.trial178.stdout:2240: finishing copy_from to trial17818531-34 2026-02-21T22:24:53.836 INFO:tasks.rados.rados.0.trial178.stdout:2240: got expected ENOENT (src dne) 2026-02-21T22:24:53.836 INFO:tasks.rados.rados.0.trial178.stdout:2240: finishing copy_from racing read to trial17818531-34 2026-02-21T22:24:53.836 INFO:tasks.rados.rados.0.trial178.stdout:2242: expect (ObjNum 780 snap 218 seq_num 780) 2026-02-21T22:24:54.789 INFO:tasks.rados.rados.0.trial178.stdout:2239: done (4 left) 2026-02-21T22:24:54.789 INFO:tasks.rados.rados.0.trial178.stdout:2240: done (3 left) 2026-02-21T22:24:54.789 INFO:tasks.rados.rados.0.trial178.stdout:2241: done (2 left) 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2242: done (1 left) 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2243: done (0 left) 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2244: copy_from oid 37 from oid 38 current snap is 221 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2245: copy_from oid 17 from oid 36 current snap is 221 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2246: read oid 27 snap -1 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2246: expect deleted 2026-02-21T22:24:54.790 INFO:tasks.rados.rados.0.trial178.stdout:2247: delete oid 44 current snap is 221 2026-02-21T22:24:54.792 INFO:tasks.rados.rados.0.trial178.stdout:2244: finishing copy_from to trial17818531-37 2026-02-21T22:24:54.792 INFO:tasks.rados.rados.0.trial178.stdout:2244: got expected ENOENT (src dne) 2026-02-21T22:24:54.792 INFO:tasks.rados.rados.0.trial178.stdout:2244: finishing copy_from racing read to trial17818531-37 2026-02-21T22:24:54.793 INFO:tasks.rados.rados.0.trial178.stdout:2244: done (3 left) 2026-02-21T22:24:54.793 INFO:tasks.rados.rados.0.trial178.stdout:2246: done (2 left) 2026-02-21T22:24:54.794 INFO:tasks.rados.rados.0.trial178.stdout:2247: done (1 left) 2026-02-21T22:24:54.794 INFO:tasks.rados.rados.0.trial178.stdout:2248: snap_create 2026-02-21T22:24:54.806 INFO:tasks.rados.rados.0.trial178.stdout:2245: finishing copy_from racing read to trial17818531-17 2026-02-21T22:24:54.807 INFO:tasks.rados.rados.0.trial178.stdout:2245: finishing copy_from to trial17818531-17 2026-02-21T22:24:54.807 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 801 (ObjNum 754 snap 213 seq_num 754) dirty exists 2026-02-21T22:24:55.795 INFO:tasks.rados.rados.0.trial178.stdout:2245: done (1 left) 2026-02-21T22:24:55.795 INFO:tasks.rados.rados.0.trial178.stdout:2248: done (0 left) 2026-02-21T22:24:55.795 INFO:tasks.rados.rados.0.trial178.stdout:2249: snap_create 2026-02-21T22:24:56.764 INFO:tasks.rados.rados.0.trial178.stdout:2249: done (0 left) 2026-02-21T22:24:56.764 INFO:tasks.rados.rados.0.trial178.stdout:2250: write (excl) oid 46 current snap is 223 2026-02-21T22:24:56.764 INFO:tasks.rados.rados.0.trial178.stdout:2250: seq_num 793 ranges {527951=573169,1504456=556933,2743845=467765} 2026-02-21T22:24:56.771 INFO:tasks.rados.rados.0.trial178.stdout:2250: writing trial17818531-46 from 527951 to 1101120 tid 1 2026-02-21T22:24:56.778 INFO:tasks.rados.rados.0.trial178.stdout:2250: writing trial17818531-46 from 1504456 to 2061389 tid 2 2026-02-21T22:24:56.786 INFO:tasks.rados.rados.0.trial178.stdout:2250: writing trial17818531-46 from 2743845 to 3211610 tid 3 2026-02-21T22:24:56.786 INFO:tasks.rados.rados.0.trial178.stdout:2251: write (excl) oid 39 current snap is 223 2026-02-21T22:24:56.786 INFO:tasks.rados.rados.0.trial178.stdout:2251: seq_num 794 ranges {544758=648418,1846585=416944,2771268=573490} 2026-02-21T22:24:56.794 INFO:tasks.rados.rados.0.trial178.stdout:2251: writing trial17818531-39 from 544758 to 1193176 tid 1 2026-02-21T22:24:56.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: writing trial17818531-39 from 1846585 to 2263529 tid 2 2026-02-21T22:24:56.808 INFO:tasks.rados.rados.0.trial178.stdout:2251: writing trial17818531-39 from 2771268 to 3344758 tid 3 2026-02-21T22:24:56.808 INFO:tasks.rados.rados.0.trial178.stdout:2252: snap_remove snap 220 2026-02-21T22:24:57.799 INFO:tasks.rados.rados.0.trial178.stdout:2252: done (2 left) 2026-02-21T22:24:57.799 INFO:tasks.rados.rados.0.trial178.stdout:2253: rollback oid 43 current snap is 223 2026-02-21T22:24:57.799 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 219 2026-02-21T22:24:57.799 INFO:tasks.rados.rados.0.trial178.stdout:2250: finishing write tid 1 to trial17818531-46 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: finishing write tid 2 to trial17818531-46 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: finishing write tid 3 to trial17818531-46 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: finishing write tid 4 to trial17818531-46 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: finishing write tid 5 to trial17818531-46 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: oid 46 updating version 0 to 1002 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: oid 46 version 1002 is already newer than 1001 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: oid 46 updating version 1002 to 1003 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: oid 46 updating version 1003 to 1004 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1004 (ObjNum 793 snap 223 seq_num 793) dirty exists 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2250: left oid 46 (ObjNum 793 snap 223 seq_num 793) 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2251: finishing write tid 1 to trial17818531-39 2026-02-21T22:24:57.800 INFO:tasks.rados.rados.0.trial178.stdout:2251: finishing write tid 2 to trial17818531-39 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: finishing write tid 3 to trial17818531-39 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: finishing write tid 4 to trial17818531-39 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: finishing write tid 5 to trial17818531-39 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: oid 39 updating version 0 to 1118 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: oid 39 version 1118 is already newer than 1117 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: oid 39 updating version 1118 to 1119 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: oid 39 updating version 1119 to 1120 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1120 (ObjNum 794 snap 223 seq_num 794) dirty exists 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: left oid 39 (ObjNum 794 snap 223 seq_num 794) 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2250: done (2 left) 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2251: done (1 left) 2026-02-21T22:24:57.801 INFO:tasks.rados.rados.0.trial178.stdout:2254: read oid 42 snap 212 2026-02-21T22:24:57.802 INFO:tasks.rados.rados.0.trial178.stdout:2254: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:24:57.802 INFO:tasks.rados.rados.0.trial178.stdout:2255: snap_remove snap 222 2026-02-21T22:24:58.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:24:58.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:24:58.800 INFO:tasks.rados.rados.0.trial178.stdout:2255: done (2 left) 2026-02-21T22:24:58.800 INFO:tasks.rados.rados.0.trial178.stdout:2256: write oid 45 current snap is 223 2026-02-21T22:24:58.800 INFO:tasks.rados.rados.0.trial178.stdout:2256: seq_num 795 ranges {561565=723667,1905067=560602,3082338=279227} 2026-02-21T22:24:58.808 INFO:tasks.rados.rados.0.trial178.stdout:2256: writing trial17818531-45 from 561565 to 1285232 tid 1 2026-02-21T22:24:58.816 INFO:tasks.rados.rados.0.trial178.stdout:2256: writing trial17818531-45 from 1905067 to 2465669 tid 2 2026-02-21T22:24:58.823 INFO:tasks.rados.rados.0.trial178.stdout:2256: writing trial17818531-45 from 3082338 to 3361565 tid 3 2026-02-21T22:24:58.823 INFO:tasks.rados.rados.0.trial178.stdout:2257: read oid 15 snap 212 2026-02-21T22:24:58.823 INFO:tasks.rados.rados.0.trial178.stdout:2257: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:24:58.823 INFO:tasks.rados.rados.0.trial178.stdout:2258: write oid 12 current snap is 223 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:2253: finishing rollback tid 0 to trial17818531-43 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:2253: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:2253: finishing rollback tid 2 to trial17818531-43 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1133 (ObjNum 720 snap 200 seq_num 720) dirty exists 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:2254: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:24:58.824 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:58.892 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:58.978 INFO:tasks.rados.rados.0.trial178.stdout:2256: finishing write tid 1 to trial17818531-45 2026-02-21T22:24:58.978 INFO:tasks.rados.rados.0.trial178.stdout:2256: finishing write tid 2 to trial17818531-45 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: finishing write tid 3 to trial17818531-45 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: finishing write tid 4 to trial17818531-45 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: finishing write tid 5 to trial17818531-45 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: oid 45 updating version 0 to 1232 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: oid 45 version 1232 is already newer than 1231 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: oid 45 version 1232 is already newer than 1229 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: oid 45 version 1232 is already newer than 1230 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1232 (ObjNum 795 snap 223 seq_num 795) dirty exists 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2256: left oid 45 (ObjNum 795 snap 223 seq_num 795) 2026-02-21T22:24:58.979 INFO:tasks.rados.rados.0.trial178.stdout:2257: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:24:58.980 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1428108 2026-02-21T22:24:58.991 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:59.025 INFO:tasks.rados.rados.0.trial178.stdout:2258: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2026-02-21T22:24:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2258: writing trial17818531-12 from 578372 to 1377288 tid 1 2026-02-21T22:24:59.040 INFO:tasks.rados.rados.0.trial178.stdout:2258: writing trial17818531-12 from 1963549 to 2384162 tid 2 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2258: writing trial17818531-12 from 2826114 to 3244048 tid 3 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2253: done (4 left) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2254: done (3 left) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2256: done (2 left) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2257: done (1 left) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2259: read oid 2 snap -1 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2259: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2260: read oid 36 snap -1 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2260: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2261: copy_from oid 19 from oid 16 current snap is 223 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2258: finishing write tid 1 to trial17818531-12 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2258: finishing write tid 2 to trial17818531-12 2026-02-21T22:24:59.047 INFO:tasks.rados.rados.0.trial178.stdout:2262: snap_create 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: finishing write tid 3 to trial17818531-12 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: finishing write tid 4 to trial17818531-12 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: finishing write tid 5 to trial17818531-12 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: oid 12 updating version 0 to 803 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: oid 12 version 803 is already newer than 802 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: oid 12 updating version 803 to 804 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: oid 12 version 804 is already newer than 801 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 804 (ObjNum 796 snap 223 seq_num 796) dirty exists 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2258: left oid 12 (ObjNum 796 snap 223 seq_num 796) 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2261: finishing copy_from to trial17818531-19 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2261: got expected ENOENT (src dne) 2026-02-21T22:24:59.048 INFO:tasks.rados.rados.0.trial178.stdout:2261: finishing copy_from racing read to trial17818531-19 2026-02-21T22:24:59.049 INFO:tasks.rados.rados.0.trial178.stdout:2260: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:24:59.049 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:59.090 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2241437 2026-02-21T22:24:59.120 INFO:tasks.rados.rados.0.trial178.stdout:2259: expect (ObjNum 667 snap 188 seq_num 667) 2026-02-21T22:24:59.182 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2258: done (4 left) 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2259: done (3 left) 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2260: done (2 left) 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2261: done (1 left) 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2262: done (0 left) 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2263: write oid 17 current snap is 224 2026-02-21T22:24:59.805 INFO:tasks.rados.rados.0.trial178.stdout:2263: seq_num 797 ranges {595179=474165,1622031=564271,2737184=629206} 2026-02-21T22:24:59.812 INFO:tasks.rados.rados.0.trial178.stdout:2263: writing trial17818531-17 from 595179 to 1069344 tid 1 2026-02-21T22:24:59.819 INFO:tasks.rados.rados.0.trial178.stdout:2263: writing trial17818531-17 from 1622031 to 2186302 tid 2 2026-02-21T22:24:59.827 INFO:tasks.rados.rados.0.trial178.stdout:2263: writing trial17818531-17 from 2737184 to 3366390 tid 3 2026-02-21T22:24:59.827 INFO:tasks.rados.rados.0.trial178.stdout:2264: rollback oid 14 current snap is 224 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 212 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2263: finishing write tid 1 to trial17818531-17 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2263: finishing write tid 2 to trial17818531-17 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2265: read oid 18 snap -1 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2265: expect (ObjNum 721 snap 201 seq_num 721) 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2266: copy_from oid 19 from oid 6 current snap is 224 2026-02-21T22:24:59.828 INFO:tasks.rados.rados.0.trial178.stdout:2267: snap_create 2026-02-21T22:24:59.829 INFO:tasks.rados.rados.0.trial178.stdout:2265: expect (ObjNum 721 snap 201 seq_num 721) 2026-02-21T22:24:59.842 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 995316 2026-02-21T22:24:59.929 INFO:tasks.rados.rados.0.trial178.stdout:2263: finishing write tid 3 to trial17818531-17 2026-02-21T22:24:59.929 INFO:tasks.rados.rados.0.trial178.stdout:2263: finishing write tid 4 to trial17818531-17 2026-02-21T22:24:59.929 INFO:tasks.rados.rados.0.trial178.stdout:2264: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:24:59.929 INFO:tasks.rados.rados.0.trial178.stdout:2263: finishing write tid 5 to trial17818531-17 2026-02-21T22:24:59.929 INFO:tasks.rados.rados.0.trial178.stdout:2263: oid 17 updating version 0 to 833 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2263: oid 17 updating version 833 to 835 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2263: oid 17 version 835 is already newer than 834 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2263: oid 17 version 835 is already newer than 832 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 835 (ObjNum 797 snap 224 seq_num 797) dirty exists 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2263: left oid 17 (ObjNum 797 snap 224 seq_num 797) 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2264: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1122 (ObjNum 746 snap 209 seq_num 746) dirty exists 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2266: finishing copy_from racing read to trial17818531-19 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:2266: finishing copy_from to trial17818531-19 2026-02-21T22:24:59.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1127 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:25:00.806 INFO:tasks.rados.rados.0.trial178.stdout:2263: done (4 left) 2026-02-21T22:25:00.806 INFO:tasks.rados.rados.0.trial178.stdout:2264: done (3 left) 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2265: done (2 left) 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2266: done (1 left) 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2267: done (0 left) 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2268: copy_from oid 27 from oid 13 current snap is 225 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2269: write (excl) oid 47 current snap is 225 2026-02-21T22:25:00.807 INFO:tasks.rados.rados.0.trial178.stdout:2269: seq_num 798 ranges {611986=549414,1564160=424282,2764607=440478} 2026-02-21T22:25:00.814 INFO:tasks.rados.rados.0.trial178.stdout:2269: writing trial17818531-47 from 611986 to 1161400 tid 1 2026-02-21T22:25:00.819 INFO:tasks.rados.rados.0.trial178.stdout:2269: writing trial17818531-47 from 1564160 to 1988442 tid 2 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2269: writing trial17818531-47 from 2764607 to 3205085 tid 3 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2270: delete oid 11 current snap is 225 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2269: finishing write tid 1 to trial17818531-47 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2268: finishing copy_from racing read to trial17818531-27 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2268: finishing copy_from to trial17818531-27 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1001 (ObjNum 545 snap 153 seq_num 545) dirty exists 2026-02-21T22:25:00.828 INFO:tasks.rados.rados.0.trial178.stdout:2269: finishing write tid 2 to trial17818531-47 2026-02-21T22:25:00.829 INFO:tasks.rados.rados.0.trial178.stdout:2269: finishing write tid 3 to trial17818531-47 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: finishing write tid 4 to trial17818531-47 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: finishing write tid 5 to trial17818531-47 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: oid 47 updating version 0 to 1252 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: oid 47 version 1252 is already newer than 1249 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: oid 47 version 1252 is already newer than 1251 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: oid 47 version 1252 is already newer than 1250 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1252 (ObjNum 798 snap 225 seq_num 798) dirty exists 2026-02-21T22:25:00.830 INFO:tasks.rados.rados.0.trial178.stdout:2269: left oid 47 (ObjNum 798 snap 225 seq_num 798) 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2268: done (2 left) 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2269: done (1 left) 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2270: done (0 left) 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2271: read oid 1 snap -1 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2271: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:25:00.831 INFO:tasks.rados.rados.0.trial178.stdout:2272: delete oid 31 current snap is 225 2026-02-21T22:25:00.833 INFO:tasks.rados.rados.0.trial178.stdout:2272: done (1 left) 2026-02-21T22:25:00.833 INFO:tasks.rados.rados.0.trial178.stdout:2273: delete oid 4 current snap is 225 2026-02-21T22:25:00.839 INFO:tasks.rados.rados.0.trial178.stdout:2273: done (1 left) 2026-02-21T22:25:00.839 INFO:tasks.rados.rados.0.trial178.stdout:2274: read oid 38 snap -1 2026-02-21T22:25:00.839 INFO:tasks.rados.rados.0.trial178.stdout:2274: expect deleted 2026-02-21T22:25:00.839 INFO:tasks.rados.rados.0.trial178.stdout:2275: copy_from oid 45 from oid 25 current snap is 225 2026-02-21T22:25:00.839 INFO:tasks.rados.rados.0.trial178.stdout:2276: delete oid 4 current snap is 225 2026-02-21T22:25:00.840 INFO:tasks.rados.rados.0.trial178.stdout:2271: expect (ObjNum 688 snap 190 seq_num 688) 2026-02-21T22:25:00.923 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:00.987 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2275: finishing copy_from racing read to trial17818531-45 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2271: done (3 left) 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2274: done (2 left) 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2276: done (1 left) 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2277: write (excl) oid 31 current snap is 225 2026-02-21T22:25:01.050 INFO:tasks.rados.rados.0.trial178.stdout:2277: seq_num 803 ranges {696021=642012,1856570=575278,3185369=310652} 2026-02-21T22:25:01.057 INFO:tasks.rados.rados.0.trial178.stdout:2277: writing trial17818531-31 from 696021 to 1338033 tid 1 2026-02-21T22:25:01.064 INFO:tasks.rados.rados.0.trial178.stdout:2277: writing trial17818531-31 from 1856570 to 2431848 tid 2 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2277: writing trial17818531-31 from 3185369 to 3496021 tid 3 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2278: write (excl) oid 49 current snap is 225 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2275: finishing copy_from to trial17818531-45 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 801 (ObjNum 754 snap 213 seq_num 754) dirty exists 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2277: finishing write tid 1 to trial17818531-31 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2277: finishing write tid 2 to trial17818531-31 2026-02-21T22:25:01.072 INFO:tasks.rados.rados.0.trial178.stdout:2278: seq_num 804 ranges {712828=717261,1915052=718936,3212792=300036} 2026-02-21T22:25:01.081 INFO:tasks.rados.rados.0.trial178.stdout:2278: writing trial17818531-49 from 712828 to 1430089 tid 1 2026-02-21T22:25:01.089 INFO:tasks.rados.rados.0.trial178.stdout:2278: writing trial17818531-49 from 1915052 to 2633988 tid 2 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2278: writing trial17818531-49 from 3212792 to 3512828 tid 3 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2275: done (2 left) 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2279: read oid 42 snap -1 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2279: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2280: delete oid 1 current snap is 225 2026-02-21T22:25:01.095 INFO:tasks.rados.rados.0.trial178.stdout:2277: finishing write tid 3 to trial17818531-31 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: finishing write tid 4 to trial17818531-31 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: finishing write tid 5 to trial17818531-31 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: oid 31 updating version 0 to 1142 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: oid 31 version 1142 is already newer than 1140 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: oid 31 version 1142 is already newer than 1141 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: oid 31 version 1142 is already newer than 1139 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1142 (ObjNum 803 snap 225 seq_num 803) dirty exists 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2277: left oid 31 (ObjNum 803 snap 225 seq_num 803) 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2278: finishing write tid 1 to trial17818531-49 2026-02-21T22:25:01.096 INFO:tasks.rados.rados.0.trial178.stdout:2278: finishing write tid 2 to trial17818531-49 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: finishing write tid 3 to trial17818531-49 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: finishing write tid 4 to trial17818531-49 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: finishing write tid 5 to trial17818531-49 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: oid 49 updating version 0 to 1145 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: oid 49 updating version 1145 to 1146 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: oid 49 version 1146 is already newer than 1144 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: oid 49 version 1146 is already newer than 1143 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1146 (ObjNum 804 snap 225 seq_num 804) dirty exists 2026-02-21T22:25:01.097 INFO:tasks.rados.rados.0.trial178.stdout:2278: left oid 49 (ObjNum 804 snap 225 seq_num 804) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2277: done (3 left) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2278: done (2 left) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2280: done (1 left) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2281: read oid 40 snap -1 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2281: expect deleted 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2282: read oid 8 snap -1 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2282: expect (ObjNum 758 snap 214 seq_num 758) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2283: read oid 10 snap 223 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2283: expect (ObjNum 720 snap 200 seq_num 720) 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2284: copy_from oid 20 from oid 30 current snap is 225 2026-02-21T22:25:01.098 INFO:tasks.rados.rados.0.trial178.stdout:2285: snap_create 2026-02-21T22:25:01.099 INFO:tasks.rados.rados.0.trial178.stdout:2284: finishing copy_from to trial17818531-20 2026-02-21T22:25:01.099 INFO:tasks.rados.rados.0.trial178.stdout:2284: got expected ENOENT (src dne) 2026-02-21T22:25:01.099 INFO:tasks.rados.rados.0.trial178.stdout:2284: finishing copy_from racing read to trial17818531-20 2026-02-21T22:25:01.103 INFO:tasks.rados.rados.0.trial178.stdout:2283: expect (ObjNum 720 snap 200 seq_num 720) 2026-02-21T22:25:01.130 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:01.164 INFO:tasks.rados.rados.0.trial178.stdout:2279: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:25:01.213 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:01.297 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3699849 2026-02-21T22:25:01.299 INFO:tasks.rados.rados.0.trial178.stdout:2282: expect (ObjNum 758 snap 214 seq_num 758) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2279: done (5 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2281: done (4 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2282: done (3 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2283: done (2 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2284: done (1 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2285: done (0 left) 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2286: write oid 24 current snap is 226 2026-02-21T22:25:01.812 INFO:tasks.rados.rados.0.trial178.stdout:2286: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2026-02-21T22:25:01.821 INFO:tasks.rados.rados.0.trial178.stdout:2286: writing trial17818531-24 from 746442 to 1330554 tid 1 2026-02-21T22:25:01.828 INFO:tasks.rados.rados.0.trial178.stdout:2286: writing trial17818531-24 from 1748369 to 2470974 tid 2 2026-02-21T22:25:01.836 INFO:tasks.rados.rados.0.trial178.stdout:2286: writing trial17818531-24 from 2983991 to 3546442 tid 3 2026-02-21T22:25:01.836 INFO:tasks.rados.rados.0.trial178.stdout:2287: write oid 33 current snap is 226 2026-02-21T22:25:01.836 INFO:tasks.rados.rados.0.trial178.stdout:2287: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2026-02-21T22:25:01.845 INFO:tasks.rados.rados.0.trial178.stdout:2287: writing trial17818531-33 from 763249 to 1422610 tid 1 2026-02-21T22:25:01.854 INFO:tasks.rados.rados.0.trial178.stdout:2287: writing trial17818531-33 from 2090498 to 2673114 tid 2 2026-02-21T22:25:01.859 INFO:tasks.rados.rados.0.trial178.stdout:2287: writing trial17818531-33 from 3295061 to 3563249 tid 3 2026-02-21T22:25:01.859 INFO:tasks.rados.rados.0.trial178.stdout:2288: write (excl) oid 47 current snap is 226 2026-02-21T22:25:01.859 INFO:tasks.rados.rados.0.trial178.stdout:2288: seq_num 808 ranges {780056=734610,2148980=726274,3322484=257572} 2026-02-21T22:25:01.864 INFO:tasks.rados.rados.0.trial178.stdout:2288: writing trial17818531-47 from 780056 to 1514666 tid 1 2026-02-21T22:25:01.869 INFO:tasks.rados.rados.0.trial178.stdout:2288: writing trial17818531-47 from 2148980 to 2875254 tid 2 2026-02-21T22:25:01.872 INFO:tasks.rados.rados.0.trial178.stdout:2288: writing trial17818531-47 from 3322484 to 3580056 tid 3 2026-02-21T22:25:01.872 INFO:tasks.rados.rados.0.trial178.stdout:2289: rollback oid 27 current snap is 226 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 224 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2286: finishing write tid 1 to trial17818531-24 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2286: finishing write tid 2 to trial17818531-24 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2286: finishing write tid 3 to trial17818531-24 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2290: read oid 11 snap -1 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2290: expect deleted 2026-02-21T22:25:01.873 INFO:tasks.rados.rados.0.trial178.stdout:2291: snap_remove snap 212 2026-02-21T22:25:02.816 INFO:tasks.rados.rados.0.trial178.stdout:2291: done (5 left) 2026-02-21T22:25:02.816 INFO:tasks.rados.rados.0.trial178.stdout:2292: copy_from oid 39 from oid 14 current snap is 226 2026-02-21T22:25:02.816 INFO:tasks.rados.rados.0.trial178.stdout:2286: finishing write tid 4 to trial17818531-24 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: finishing write tid 5 to trial17818531-24 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: oid 24 updating version 0 to 1257 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: oid 24 version 1257 is already newer than 1255 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: oid 24 version 1257 is already newer than 1256 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: oid 24 version 1257 is already newer than 1254 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1257 (ObjNum 806 snap 226 seq_num 806) dirty exists 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2286: left oid 24 (ObjNum 806 snap 226 seq_num 806) 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2287: finishing write tid 1 to trial17818531-33 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2287: finishing write tid 2 to trial17818531-33 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2287: finishing write tid 3 to trial17818531-33 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2287: finishing write tid 4 to trial17818531-33 2026-02-21T22:25:02.817 INFO:tasks.rados.rados.0.trial178.stdout:2287: finishing write tid 5 to trial17818531-33 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: oid 33 updating version 0 to 475 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: oid 33 version 475 is already newer than 474 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: oid 33 version 475 is already newer than 473 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: oid 33 updating version 475 to 476 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 476 (ObjNum 807 snap 226 seq_num 807) dirty exists 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: left oid 33 (ObjNum 807 snap 226 seq_num 807) 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2288: finishing write tid 1 to trial17818531-47 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2288: finishing write tid 2 to trial17818531-47 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2286: done (5 left) 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2287: done (4 left) 2026-02-21T22:25:02.818 INFO:tasks.rados.rados.0.trial178.stdout:2293: snap_remove snap 221 2026-02-21T22:25:03.567 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:03.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:03.821 INFO:tasks.rados.rados.0.trial178.stdout:2293: done (4 left) 2026-02-21T22:25:03.821 INFO:tasks.rados.rados.0.trial178.stdout:2294: snap_remove snap 225 2026-02-21T22:25:04.822 INFO:tasks.rados.rados.0.trial178.stdout:2294: done (4 left) 2026-02-21T22:25:04.822 INFO:tasks.rados.rados.0.trial178.stdout:2295: write (excl) oid 2 current snap is 226 2026-02-21T22:25:04.823 INFO:tasks.rados.rados.0.trial178.stdout:2295: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2026-02-21T22:25:04.830 INFO:tasks.rados.rados.0.trial178.stdout:2295: writing trial17818531-2 from 796863 to 1206722 tid 1 2026-02-21T22:25:04.838 INFO:tasks.rados.rados.0.trial178.stdout:2295: writing trial17818531-2 from 1807462 to 2393747 tid 2 2026-02-21T22:25:04.847 INFO:tasks.rados.rados.0.trial178.stdout:2295: writing trial17818531-2 from 3066260 to 3596863 tid 3 2026-02-21T22:25:04.847 INFO:tasks.rados.rados.0.trial178.stdout:2296: read oid 15 snap -1 2026-02-21T22:25:04.847 INFO:tasks.rados.rados.0.trial178.stdout:2296: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:25:04.847 INFO:tasks.rados.rados.0.trial178.stdout:2297: read oid 18 snap 223 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2297: expect (ObjNum 721 snap 201 seq_num 721) 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2298: copy_from oid 10 from oid 33 current snap is 226 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: finishing write tid 3 to trial17818531-47 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: finishing write tid 4 to trial17818531-47 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: finishing write tid 5 to trial17818531-47 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: oid 47 updating version 0 to 1259 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: oid 47 updating version 1259 to 1262 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: oid 47 version 1262 is already newer than 1260 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:2288: oid 47 version 1262 is already newer than 1261 2026-02-21T22:25:04.848 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1262 (ObjNum 808 snap 226 seq_num 808) dirty exists 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2288: left oid 47 (ObjNum 808 snap 226 seq_num 808) 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2289: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2289: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1124 (ObjNum 33 snap 0 seq_num 1353482392) dirty dne 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2288: done (7 left) 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2289: done (6 left) 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2290: done (5 left) 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2299: write (excl) oid 28 current snap is 226 2026-02-21T22:25:04.849 INFO:tasks.rados.rados.0.trial178.stdout:2299: seq_num 810 ranges {413670=485108,1465944=729943,2977330=524801} 2026-02-21T22:25:04.852 INFO:tasks.rados.rados.0.trial178.stdout:2299: writing trial17818531-28 from 413670 to 898778 tid 1 2026-02-21T22:25:04.861 INFO:tasks.rados.rados.0.trial178.stdout:2299: writing trial17818531-28 from 1465944 to 2195887 tid 2 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2299: writing trial17818531-28 from 2977330 to 3502131 tid 3 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2300: rollback oid 38 current snap is 226 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 219 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2292: finishing copy_from racing read to trial17818531-39 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2292: finishing copy_from to trial17818531-39 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1122 (ObjNum 746 snap 209 seq_num 746) dirty exists 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2295: finishing write tid 1 to trial17818531-2 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2295: finishing write tid 2 to trial17818531-2 2026-02-21T22:25:04.871 INFO:tasks.rados.rados.0.trial178.stdout:2292: done (6 left) 2026-02-21T22:25:04.872 INFO:tasks.rados.rados.0.trial178.stdout:2301: snap_create 2026-02-21T22:25:04.872 INFO:tasks.rados.rados.0.trial178.stdout:2297: expect (ObjNum 721 snap 201 seq_num 721) 2026-02-21T22:25:04.937 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:04.966 INFO:tasks.rados.rados.0.trial178.stdout:2295: finishing write tid 3 to trial17818531-2 2026-02-21T22:25:04.966 INFO:tasks.rados.rados.0.trial178.stdout:2295: finishing write tid 4 to trial17818531-2 2026-02-21T22:25:04.966 INFO:tasks.rados.rados.0.trial178.stdout:2295: finishing write tid 5 to trial17818531-2 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2295: oid 2 updating version 0 to 1137 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2295: oid 2 version 1137 is already newer than 1136 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2295: oid 2 updating version 1137 to 1138 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2295: oid 2 version 1138 is already newer than 1135 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1138 (ObjNum 809 snap 226 seq_num 809) dirty exists 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2295: left oid 2 (ObjNum 809 snap 226 seq_num 809) 2026-02-21T22:25:04.967 INFO:tasks.rados.rados.0.trial178.stdout:2296: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:25:05.005 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1438041 2026-02-21T22:25:05.016 INFO:tasks.rados.rados.0.trial178.stdout:2299: finishing write tid 1 to trial17818531-28 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2298: finishing copy_from racing read to trial17818531-10 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2298: finishing copy_from to trial17818531-10 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 476 (ObjNum 807 snap 226 seq_num 807) dirty exists 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: finishing write tid 2 to trial17818531-28 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2300: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: finishing write tid 3 to trial17818531-28 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: finishing write tid 4 to trial17818531-28 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: finishing write tid 5 to trial17818531-28 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: oid 28 updating version 0 to 489 2026-02-21T22:25:05.017 INFO:tasks.rados.rados.0.trial178.stdout:2299: oid 28 version 489 is already newer than 486 2026-02-21T22:25:05.018 INFO:tasks.rados.rados.0.trial178.stdout:2299: oid 28 version 489 is already newer than 487 2026-02-21T22:25:05.018 INFO:tasks.rados.rados.0.trial178.stdout:2299: oid 28 version 489 is already newer than 488 2026-02-21T22:25:05.018 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 489 (ObjNum 810 snap 226 seq_num 810) dirty exists 2026-02-21T22:25:05.018 INFO:tasks.rados.rados.0.trial178.stdout:2299: left oid 28 (ObjNum 810 snap 226 seq_num 810) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2295: done (6 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2296: done (5 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2297: done (4 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2298: done (3 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2299: done (2 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2300: done (1 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2301: done (0 left) 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2302: write oid 4 current snap is 227 2026-02-21T22:25:05.827 INFO:tasks.rados.rados.0.trial178.stdout:2302: seq_num 811 ranges {430477=560357,1408073=473601,2488400=736073} 2026-02-21T22:25:05.834 INFO:tasks.rados.rados.0.trial178.stdout:2302: writing trial17818531-4 from 430477 to 990834 tid 1 2026-02-21T22:25:05.840 INFO:tasks.rados.rados.0.trial178.stdout:2302: writing trial17818531-4 from 1408073 to 1881674 tid 2 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2302: writing trial17818531-4 from 2488400 to 3224473 tid 3 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2303: copy_from oid 19 from oid 7 current snap is 227 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2302: finishing write tid 1 to trial17818531-4 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2302: finishing write tid 2 to trial17818531-4 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2304: rollback oid 26 current snap is 227 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 219 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2305: read oid 42 snap 224 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2305: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2306: copy_from oid 37 from oid 47 current snap is 227 2026-02-21T22:25:05.849 INFO:tasks.rados.rados.0.trial178.stdout:2307: snap_create 2026-02-21T22:25:05.850 INFO:tasks.rados.rados.0.trial178.stdout:2304: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:25:05.850 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: finishing write tid 3 to trial17818531-4 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: finishing write tid 4 to trial17818531-4 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: finishing write tid 5 to trial17818531-4 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: oid 4 updating version 0 to 807 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: oid 4 updating version 807 to 808 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: oid 4 updating version 808 to 809 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: oid 4 version 809 is already newer than 806 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 809 (ObjNum 811 snap 227 seq_num 811) dirty exists 2026-02-21T22:25:05.851 INFO:tasks.rados.rados.0.trial178.stdout:2302: left oid 4 (ObjNum 811 snap 227 seq_num 811) 2026-02-21T22:25:05.857 INFO:tasks.rados.rados.0.trial178.stdout:2305: expect (ObjNum 697 snap 192 seq_num 697) 2026-02-21T22:25:05.925 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:06.007 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3505614 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:2303: finishing copy_from racing read to trial17818531-19 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:2303: finishing copy_from to trial17818531-19 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1169 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:2306: finishing copy_from racing read to trial17818531-37 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:2306: finishing copy_from to trial17818531-37 2026-02-21T22:25:06.012 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1262 (ObjNum 808 snap 226 seq_num 808) dirty exists 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2302: done (5 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2303: done (4 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2304: done (3 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2305: done (2 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2306: done (1 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2307: done (0 left) 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2308: write (excl) oid 50 current snap is 228 2026-02-21T22:25:06.766 INFO:tasks.rados.rados.0.trial178.stdout:2308: seq_num 812 ranges {447284=635606,1866555=733612,3315823=331461} 2026-02-21T22:25:06.773 INFO:tasks.rados.rados.0.trial178.stdout:2308: writing trial17818531-50 from 447284 to 1082890 tid 1 2026-02-21T22:25:06.783 INFO:tasks.rados.rados.0.trial178.stdout:2308: writing trial17818531-50 from 1866555 to 2600167 tid 2 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2308: writing trial17818531-50 from 3315823 to 3647284 tid 3 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2309: read oid 17 snap -1 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2309: expect (ObjNum 797 snap 224 seq_num 797) 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2310: rollback oid 9 current snap is 228 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 226 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2308: finishing write tid 1 to trial17818531-50 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2308: finishing write tid 2 to trial17818531-50 2026-02-21T22:25:06.791 INFO:tasks.rados.rados.0.trial178.stdout:2311: snap_create 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: finishing write tid 3 to trial17818531-50 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: finishing write tid 4 to trial17818531-50 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: finishing write tid 5 to trial17818531-50 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: oid 50 updating version 0 to 1103 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: oid 50 version 1103 is already newer than 1102 2026-02-21T22:25:06.793 INFO:tasks.rados.rados.0.trial178.stdout:2308: oid 50 updating version 1103 to 1104 2026-02-21T22:25:06.794 INFO:tasks.rados.rados.0.trial178.stdout:2308: oid 50 version 1104 is already newer than 1101 2026-02-21T22:25:06.794 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1104 (ObjNum 812 snap 228 seq_num 812) dirty exists 2026-02-21T22:25:06.794 INFO:tasks.rados.rados.0.trial178.stdout:2308: left oid 50 (ObjNum 812 snap 228 seq_num 812) 2026-02-21T22:25:06.799 INFO:tasks.rados.rados.0.trial178.stdout:2310: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:25:06.799 INFO:tasks.rados.rados.0.trial178.stdout:2310: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:25:06.799 INFO:tasks.rados.rados.0.trial178.stdout:2310: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:25:06.799 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1127 (ObjNum 761 snap 214 seq_num 761) dirty exists 2026-02-21T22:25:06.803 INFO:tasks.rados.rados.0.trial178.stdout:2309: expect (ObjNum 797 snap 224 seq_num 797) 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2308: done (3 left) 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2309: done (2 left) 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2310: done (1 left) 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2311: done (0 left) 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2312: rollback oid 47 current snap is 229 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 223 2026-02-21T22:25:07.770 INFO:tasks.rados.rados.0.trial178.stdout:2313: snap_create 2026-02-21T22:25:07.774 INFO:tasks.rados.rados.0.trial178.stdout:2312: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:25:07.774 INFO:tasks.rados.rados.0.trial178.stdout:2312: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:25:07.774 INFO:tasks.rados.rados.0.trial178.stdout:2312: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:25:07.774 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1288 (ObjNum 653 snap 183 seq_num 653) dirty exists 2026-02-21T22:25:08.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:08.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:08.771 INFO:tasks.rados.rados.0.trial178.stdout:2312: done (1 left) 2026-02-21T22:25:08.771 INFO:tasks.rados.rados.0.trial178.stdout:2313: done (0 left) 2026-02-21T22:25:08.771 INFO:tasks.rados.rados.0.trial178.stdout:2314: rollback oid 34 current snap is 230 2026-02-21T22:25:08.771 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 34 to 229 2026-02-21T22:25:08.771 INFO:tasks.rados.rados.0.trial178.stdout:2315: delete oid 12 current snap is 230 2026-02-21T22:25:08.773 INFO:tasks.rados.rados.0.trial178.stdout:2314: finishing rollback tid 1 to trial17818531-34 2026-02-21T22:25:08.773 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 0 (ObjNum 257 snap 0 seq_num 99636416) dirty dne 2026-02-21T22:25:08.776 INFO:tasks.rados.rados.0.trial178.stdout:2314: done (1 left) 2026-02-21T22:25:08.776 INFO:tasks.rados.rados.0.trial178.stdout:2315: done (0 left) 2026-02-21T22:25:08.777 INFO:tasks.rados.rados.0.trial178.stdout:2316: delete oid 35 current snap is 230 2026-02-21T22:25:08.779 INFO:tasks.rados.rados.0.trial178.stdout:2316: done (0 left) 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2317: read oid 28 snap -1 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2317: expect (ObjNum 810 snap 226 seq_num 810) 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2318: read oid 13 snap -1 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2318: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2319: rollback oid 49 current snap is 230 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 227 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2320: read oid 20 snap -1 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2320: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:25:08.780 INFO:tasks.rados.rados.0.trial178.stdout:2321: snap_remove snap 219 2026-02-21T22:25:09.776 INFO:tasks.rados.rados.0.trial178.stdout:2321: done (4 left) 2026-02-21T22:25:09.776 INFO:tasks.rados.rados.0.trial178.stdout:2322: write oid 48 current snap is 230 2026-02-21T22:25:09.777 INFO:tasks.rados.rados.0.trial178.stdout:2322: seq_num 815 ranges {497705=577706,1642001=480939,2598092=497514} 2026-02-21T22:25:09.783 INFO:tasks.rados.rados.0.trial178.stdout:2322: writing trial17818531-48 from 497705 to 1075411 tid 1 2026-02-21T22:25:09.790 INFO:tasks.rados.rados.0.trial178.stdout:2322: writing trial17818531-48 from 1642001 to 2122940 tid 2 2026-02-21T22:25:09.797 INFO:tasks.rados.rados.0.trial178.stdout:2322: writing trial17818531-48 from 2598092 to 3095606 tid 3 2026-02-21T22:25:09.797 INFO:tasks.rados.rados.0.trial178.stdout:2323: write (excl) oid 4 current snap is 230 2026-02-21T22:25:09.797 INFO:tasks.rados.rados.0.trial178.stdout:2323: seq_num 816 ranges {514512=652955,1700483=740950,3025515=688997} 2026-02-21T22:25:09.804 INFO:tasks.rados.rados.0.trial178.stdout:2323: writing trial17818531-4 from 514512 to 1167467 tid 1 2026-02-21T22:25:09.813 INFO:tasks.rados.rados.0.trial178.stdout:2323: writing trial17818531-4 from 1700483 to 2441433 tid 2 2026-02-21T22:25:09.822 INFO:tasks.rados.rados.0.trial178.stdout:2323: writing trial17818531-4 from 3025515 to 3714512 tid 3 2026-02-21T22:25:09.822 INFO:tasks.rados.rados.0.trial178.stdout:2324: copy_from oid 45 from oid 43 current snap is 230 2026-02-21T22:25:09.822 INFO:tasks.rados.rados.0.trial178.stdout:2317: expect (ObjNum 810 snap 226 seq_num 810) 2026-02-21T22:25:09.949 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3350722 2026-02-21T22:25:09.954 INFO:tasks.rados.rados.0.trial178.stdout:2319: finishing rollback tid 0 to trial17818531-49 2026-02-21T22:25:09.954 INFO:tasks.rados.rados.0.trial178.stdout:2318: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:25:09.968 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1258092 2026-02-21T22:25:10.153 INFO:tasks.rados.rados.0.trial178.stdout:2320: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2319: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2319: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1149 (ObjNum 804 snap 225 seq_num 804) dirty exists 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2322: finishing write tid 1 to trial17818531-48 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2322: finishing write tid 2 to trial17818531-48 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2322: finishing write tid 3 to trial17818531-48 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2317: done (6 left) 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2318: done (5 left) 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2319: done (4 left) 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2320: done (3 left) 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2325: write oid 28 current snap is 230 2026-02-21T22:25:10.276 INFO:tasks.rados.rados.0.trial178.stdout:2325: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2026-02-21T22:25:10.284 INFO:tasks.rados.rados.0.trial178.stdout:2325: writing trial17818531-28 from 531319 to 1259523 tid 1 2026-02-21T22:25:10.291 INFO:tasks.rados.rados.0.trial178.stdout:2325: writing trial17818531-28 from 1758965 to 2243573 tid 2 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2325: writing trial17818531-28 from 2652938 to 3172996 tid 3 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2326: read oid 15 snap -1 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2326: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2327: read oid 19 snap -1 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2327: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2328: rollback oid 20 current snap is 230 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 227 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: finishing write tid 4 to trial17818531-48 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: finishing write tid 5 to trial17818531-48 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: oid 48 updating version 0 to 1263 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: oid 48 updating version 1263 to 1265 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: oid 48 updating version 1265 to 1266 2026-02-21T22:25:10.297 INFO:tasks.rados.rados.0.trial178.stdout:2322: oid 48 version 1266 is already newer than 1264 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1266 (ObjNum 815 snap 230 seq_num 815) dirty exists 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2322: left oid 48 (ObjNum 815 snap 230 seq_num 815) 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2323: finishing write tid 1 to trial17818531-4 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2323: finishing write tid 2 to trial17818531-4 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2322: done (6 left) 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2329: write oid 44 current snap is 230 2026-02-21T22:25:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2329: seq_num 818 ranges {548126=403453,1417447=744619,2680361=447683,3680450=67676} 2026-02-21T22:25:10.303 INFO:tasks.rados.rados.0.trial178.stdout:2329: writing trial17818531-44 from 548126 to 951579 tid 1 2026-02-21T22:25:10.312 INFO:tasks.rados.rados.0.trial178.stdout:2329: writing trial17818531-44 from 1417447 to 2162066 tid 2 2026-02-21T22:25:10.318 INFO:tasks.rados.rados.0.trial178.stdout:2329: writing trial17818531-44 from 2680361 to 3128044 tid 3 2026-02-21T22:25:10.322 INFO:tasks.rados.rados.0.trial178.stdout:2329: writing trial17818531-44 from 3680450 to 3748126 tid 4 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2330: delete oid 47 current snap is 230 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: finishing write tid 3 to trial17818531-4 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: finishing write tid 4 to trial17818531-4 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: finishing write tid 5 to trial17818531-4 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: oid 4 updating version 0 to 812 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: oid 4 version 812 is already newer than 811 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: oid 4 updating version 812 to 814 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: oid 4 version 814 is already newer than 813 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 814 (ObjNum 816 snap 230 seq_num 816) dirty exists 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2323: left oid 4 (ObjNum 816 snap 230 seq_num 816) 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2324: finishing copy_from racing read to trial17818531-45 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2324: finishing copy_from to trial17818531-45 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1133 (ObjNum 720 snap 200 seq_num 720) dirty exists 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: finishing write tid 1 to trial17818531-28 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: finishing write tid 2 to trial17818531-28 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: finishing write tid 3 to trial17818531-28 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: finishing write tid 4 to trial17818531-28 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: finishing write tid 5 to trial17818531-28 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: oid 28 updating version 0 to 498 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: oid 28 version 498 is already newer than 496 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: oid 28 version 498 is already newer than 497 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: oid 28 version 498 is already newer than 495 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 498 (ObjNum 817 snap 230 seq_num 817) dirty exists 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2325: left oid 28 (ObjNum 817 snap 230 seq_num 817) 2026-02-21T22:25:10.323 INFO:tasks.rados.rados.0.trial178.stdout:2326: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:25:10.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:10.368 INFO:tasks.rados.rados.0.trial178.stdout:2328: finishing rollback tid 0 to trial17818531-20 2026-02-21T22:25:10.368 INFO:tasks.rados.rados.0.trial178.stdout:2327: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:25:10.370 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1107425 2026-02-21T22:25:10.391 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:10.431 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3248034 2026-02-21T22:25:10.434 INFO:tasks.rados.rados.0.trial178.stdout:2328: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2328: finishing rollback tid 2 to trial17818531-20 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 847 (ObjNum 724 snap 204 seq_num 724) dirty exists 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 1 to trial17818531-44 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 2 to trial17818531-44 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 3 to trial17818531-44 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 4 to trial17818531-44 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 5 to trial17818531-44 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2323: done (7 left) 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2324: done (6 left) 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2325: done (5 left) 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2326: done (4 left) 2026-02-21T22:25:10.435 INFO:tasks.rados.rados.0.trial178.stdout:2327: done (3 left) 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2328: done (2 left) 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2330: done (1 left) 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2331: copy_from oid 18 from oid 41 current snap is 230 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: finishing write tid 6 to trial17818531-44 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: oid 44 updating version 0 to 815 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: oid 44 updating version 815 to 817 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: oid 44 updating version 817 to 819 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: oid 44 version 819 is already newer than 816 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: oid 44 version 819 is already newer than 818 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 819 (ObjNum 818 snap 230 seq_num 818) dirty exists 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: left oid 44 (ObjNum 818 snap 230 seq_num 818) 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2329: done (1 left) 2026-02-21T22:25:10.436 INFO:tasks.rados.rados.0.trial178.stdout:2332: snap_create 2026-02-21T22:25:10.446 INFO:tasks.rados.rados.0.trial178.stdout:2331: finishing copy_from racing read to trial17818531-18 2026-02-21T22:25:10.447 INFO:tasks.rados.rados.0.trial178.stdout:2331: finishing copy_from to trial17818531-18 2026-02-21T22:25:10.448 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 808 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:25:11.133 INFO:tasks.rados.rados.0.trial178.stdout:2331: done (1 left) 2026-02-21T22:25:11.133 INFO:tasks.rados.rados.0.trial178.stdout:2332: done (0 left) 2026-02-21T22:25:11.133 INFO:tasks.rados.rados.0.trial178.stdout:2333: write oid 18 current snap is 231 2026-02-21T22:25:11.133 INFO:tasks.rados.rados.0.trial178.stdout:2333: seq_num 820 ranges {581740=553951,1818058=748288,3018854=470227} 2026-02-21T22:25:11.140 INFO:tasks.rados.rados.0.trial178.stdout:2333: writing trial17818531-18 from 581740 to 1135691 tid 1 2026-02-21T22:25:11.150 INFO:tasks.rados.rados.0.trial178.stdout:2333: writing trial17818531-18 from 1818058 to 2566346 tid 2 2026-02-21T22:25:11.156 INFO:tasks.rados.rados.0.trial178.stdout:2333: writing trial17818531-18 from 3018854 to 3489081 tid 3 2026-02-21T22:25:11.156 INFO:tasks.rados.rados.0.trial178.stdout:2334: snap_create 2026-02-21T22:25:11.156 INFO:tasks.rados.rados.0.trial178.stdout:2333: finishing write tid 1 to trial17818531-18 2026-02-21T22:25:11.156 INFO:tasks.rados.rados.0.trial178.stdout:2333: finishing write tid 2 to trial17818531-18 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: finishing write tid 3 to trial17818531-18 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: finishing write tid 4 to trial17818531-18 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: finishing write tid 5 to trial17818531-18 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: oid 18 updating version 0 to 1142 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: oid 18 version 1142 is already newer than 1141 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: oid 18 version 1142 is already newer than 1139 2026-02-21T22:25:11.158 INFO:tasks.rados.rados.0.trial178.stdout:2333: oid 18 version 1142 is already newer than 1140 2026-02-21T22:25:11.159 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1142 (ObjNum 820 snap 231 seq_num 820) dirty exists 2026-02-21T22:25:11.159 INFO:tasks.rados.rados.0.trial178.stdout:2333: left oid 18 (ObjNum 820 snap 231 seq_num 820) 2026-02-21T22:25:11.767 INFO:tasks.rados.rados.0.trial178.stdout:2333: done (1 left) 2026-02-21T22:25:11.767 INFO:tasks.rados.rados.0.trial178.stdout:2334: done (0 left) 2026-02-21T22:25:11.767 INFO:tasks.rados.rados.0.trial178.stdout:2335: delete oid 44 current snap is 232 2026-02-21T22:25:11.772 INFO:tasks.rados.rados.0.trial178.stdout:2335: done (0 left) 2026-02-21T22:25:11.772 INFO:tasks.rados.rados.0.trial178.stdout:2336: write (excl) oid 27 current snap is 232 2026-02-21T22:25:11.772 INFO:tasks.rados.rados.0.trial178.stdout:2336: seq_num 822 ranges {615354=420802,1651375=751957,3190053=492771} 2026-02-21T22:25:11.780 INFO:tasks.rados.rados.0.trial178.stdout:2336: writing trial17818531-27 from 615354 to 1036156 tid 1 2026-02-21T22:25:11.789 INFO:tasks.rados.rados.0.trial178.stdout:2336: writing trial17818531-27 from 1651375 to 2403332 tid 2 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:2336: writing trial17818531-27 from 3190053 to 3682824 tid 3 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:2337: rollback oid 47 current snap is 232 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 230 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:2336: finishing write tid 1 to trial17818531-27 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:2338: write (excl) oid 43 current snap is 232 2026-02-21T22:25:11.798 INFO:tasks.rados.rados.0.trial178.stdout:2338: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2026-02-21T22:25:11.805 INFO:tasks.rados.rados.0.trial178.stdout:2338: writing trial17818531-43 from 632161 to 1128212 tid 1 2026-02-21T22:25:11.812 INFO:tasks.rados.rados.0.trial178.stdout:2338: writing trial17818531-43 from 1593504 to 2089119 tid 2 2026-02-21T22:25:11.818 INFO:tasks.rados.rados.0.trial178.stdout:2338: writing trial17818531-43 from 2701123 to 3121519 tid 3 2026-02-21T22:25:11.820 INFO:tasks.rados.rados.0.trial178.stdout:2338: writing trial17818531-43 from 3611645 to 3832161 tid 4 2026-02-21T22:25:11.820 INFO:tasks.rados.rados.0.trial178.stdout:2339: write (excl) oid 10 current snap is 232 2026-02-21T22:25:11.820 INFO:tasks.rados.rados.0.trial178.stdout:2339: seq_num 824 ranges {648968=571300,1651986=639273,2728546=631668} 2026-02-21T22:25:11.825 INFO:tasks.rados.rados.0.trial178.stdout:2339: writing trial17818531-10 from 648968 to 1220268 tid 1 2026-02-21T22:25:11.829 INFO:tasks.rados.rados.0.trial178.stdout:2339: writing trial17818531-10 from 1651986 to 2291259 tid 2 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2339: writing trial17818531-10 from 2728546 to 3360214 tid 3 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2340: copy_from oid 26 from oid 15 current snap is 232 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2336: finishing write tid 2 to trial17818531-27 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2336: finishing write tid 3 to trial17818531-27 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2336: finishing write tid 4 to trial17818531-27 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2341: write (excl) oid 36 current snap is 232 2026-02-21T22:25:11.833 INFO:tasks.rados.rados.0.trial178.stdout:2341: seq_num 825 ranges {665775=646549,2110468=499284,3155969=442940} 2026-02-21T22:25:11.838 INFO:tasks.rados.rados.0.trial178.stdout:2341: writing trial17818531-36 from 665775 to 1312324 tid 1 2026-02-21T22:25:11.843 INFO:tasks.rados.rados.0.trial178.stdout:2341: writing trial17818531-36 from 2110468 to 2609752 tid 2 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2341: writing trial17818531-36 from 3155969 to 3598909 tid 3 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2342: copy_from oid 33 from oid 17 current snap is 232 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: finishing write tid 5 to trial17818531-27 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: oid 27 updating version 0 to 1128 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: oid 27 updating version 1128 to 1130 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: oid 27 updating version 1130 to 1131 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: oid 27 version 1131 is already newer than 1129 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1131 (ObjNum 822 snap 232 seq_num 822) dirty exists 2026-02-21T22:25:11.847 INFO:tasks.rados.rados.0.trial178.stdout:2336: left oid 27 (ObjNum 822 snap 232 seq_num 822) 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2337: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1296 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 1 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 2 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 3 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2336: done (6 left) 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2337: done (5 left) 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2343: snap_create 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 4 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 5 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: finishing write tid 6 to trial17818531-43 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: oid 43 updating version 0 to 1134 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: oid 43 updating version 1134 to 1138 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: oid 43 version 1138 is already newer than 1135 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: oid 43 version 1138 is already newer than 1137 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: oid 43 version 1138 is already newer than 1136 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1138 (ObjNum 823 snap 232 seq_num 823) dirty exists 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2338: left oid 43 (ObjNum 823 snap 232 seq_num 823) 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: finishing write tid 1 to trial17818531-10 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: finishing write tid 2 to trial17818531-10 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: finishing write tid 3 to trial17818531-10 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: finishing write tid 4 to trial17818531-10 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: finishing write tid 5 to trial17818531-10 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: oid 10 updating version 0 to 1142 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: oid 10 version 1142 is already newer than 1141 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: oid 10 version 1142 is already newer than 1140 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: oid 10 version 1142 is already newer than 1139 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1142 (ObjNum 824 snap 232 seq_num 824) dirty exists 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2339: left oid 10 (ObjNum 824 snap 232 seq_num 824) 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2340: finishing copy_from racing read to trial17818531-26 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2341: finishing write tid 1 to trial17818531-36 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2340: finishing copy_from to trial17818531-26 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 765 (ObjNum 745 snap 209 seq_num 745) dirty exists 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2341: finishing write tid 2 to trial17818531-36 2026-02-21T22:25:11.848 INFO:tasks.rados.rados.0.trial178.stdout:2341: finishing write tid 3 to trial17818531-36 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: finishing write tid 4 to trial17818531-36 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: finishing write tid 5 to trial17818531-36 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: oid 36 updating version 0 to 1083 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: oid 36 updating version 1083 to 1085 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: oid 36 version 1085 is already newer than 1084 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: oid 36 version 1085 is already newer than 1082 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1085 (ObjNum 825 snap 232 seq_num 825) dirty exists 2026-02-21T22:25:11.849 INFO:tasks.rados.rados.0.trial178.stdout:2341: left oid 36 (ObjNum 825 snap 232 seq_num 825) 2026-02-21T22:25:11.862 INFO:tasks.rados.rados.0.trial178.stdout:2342: finishing copy_from racing read to trial17818531-33 2026-02-21T22:25:11.862 INFO:tasks.rados.rados.0.trial178.stdout:2342: finishing copy_from to trial17818531-33 2026-02-21T22:25:11.862 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 835 (ObjNum 797 snap 224 seq_num 797) dirty exists 2026-02-21T22:25:12.768 INFO:tasks.rados.rados.0.trial178.stdout:2338: done (5 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2339: done (4 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2340: done (3 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2341: done (2 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2342: done (1 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2343: done (0 left) 2026-02-21T22:25:12.769 INFO:tasks.rados.rados.0.trial178.stdout:2344: snap_create 2026-02-21T22:25:13.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:13.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:13.778 INFO:tasks.rados.rados.0.trial178.stdout:2344: done (0 left) 2026-02-21T22:25:13.778 INFO:tasks.rados.rados.0.trial178.stdout:2345: snap_create 2026-02-21T22:25:14.779 INFO:tasks.rados.rados.0.trial178.stdout:2345: done (0 left) 2026-02-21T22:25:14.779 INFO:tasks.rados.rados.0.trial178.stdout:2346: read oid 2 snap -1 2026-02-21T22:25:14.779 INFO:tasks.rados.rados.0.trial178.stdout:2346: expect (ObjNum 809 snap 226 seq_num 809) 2026-02-21T22:25:14.779 INFO:tasks.rados.rados.0.trial178.stdout:2347: snap_remove snap 232 2026-02-21T22:25:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2347: done (1 left) 2026-02-21T22:25:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2348: rollback oid 32 current snap is 235 2026-02-21T22:25:15.780 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 233 2026-02-21T22:25:15.780 INFO:tasks.rados.rados.0.trial178.stdout:2346: expect (ObjNum 809 snap 226 seq_num 809) 2026-02-21T22:25:15.846 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:15.915 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2901176 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2348: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2348: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2348: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1148 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2346: done (1 left) 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2348: done (0 left) 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2349: read oid 23 snap -1 2026-02-21T22:25:15.930 INFO:tasks.rados.rados.0.trial178.stdout:2349: expect deleted 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2350: read oid 21 snap -1 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2350: expect (ObjNum 782 snap 219 seq_num 782) 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2351: read oid 27 snap -1 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2351: expect (ObjNum 822 snap 232 seq_num 822) 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2352: write oid 8 current snap is 235 2026-02-21T22:25:15.931 INFO:tasks.rados.rados.0.trial178.stdout:2352: seq_num 826 ranges {682582=721798,2168950=642942,3583392=299190} 2026-02-21T22:25:15.939 INFO:tasks.rados.rados.0.trial178.stdout:2352: writing trial17818531-8 from 682582 to 1404380 tid 1 2026-02-21T22:25:15.949 INFO:tasks.rados.rados.0.trial178.stdout:2352: writing trial17818531-8 from 2168950 to 2811892 tid 2 2026-02-21T22:25:15.956 INFO:tasks.rados.rados.0.trial178.stdout:2352: writing trial17818531-8 from 3583392 to 3882582 tid 3 2026-02-21T22:25:15.956 INFO:tasks.rados.rados.0.trial178.stdout:2353: rollback oid 14 current snap is 235 2026-02-21T22:25:15.956 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 227 2026-02-21T22:25:15.956 INFO:tasks.rados.rados.0.trial178.stdout:2350: expect (ObjNum 782 snap 219 seq_num 782) 2026-02-21T22:25:16.007 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2447717 2026-02-21T22:25:16.046 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2322684 2026-02-21T22:25:16.102 INFO:tasks.rados.rados.0.trial178.stdout:2351: expect (ObjNum 822 snap 232 seq_num 822) 2026-02-21T22:25:16.161 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2470163 2026-02-21T22:25:16.175 INFO:tasks.rados.rados.0.trial178.stdout:2352: finishing write tid 1 to trial17818531-8 2026-02-21T22:25:16.175 INFO:tasks.rados.rados.0.trial178.stdout:2352: finishing write tid 2 to trial17818531-8 2026-02-21T22:25:16.175 INFO:tasks.rados.rados.0.trial178.stdout:2352: finishing write tid 3 to trial17818531-8 2026-02-21T22:25:16.175 INFO:tasks.rados.rados.0.trial178.stdout:2349: done (4 left) 2026-02-21T22:25:16.175 INFO:tasks.rados.rados.0.trial178.stdout:2350: done (3 left) 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2351: done (2 left) 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2354: snap_create 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: finishing write tid 4 to trial17818531-8 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2353: finishing rollback tid 0 to trial17818531-14 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: finishing write tid 5 to trial17818531-8 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: oid 8 updating version 0 to 1301 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: oid 8 updating version 1301 to 1302 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: oid 8 version 1302 is already newer than 1300 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: oid 8 version 1302 is already newer than 1299 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1302 (ObjNum 826 snap 235 seq_num 826) dirty exists 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2352: left oid 8 (ObjNum 826 snap 235 seq_num 826) 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2353: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:2353: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:25:16.176 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1134 (ObjNum 746 snap 209 seq_num 746) dirty exists 2026-02-21T22:25:16.768 INFO:tasks.rados.rados.0.trial178.stdout:2352: done (2 left) 2026-02-21T22:25:16.768 INFO:tasks.rados.rados.0.trial178.stdout:2353: done (1 left) 2026-02-21T22:25:16.768 INFO:tasks.rados.rados.0.trial178.stdout:2354: done (0 left) 2026-02-21T22:25:16.768 INFO:tasks.rados.rados.0.trial178.stdout:2355: write (excl) oid 48 current snap is 236 2026-02-21T22:25:16.768 INFO:tasks.rados.rados.0.trial178.stdout:2355: seq_num 827 ranges {699389=797047,2111079=502953,3094462=581837} 2026-02-21T22:25:16.778 INFO:tasks.rados.rados.0.trial178.stdout:2355: writing trial17818531-48 from 699389 to 1496436 tid 1 2026-02-21T22:25:16.786 INFO:tasks.rados.rados.0.trial178.stdout:2355: writing trial17818531-48 from 2111079 to 2614032 tid 2 2026-02-21T22:25:16.793 INFO:tasks.rados.rados.0.trial178.stdout:2355: writing trial17818531-48 from 3094462 to 3676299 tid 3 2026-02-21T22:25:16.793 INFO:tasks.rados.rados.0.trial178.stdout:2356: snap_create 2026-02-21T22:25:16.793 INFO:tasks.rados.rados.0.trial178.stdout:2355: finishing write tid 1 to trial17818531-48 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: finishing write tid 2 to trial17818531-48 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: finishing write tid 3 to trial17818531-48 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: finishing write tid 4 to trial17818531-48 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: finishing write tid 5 to trial17818531-48 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: oid 48 updating version 0 to 1268 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: oid 48 updating version 1268 to 1269 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: oid 48 updating version 1269 to 1270 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: oid 48 version 1270 is already newer than 1267 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1270 (ObjNum 827 snap 236 seq_num 827) dirty exists 2026-02-21T22:25:16.796 INFO:tasks.rados.rados.0.trial178.stdout:2355: left oid 48 (ObjNum 827 snap 236 seq_num 827) 2026-02-21T22:25:17.133 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:25:17.133 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:25:17.133 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-21T22:25:17.133 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-21T22:25:17.145 INFO:tasks.ceph.mon.c:Started 2026-02-21T22:25:17.145 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.a 2026-02-21T22:25:17.145 INFO:tasks.mon_thrash:Sending STOP to mon a 2026-02-21T22:25:17.145 INFO:tasks.ceph.mon.a:Sent signal 19 2026-02-21T22:25:17.145 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:25:18.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:18.568 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:25:23] 2026-02-21T22:25:23.530 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:25:23, mode upmap, max misplaced 0.050000 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['unique_pool_0', 'rbd', '.mgr'] 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:25:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:25:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:23.550 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:25:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:25:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:25:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:25:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:25:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:25:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:25:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.554 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:25:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:25:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:25:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:25:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:25:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:25:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:25:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:25:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:25:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:25:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:25:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "485", 2026-02-21T22:25:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:25:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:25:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:25:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 485, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 242, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:25:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:25:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:25:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00045362401891637733 of space, bias 1.0, pg target 0.1360872056749132 quantized to 32 (current 16) 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:25:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:28.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:28.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:31.894 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:25:31.894 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Check health 2026-02-21T22:25:31.897 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Last scrape 20260221-221531, next scrape due 20260222-000000 2026-02-21T22:25:31.897 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 600 seconds 2026-02-21T22:25:32.146 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.a 2026-02-21T22:25:32.146 INFO:tasks.mon_thrash:Sending CONT to mon a 2026-02-21T22:25:32.146 INFO:tasks.ceph.mon.a:Sent signal 18 2026-02-21T22:25:32.146 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:25:32.146 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: get_monmap_and_config 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: build_initial_monmap 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: monmap: 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:32.210906+0000 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:32.210906+0000 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:32.209 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 1 Processor -- start 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 1 -- start start 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: init 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient(hunting): picked mon.noname-a con 0x7f291808e5b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient(hunting): picked mon.noname-b con 0x7f2918059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient(hunting): picked mon.noname-c con 0x7f291808f090 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2918057ed0 con 0x7f291808e5b0 2026-02-21T22:25:32.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29180581b0 con 0x7f291808f090 2026-02-21T22:25:32.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2918058490 con 0x7f2918059150 2026-02-21T22:25:32.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:32.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291f149640 10 monclient(hunting): authenticate will time out at 1741.446836s 2026-02-21T22:25:32.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291cebe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 0x7f291808e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:48392/0 (socket says 10.20.193.102:48392) 2026-02-21T22:25:32.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.208+0000 7f291cebe640 1 -- 10.20.193.102:0/667647586 learned_addr learned my addr 10.20.193.102:0/667647586 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:32.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291cebe640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 legacy=0x7f291808e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290ffff640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.102:6790/0 conn(0x7f2918059150 legacy=0x7f2918183fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990312573 0 0) 0x7f2918058490 con 0x7f2918059150 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient(hunting): my global_id is 84101 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291d6bf640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.178:6789/0 conn(0x7f291808f090 legacy=0x7f29181a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2904003620 con 0x7f2918059150 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1372420851 0 0) 0x7f2904003620 con 0x7f2918059150 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.178:6789/0 conn(0x7f291808f090 legacy=0x7f29181a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 legacy=0x7f291808e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 monclient: found mon.noname-b 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291805e000 con 0x7f2918059150 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2900003160 con 0x7f2918059150 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:32.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: dump: 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 5 monclient: authenticate success, global_id 84101 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f29000034a0 con 0x7f2918059150 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 1 -- 10.20.193.102:0/667647586 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f29000038c0 con 0x7f2918059150 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290f7fe640 10 monclient: dump: 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.216 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 4 set_mon_vals no callback set 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: shutdown 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- 10.20.193.102:0/667647586 >> v1:10.20.193.102:6790/0 conn(0x7f2918059150 legacy=0x7f2918183fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- 10.20.193.102:0/667647586 shutdown_connections 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- 10.20.193.102:0/667647586 >> 10.20.193.102:0/667647586 conn(0x7f2918056d40 msgr2=0x7f2918093bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- 10.20.193.102:0/667647586 shutdown_connections 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- 10.20.193.102:0/667647586 wait complete. 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: build_initial_monmap 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: monmap: 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:32.217 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 Processor -- start 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- start start 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: init 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient(hunting): picked mon.noname-b con 0x7f2918059150 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient(hunting): picked mon.noname-a con 0x7f291808e5b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient(hunting): picked mon.noname-c con 0x7f291808f090 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2918140100 con 0x7f291808e5b0 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291812df10 con 0x7f2918059150 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291805e900 con 0x7f291808f090 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291f149640 10 monclient(hunting): authenticate will time out at 1741.450934s 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290ffff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 0x7f291813c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:48396/0 (socket says 10.20.193.102:48396) 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f291d6bf640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f291808f090 0x7f2918195620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59718/0 (socket says 10.20.193.102:59718) 2026-02-21T22:25:32.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.212+0000 7f290ffff640 1 -- 10.20.193.102:0/4064304352 learned_addr learned my addr 10.20.193.102:0/4064304352 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291d6bf640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.102:6790/0 conn(0x7f291808f090 legacy=0x7f2918195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290ffff640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 legacy=0x7f291813c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554502197 0 0) 0x7f291805e900 con 0x7f291808f090 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): my global_id is 84104 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28e8003620 con 0x7f291808f090 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291cebe640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.178:6789/0 conn(0x7f2918059150 legacy=0x7f29181532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3534285624 0 0) 0x7f28e8003620 con 0x7f291808f090 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f291805e900 con 0x7f291808f090 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2914004890 con 0x7f291808f090 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): dump: 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:32.220 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852381174 0 0) 0x7f291812df10 con 0x7f2918059150 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient(hunting): my global_id is 54265 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28e8003620 con 0x7f2918059150 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 293258482 0 0) 0x7f291805e900 con 0x7f291808f090 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.178:6789/0 conn(0x7f2918059150 legacy=0x7f29181532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.102:6789/0 conn(0x7f291808e5b0 legacy=0x7f291813c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 monclient: found mon.c 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291812bbb0 con 0x7f291808f090 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:25:32.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 5 monclient: authenticate success, global_id 84104 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _renew_subs 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f291812dc30 con 0x7f291808f090 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _renew_subs 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f291813cdf0 con 0x7f291808f090 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291812d8b0 con 0x7f291808f090 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2914004ed0 con 0x7f291808f090 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2914005260 con 0x7f291808f090 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 10 monclient: dump: 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:32.222 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290effd640 4 set_mon_vals no callback set 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f29140054b0 con 0x7f291808f090 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(485..485 src has 112..485) ==== 3927+0+0 (unknown 2442892133 0 0) 0x7f291404db50 con 0x7f291808f090 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.216+0000 7f291cebe640 10 monclient: get_auth_request con 0x7f28e803d5e0 auth_method 0 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.220+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f2914018af0 con 0x7f291808f090 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.220+0000 7f290d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:32.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.220+0000 7f290d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.332+0000 7f291f149640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.332+0000 7f291f149640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.332+0000 7f291f149640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.332+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2918062ce0 con 0x7f291808f090 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.332+0000 7f290d7fa640 1 -- 10.20.193.102:0/4064304352 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 2164488820) 0x7f291401ddb0 con 0x7f291808f090 2026-02-21T22:25:32.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f290d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:32.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f290d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:32.336 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":36,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.178:6800/2549949439 conn(0x7f28e803d5e0 legacy=0x7f28e805da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 10 monclient: shutdown 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 >> v1:10.20.193.102:6790/0 conn(0x7f291808f090 legacy=0x7f2918195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 shutdown_connections 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 >> 10.20.193.102:0/4064304352 conn(0x7f2918056d40 msgr2=0x7f2918140620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 shutdown_connections 2026-02-21T22:25:32.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:32.336+0000 7f291f149640 1 -- 10.20.193.102:0/4064304352 wait complete. 2026-02-21T22:25:33.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:33.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:35.346 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:35.420 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: get_monmap_and_config 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: build_initial_monmap 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: monmap: 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:35.422147+0000 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:35.422147+0000 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:35.421 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:35.422 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 1 Processor -- start 2026-02-21T22:25:35.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 1 -- start start 2026-02-21T22:25:35.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: init 2026-02-21T22:25:35.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:35.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-c con 0x7f3d9c13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-b con 0x7f3d9c13fd90 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-a con 0x7f3d9c1401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c092490 con 0x7f3d9c1401f0 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c0571f0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c0574d0 con 0x7f3d9c13fd90 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3da08f5640 10 monclient(hunting): authenticate will time out at 1744.658268s 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3d9affd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13fd90 0x7f3d9c183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59722/0 (socket says 10.20.193.102:59722) 2026-02-21T22:25:35.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.420+0000 7f3d9affd640 1 -- 10.20.193.102:0/768039641 learned_addr learned my addr 10.20.193.102:0/768039641 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9affd640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13fd90 legacy=0x7f3d9c183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9bfff640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.102:6789/0 conn(0x7f3d9c1401f0 legacy=0x7f3d9c1a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9b7fe640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.178:6789/0 conn(0x7f3d9c13f9c0 legacy=0x7f3d9c0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3895986906 0 0) 0x7f3d9c0574d0 con 0x7f3d9c13fd90 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): my global_id is 84107 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d80003620 con 0x7f3d9c13fd90 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1345876587 0 0) 0x7f3d9c0571f0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:35.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient(hunting): my global_id is 54268 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d9c0574d0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1459285565 0 0) 0x7f3d80003620 con 0x7f3d9c13fd90 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.178:6789/0 conn(0x7f3d9c13f9c0 legacy=0x7f3d9c0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.102:6789/0 conn(0x7f3d9c1401f0 legacy=0x7f3d9c1a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 monclient: found mon.noname-b 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c0577b0 con 0x7f3d9c13fd90 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:25:35.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 5 monclient: authenticate success, global_id 84107 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d8c003110 con 0x7f3d9c13fd90 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: dump: 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:35.430 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d8c0033e0 con 0x7f3d9c13fd90 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 1 -- 10.20.193.102:0/768039641 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d8c004b50 con 0x7f3d9c13fd90 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:35.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9a7fc640 10 monclient: dump: 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:35.432 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 4 set_mon_vals no callback set 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: shutdown 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- 10.20.193.102:0/768039641 >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13fd90 legacy=0x7f3d9c183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- 10.20.193.102:0/768039641 shutdown_connections 2026-02-21T22:25:35.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- 10.20.193.102:0/768039641 >> 10.20.193.102:0/768039641 conn(0x7f3d9c05e3f0 msgr2=0x7f3d9c095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- 10.20.193.102:0/768039641 shutdown_connections 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- 10.20.193.102:0/768039641 wait complete. 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: build_initial_monmap 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: monmap: 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:35.434 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 Processor -- start 2026-02-21T22:25:35.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- start start 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: init 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-c con 0x7f3d9c13f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-b con 0x7f3d9c13fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient(hunting): picked mon.noname-a con 0x7f3d9c1401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c1335a0 con 0x7f3d9c1401f0 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c08fc10 con 0x7f3d9c13fd90 2026-02-21T22:25:35.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c08a5e0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:35.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3da08f5640 10 monclient(hunting): authenticate will time out at 1744.662899s 2026-02-21T22:25:35.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9b7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13f9c0 0x7f3d9c13eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59732/0 (socket says 10.20.193.102:59732) 2026-02-21T22:25:35.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.424+0000 7f3d9b7fe640 1 -- 10.20.193.102:0/3427200843 learned_addr learned my addr 10.20.193.102:0/3427200843 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:35.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d9bfff640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.102:6789/0 conn(0x7f3d9c1401f0 legacy=0x7f3d9c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d9b7fe640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13f9c0 legacy=0x7f3d9c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d9affd640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.178:6789/0 conn(0x7f3d9c13fd90 legacy=0x7f3d9c193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024232136 0 0) 0x7f3d9c08a5e0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): my global_id is 84110 2026-02-21T22:25:35.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d6c003620 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1814204600 0 0) 0x7f3d6c003620 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d9c08a5e0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d88003490 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): dump: 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:35.444 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:35.445 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939048111 0 0) 0x7f3d9c08fc10 con 0x7f3d9c13fd90 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient(hunting): my global_id is 54271 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d6c003620 con 0x7f3d9c13fd90 2026-02-21T22:25:35.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2812390658 0 0) 0x7f3d9c08a5e0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.178:6789/0 conn(0x7f3d9c13fd90 legacy=0x7f3d9c193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.102:6789/0 conn(0x7f3d9c1401f0 legacy=0x7f3d9c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 monclient: found mon.c 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c131240 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: _finish_auth 0 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 5 monclient: authenticate success, global_id 84110 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _renew_subs 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c1332c0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _renew_subs 2026-02-21T22:25:35.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f3d9c13f280 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d88003710 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d88003d60 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 10 monclient: dump: 2026-02-21T22:25:35.449 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:35.450 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d99ffb640 4 set_mon_vals no callback set 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f3d88004020 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c0008e0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(485..485 src has 112..485) ==== 3927+0+0 (unknown 2442892133 0 0) 0x7f3d8804c790 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.428+0000 7f3d9affd640 10 monclient: get_auth_request con 0x7f3d6c03d5c0 auth_method 0 2026-02-21T22:25:35.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.432+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f3d88017730 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.432+0000 7f3d77fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:35.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.432+0000 7f3d77fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:35.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:35.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3d9c1a6da0 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3d77fff640 1 -- 10.20.193.102:0/3427200843 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 3118854868) 0x7f3d88017010 con 0x7f3d9c13f9c0 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3d77fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3d77fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:35.545 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":36,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:35.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.178:6800/2549949439 conn(0x7f3d6c03d5c0 legacy=0x7f3d6c05da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 10 monclient: shutdown 2026-02-21T22:25:35.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:35.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 >> v1:10.20.193.102:6790/0 conn(0x7f3d9c13f9c0 legacy=0x7f3d9c13eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:35.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 shutdown_connections 2026-02-21T22:25:35.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 >> 10.20.193.102:0/3427200843 conn(0x7f3d9c05e3f0 msgr2=0x7f3d9c08bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:35.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 shutdown_connections 2026-02-21T22:25:35.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:35.544+0000 7f3da08f5640 1 -- 10.20.193.102:0/3427200843 wait complete. 2026-02-21T22:25:38.555 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:38.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:38.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 10 monclient: get_monmap_and_config 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 10 monclient: build_initial_monmap 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 10 monclient: monmap: 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:38.628339+0000 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:38.628339+0000 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:38.626 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 1 Processor -- start 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 1 -- start start 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.624+0000 7f53d919e640 10 monclient: init 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-c con 0x7f53d408cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-b con 0x7f53d408d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-a con 0x7f53d408b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:38.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d40890e0 con 0x7f53d408b8d0 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d4088ea0 con 0x7f53d408cd00 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d408aea0 con 0x7f53d408d7e0 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient(hunting): authenticate will time out at 1747.864419s 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d2575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f53d408d7e0 0x7f53d41840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59740/0 (socket says 10.20.193.102:59740) 2026-02-21T22:25:38.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d2575640 1 -- 10.20.193.102:0/2540716939 learned_addr learned my addr 10.20.193.102:0/2540716939 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:38.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d2575640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.102:6790/0 conn(0x7f53d408d7e0 legacy=0x7f53d41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d3577640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.102:6789/0 conn(0x7f53d408b8d0 legacy=0x7f53d41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059384588 0 0) 0x7f53d408aea0 con 0x7f53d408d7e0 2026-02-21T22:25:38.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient(hunting): _negotiate 2026-02-21T22:25:38.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient(hunting): my global_id is 84113 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d2d76640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.178:6789/0 conn(0x7f53d408cd00 legacy=0x7f53d408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c0003620 con 0x7f53d408d7e0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3913606769 0 0) 0x7f53c0003620 con 0x7f53d408d7e0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.178:6789/0 conn(0x7f53d408cd00 legacy=0x7f53d408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.102:6789/0 conn(0x7f53d408b8d0 legacy=0x7f53d41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: _finish_hunting 0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 monclient: found mon.noname-b 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d408a600 con 0x7f53d408d7e0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: _finish_auth 0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: _check_auth_tickets 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f53bc003110 con 0x7f53d408d7e0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: dump: 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:38.631 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 5 monclient: authenticate success, global_id 84113 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53bc0033e0 con 0x7f53d408d7e0 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 1 -- 10.20.193.102:0/2540716939 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f53bc004b50 con 0x7f53d408d7e0 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d1d74640 10 monclient: dump: 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:38.632 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 4 set_mon_vals no callback set 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 10 monclient: shutdown 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- 10.20.193.102:0/2540716939 >> v1:10.20.193.102:6790/0 conn(0x7f53d408d7e0 legacy=0x7f53d41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- 10.20.193.102:0/2540716939 shutdown_connections 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- 10.20.193.102:0/2540716939 >> 10.20.193.102:0/2540716939 conn(0x7f53d4091a50 msgr2=0x7f53d4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- 10.20.193.102:0/2540716939 shutdown_connections 2026-02-21T22:25:38.633 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.628+0000 7f53d919e640 1 -- 10.20.193.102:0/2540716939 wait complete. 2026-02-21T22:25:38.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: build_initial_monmap 2026-02-21T22:25:38.635 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: monmap: 2026-02-21T22:25:38.635 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:38.635 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 Processor -- start 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- start start 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: init 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-c con 0x7f53d408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-b con 0x7f53d408cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient(hunting): picked mon.noname-a con 0x7f53d408d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d407ff80 con 0x7f53d408d7e0 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d4088540 con 0x7f53d408cd00 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d4088a70 con 0x7f53d408b8d0 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:38.636 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient(hunting): authenticate will time out at 1747.869011s 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d2d76640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f53d408b8d0 0x7f53d4153360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59748/0 (socket says 10.20.193.102:59748) 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d2d76640 1 -- 10.20.193.102:0/4220650940 learned_addr learned my addr 10.20.193.102:0/4220650940 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d2d76640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.102:6790/0 conn(0x7f53d408b8d0 legacy=0x7f53d4153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d3577640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.102:6789/0 conn(0x7f53d408d7e0 legacy=0x7f53d41956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380089033 0 0) 0x7f53d4088a70 con 0x7f53d408b8d0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): my global_id is 84116 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53a0003620 con 0x7f53d408b8d0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d2575640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.178:6789/0 conn(0x7f53d408cd00 legacy=0x7f53d4083810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 431908499 0 0) 0x7f53a0003620 con 0x7f53d408b8d0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53d4088a70 con 0x7f53d408b8d0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f53c8003430 con 0x7f53d408b8d0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): dump: 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.637 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2046239797 0 0) 0x7f53d4088a70 con 0x7f53d408b8d0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.178:6789/0 conn(0x7f53d408cd00 legacy=0x7f53d4083810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.102:6789/0 conn(0x7f53d408d7e0 legacy=0x7f53d41956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 monclient: found mon.c 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d40822e0 con 0x7f53d408b8d0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 5 monclient: authenticate success, global_id 84116 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _renew_subs 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f53d4081a40 con 0x7f53d408b8d0 2026-02-21T22:25:38.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _renew_subs 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f53d4083f20 con 0x7f53d408b8d0 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.632+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53c8003ba0 con 0x7f53d408b8d0 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f53c8003f30 con 0x7f53d408b8d0 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: dump: 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d1573640 4 set_mon_vals no callback set 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d919e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d919e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:38.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d919e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d40008e0 con 0x7f53d408b8d0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f53c8004200 con 0x7f53d408b8d0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(485..485 src has 112..485) ==== 3927+0+0 (unknown 2442892133 0 0) 0x7f53c804c750 con 0x7f53d408b8d0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53d2575640 10 monclient: get_auth_request con 0x7f53a003d450 auth_method 0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f53c8017690 con 0x7f53d408b8d0 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:38.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.636+0000 7f53b37fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.752+0000 7f53d919e640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.752+0000 7f53d919e640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.752+0000 7f53d919e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.752+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f53d4090810 con 0x7f53d408b8d0 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53b37fe640 1 -- 10.20.193.102:0/4220650940 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 4040059700) 0x7f53c801c9b0 con 0x7f53d408b8d0 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53b37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53b37fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:38.755 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":36,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.178:6800/2549949439 conn(0x7f53a003d450 legacy=0x7f53a005d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 10 monclient: shutdown 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 >> v1:10.20.193.102:6790/0 conn(0x7f53d408b8d0 legacy=0x7f53d4153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 shutdown_connections 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 >> 10.20.193.102:0/4220650940 conn(0x7f53d4091a50 msgr2=0x7f53d405bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 shutdown_connections 2026-02-21T22:25:38.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:38.756+0000 7f53d919e640 1 -- 10.20.193.102:0/4220650940 wait complete. 2026-02-21T22:25:41.765 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:41.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: get_monmap_and_config 2026-02-21T22:25:41.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: build_initial_monmap 2026-02-21T22:25:41.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: monmap: 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:41.839297+0000 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:41.839297+0000 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:41.840 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 1 Processor -- start 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 1 -- start start 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: init 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-b con 0x7f92b413f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-a con 0x7f92b413fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-c con 0x7f92b41401f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4092490 con 0x7f92b413fd90 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b40571f0 con 0x7f92b41401f0 2026-02-21T22:25:41.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b40574d0 con 0x7f92b413f9c0 2026-02-21T22:25:41.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:41.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b9646640 10 monclient(hunting): authenticate will time out at 1751.075400s 2026-02-21T22:25:41.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f92b413f9c0 0x7f92b40906e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49902/0 (socket says 10.20.193.102:49902) 2026-02-21T22:25:41.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b3fff640 1 -- 10.20.193.102:0/3290602782 learned_addr learned my addr 10.20.193.102:0/3290602782 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:41.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.836+0000 7f92b37fe640 1 --1- 10.20.193.102:0/3290602782 >> v1:10.20.193.102:6789/0 conn(0x7f92b413fd90 0x7f92b4183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41514/0 (socket says 10.20.193.102:41514) 2026-02-21T22:25:41.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.840+0000 7f92b3fff640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.102:6790/0 conn(0x7f92b413f9c0 legacy=0x7f92b40906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:41.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.840+0000 7f92b37fe640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.102:6789/0 conn(0x7f92b413fd90 legacy=0x7f92b4183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:41.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:41.840+0000 7f92b8e45640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.178:6789/0 conn(0x7f92b41401f0 legacy=0x7f92b41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:43.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519441021 0 0) 0x7f92b40571f0 con 0x7f92b41401f0 2026-02-21T22:25:43.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:43.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:43.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:43.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): my global_id is 54274 2026-02-21T22:25:43.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9298003620 con 0x7f92b41401f0 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377811403 0 0) 0x7f92b4092490 con 0x7f92b413fd90 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient(hunting): my global_id is 34461 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b40571f0 con 0x7f92b413fd90 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2951829522 0 0) 0x7f92b40571f0 con 0x7f92b413fd90 2026-02-21T22:25:43.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.102:6790/0 conn(0x7f92b413f9c0 legacy=0x7f92b40906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.178:6789/0 conn(0x7f92b41401f0 legacy=0x7f92b41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 monclient: found mon.noname-a 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b40577b0 con 0x7f92b413fd90 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92a4003110 con 0x7f92b413fd90 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:43.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b2ffd640 10 monclient: dump: 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:43.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b9646640 5 monclient: authenticate success, global_id 34461 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.216+0000 7f92b9646640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92a40033e0 con 0x7f92b413fd90 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 1 -- 10.20.193.102:0/3290602782 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92a4004a90 con 0x7f92b413fd90 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b2ffd640 10 monclient: dump: 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.226 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 4 set_mon_vals no callback set 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: shutdown 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- 10.20.193.102:0/3290602782 >> v1:10.20.193.102:6789/0 conn(0x7f92b413fd90 legacy=0x7f92b4183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- 10.20.193.102:0/3290602782 shutdown_connections 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- 10.20.193.102:0/3290602782 >> 10.20.193.102:0/3290602782 conn(0x7f92b405e3f0 msgr2=0x7f92b4095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- 10.20.193.102:0/3290602782 shutdown_connections 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- 10.20.193.102:0/3290602782 wait complete. 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: build_initial_monmap 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: monmap: 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:43.228 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 Processor -- start 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- start start 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: init 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-a con 0x7f92b413f9c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-b con 0x7f92b413fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient(hunting): picked mon.noname-c con 0x7f92b41401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b40590d0 con 0x7f92b413f9c0 2026-02-21T22:25:43.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4133180 con 0x7f92b413fd90 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b4133460 con 0x7f92b41401f0 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b9646640 10 monclient(hunting): authenticate will time out at 1752.458694s 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b3fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f92b413f9c0 0x7f92b413ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41520/0 (socket says 10.20.193.102:41520) 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b8e45640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f92b41401f0 0x7f92b41955a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49914/0 (socket says 10.20.193.102:49914) 2026-02-21T22:25:43.230 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.220+0000 7f92b3fff640 1 -- 10.20.193.102:0/90552338 learned_addr learned my addr 10.20.193.102:0/90552338 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:43.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b8e45640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.102:6790/0 conn(0x7f92b41401f0 legacy=0x7f92b41955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:43.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b3fff640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.102:6789/0 conn(0x7f92b413f9c0 legacy=0x7f92b413ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:43.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b37fe640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.178:6789/0 conn(0x7f92b413fd90 legacy=0x7f92b4193e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:43.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825855309 0 0) 0x7f92b40574d0 con 0x7f92b413f9c0 2026-02-21T22:25:43.233 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): my global_id is 34464 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9284003620 con 0x7f92b413f9c0 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063853135 0 0) 0x7f92b4133180 con 0x7f92b413fd90 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): my global_id is 54277 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b40574d0 con 0x7f92b413fd90 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2708981996 0 0) 0x7f9284003620 con 0x7f92b413f9c0 2026-02-21T22:25:43.234 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92b4133180 con 0x7f92b413f9c0 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92a0003330 con 0x7f92b413f9c0 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): dump: 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:43.235 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1269159216 0 0) 0x7f92b4133180 con 0x7f92b413f9c0 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.102:6790/0 conn(0x7f92b41401f0 legacy=0x7f92b41955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.178:6789/0 conn(0x7f92b413fd90 legacy=0x7f92b4193e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 monclient: found mon.a 2026-02-21T22:25:43.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b4131100 con 0x7f92b413f9c0 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: _finish_auth 0 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:43.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 5 monclient: authenticate success, global_id 34464 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _renew_subs 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41328e0 con 0x7f92b413f9c0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _renew_subs 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f92b4152d80 con 0x7f92b413f9c0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92a0003510 con 0x7f92b413f9c0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92a0004d40 con 0x7f92b413f9c0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:43.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 10 monclient: dump: 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:43.239 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b27fc640 4 set_mon_vals no callback set 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b40008e0 con 0x7f92b413f9c0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f92a0004fd0 con 0x7f92b413f9c0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(485..485 src has 112..485) ==== 3927+0+0 (unknown 2442892133 0 0) 0x7f92a004c650 con 0x7f92b413f9c0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.224+0000 7f92b37fe640 10 monclient: get_auth_request con 0x7f928403d310 auth_method 0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.228+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f92b40590d0 con 0x7f92b413f9c0 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.228+0000 7f92b0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:43.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.228+0000 7f92b0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:43.268 INFO:tasks.rados.rados.0.trial178.stdout:2355: done (1 left) 2026-02-21T22:25:43.269 INFO:tasks.rados.rados.0.trial178.stdout:2356: done (0 left) 2026-02-21T22:25:43.269 INFO:tasks.rados.rados.0.trial178.stdout:2357: read oid 13 snap -1 2026-02-21T22:25:43.269 INFO:tasks.rados.rados.0.trial178.stdout:2357: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:25:43.269 INFO:tasks.rados.rados.0.trial178.stdout:2358: snap_create 2026-02-21T22:25:43.275 INFO:tasks.rados.rados.0.trial178.stdout:2357: expect (ObjNum 545 snap 153 seq_num 545) 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.344+0000 7f92b9646640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.344+0000 7f92b9646640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.344+0000 7f92b9646640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.344+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f92b41a6da0 con 0x7f92b413f9c0 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.344+0000 7f92b0ff9640 1 -- 10.20.193.102:0/90552338 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 3843392699) 0x7f92a0017590 con 0x7f92b413f9c0 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:43.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:43.348 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":38,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.178:6800/2549949439 conn(0x7f928403d310 legacy=0x7f928405d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 10 monclient: shutdown 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 >> v1:10.20.193.102:6789/0 conn(0x7f92b413f9c0 legacy=0x7f92b413ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 shutdown_connections 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 >> 10.20.193.102:0/90552338 conn(0x7f92b405e3f0 msgr2=0x7f92b4088de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 shutdown_connections 2026-02-21T22:25:43.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:43.348+0000 7f92b9646640 1 -- 10.20.193.102:0/90552338 wait complete. 2026-02-21T22:25:43.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2354780 2026-02-21T22:25:43.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:43.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2357: done (1 left) 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2358: done (0 left) 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2359: read oid 4 snap 235 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2359: expect (ObjNum 816 snap 230 seq_num 816) 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2360: read oid 38 snap -1 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2360: expect deleted 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2361: write oid 32 current snap is 238 2026-02-21T22:25:44.270 INFO:tasks.rados.rados.0.trial178.stdout:2361: seq_num 828 ranges {716196=472296,1769561=646611,3121885=793109} 2026-02-21T22:25:44.278 INFO:tasks.rados.rados.0.trial178.stdout:2361: writing trial17818531-32 from 716196 to 1188492 tid 1 2026-02-21T22:25:44.286 INFO:tasks.rados.rados.0.trial178.stdout:2361: writing trial17818531-32 from 1769561 to 2416172 tid 2 2026-02-21T22:25:44.296 INFO:tasks.rados.rados.0.trial178.stdout:2361: writing trial17818531-32 from 3121885 to 3914994 tid 3 2026-02-21T22:25:44.296 INFO:tasks.rados.rados.0.trial178.stdout:2359: expect (ObjNum 816 snap 230 seq_num 816) 2026-02-21T22:25:44.297 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:44.331 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:44.399 INFO:tasks.rados.rados.0.trial178.stdout:2361: finishing write tid 1 to trial17818531-32 2026-02-21T22:25:44.399 INFO:tasks.rados.rados.0.trial178.stdout:2361: finishing write tid 2 to trial17818531-32 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: finishing write tid 3 to trial17818531-32 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: finishing write tid 4 to trial17818531-32 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: finishing write tid 5 to trial17818531-32 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: oid 32 updating version 0 to 1149 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: oid 32 updating version 1149 to 1152 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: oid 32 version 1152 is already newer than 1150 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: oid 32 version 1152 is already newer than 1151 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1152 (ObjNum 828 snap 238 seq_num 828) dirty exists 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2361: left oid 32 (ObjNum 828 snap 238 seq_num 828) 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2359: done (2 left) 2026-02-21T22:25:44.400 INFO:tasks.rados.rados.0.trial178.stdout:2360: done (1 left) 2026-02-21T22:25:44.401 INFO:tasks.rados.rados.0.trial178.stdout:2361: done (0 left) 2026-02-21T22:25:44.401 INFO:tasks.rados.rados.0.trial178.stdout:2362: snap_remove snap 228 2026-02-21T22:25:45.285 INFO:tasks.rados.rados.0.trial178.stdout:2362: done (0 left) 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2363: read oid 43 snap -1 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2363: expect (ObjNum 823 snap 232 seq_num 823) 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2364: read oid 25 snap 235 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2364: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2365: read oid 34 snap -1 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2365: expect deleted 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2366: rollback oid 33 current snap is 238 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 229 2026-02-21T22:25:45.286 INFO:tasks.rados.rados.0.trial178.stdout:2367: delete oid 45 current snap is 238 2026-02-21T22:25:45.291 INFO:tasks.rados.rados.0.trial178.stdout:2364: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:25:45.291 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:45.377 INFO:tasks.rados.rados.0.trial178.stdout:2366: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:25:45.377 INFO:tasks.rados.rados.0.trial178.stdout:2366: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:25:45.377 INFO:tasks.rados.rados.0.trial178.stdout:2366: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:25:45.377 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 838 (ObjNum 807 snap 226 seq_num 807) dirty exists 2026-02-21T22:25:45.378 INFO:tasks.rados.rados.0.trial178.stdout:2363: expect (ObjNum 823 snap 232 seq_num 823) 2026-02-21T22:25:45.383 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1095299 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2363: done (4 left) 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2364: done (3 left) 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2365: done (2 left) 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2366: done (1 left) 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2367: done (0 left) 2026-02-21T22:25:45.495 INFO:tasks.rados.rados.0.trial178.stdout:2368: snap_remove snap 234 2026-02-21T22:25:46.360 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: get_monmap_and_config 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: monmap: 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:46.423113+0000 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:46.423113+0000 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:46.421 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 1 Processor -- start 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 1 -- start start 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: init 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient(hunting): picked mon.noname-c con 0x7f497c084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient(hunting): picked mon.noname-a con 0x7f497c086e90 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient(hunting): picked mon.noname-b con 0x7f497c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.422 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c086a70 con 0x7f497c086e90 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c08b130 con 0x7f497c084860 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c085340 con 0x7f497c183350 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4983119640 10 monclient(hunting): authenticate will time out at 1755.659241s 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4973fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f497c086e90 0x7f497c181c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41528/0 (socket says 10.20.193.102:41528) 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f498168f640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 0x7f497c1a3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49926/0 (socket says 10.20.193.102:49926) 2026-02-21T22:25:46.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.419+0000 7f4973fff640 1 -- 10.20.193.102:0/2536908259 learned_addr learned my addr 10.20.193.102:0/2536908259 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f498168f640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 legacy=0x7f497c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4973fff640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.102:6789/0 conn(0x7f497c086e90 legacy=0x7f497c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.424 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4980e8e640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.178:6789/0 conn(0x7f497c084860 legacy=0x7f497c084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205526303 0 0) 0x7f497c085340 con 0x7f497c183350 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): my global_id is 84119 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4968003620 con 0x7f497c183350 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190641846 0 0) 0x7f497c086a70 con 0x7f497c086e90 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient(hunting): my global_id is 34467 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f497c085340 con 0x7f497c086e90 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3777121759 0 0) 0x7f4968003620 con 0x7f497c183350 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.178:6789/0 conn(0x7f497c084860 legacy=0x7f497c084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.102:6789/0 conn(0x7f497c086e90 legacy=0x7f497c181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 monclient: found mon.noname-b 2026-02-21T22:25:46.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c086790 con 0x7f497c183350 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4978004610 con 0x7f497c183350 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: dump: 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 5 monclient: authenticate success, global_id 84119 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f49780048e0 con 0x7f497c183350 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 1 -- 10.20.193.102:0/2536908259 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4978005fa0 con 0x7f497c183350 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f49737fe640 10 monclient: dump: 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.426 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:46.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 4 set_mon_vals no callback set 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: shutdown 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- 10.20.193.102:0/2536908259 >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 legacy=0x7f497c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- 10.20.193.102:0/2536908259 shutdown_connections 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- 10.20.193.102:0/2536908259 >> 10.20.193.102:0/2536908259 conn(0x7f497c091a50 msgr2=0x7f497c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- 10.20.193.102:0/2536908259 shutdown_connections 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- 10.20.193.102:0/2536908259 wait complete. 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: monmap: 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 Processor -- start 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- start start 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: init 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient(hunting): picked mon.noname-a con 0x7f497c084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient(hunting): picked mon.noname-b con 0x7f497c086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient(hunting): picked mon.noname-c con 0x7f497c183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c088a70 con 0x7f497c084860 2026-02-21T22:25:46.428 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c07ec60 con 0x7f497c086e90 2026-02-21T22:25:46.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c080fc0 con 0x7f497c183350 2026-02-21T22:25:46.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f4983119640 10 monclient(hunting): authenticate will time out at 1755.663575s 2026-02-21T22:25:46.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f498168f640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 0x7f497c1745e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49928/0 (socket says 10.20.193.102:49928) 2026-02-21T22:25:46.429 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.423+0000 7f498168f640 1 -- 10.20.193.102:0/2478453489 learned_addr learned my addr 10.20.193.102:0/2478453489 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4980e8e640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.102:6789/0 conn(0x7f497c084860 legacy=0x7f497c083500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f498168f640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 legacy=0x7f497c1745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628101901 0 0) 0x7f497c08b130 con 0x7f497c084860 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): my global_id is 34470 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f494c003620 con 0x7f497c084860 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859718650 0 0) 0x7f497c080fc0 con 0x7f497c183350 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): my global_id is 84122 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f497c08b130 con 0x7f497c183350 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4973fff640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.178:6789/0 conn(0x7f497c086e90 legacy=0x7f497c172ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2400285579 0 0) 0x7f497c08b130 con 0x7f497c183350 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f497c080fc0 con 0x7f497c183350 2026-02-21T22:25:46.430 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4978005fa0 con 0x7f497c183350 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): dump: 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3985072491 0 0) 0x7f494c003620 con 0x7f497c084860 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f497c08b130 con 0x7f497c084860 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f496c003380 con 0x7f497c084860 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient(hunting): dump: 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.431 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2451633921 0 0) 0x7f497c080fc0 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.178:6789/0 conn(0x7f497c086e90 legacy=0x7f497c172ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.102:6789/0 conn(0x7f497c084860 legacy=0x7f497c083500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 monclient: found mon.c 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c080720 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 5 monclient: authenticate success, global_id 84122 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _renew_subs 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f497c131dc0 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _renew_subs 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f497c07f990 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f49780048e0 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f49780066f0 con 0x7f497c183350 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4972ffd640 4 set_mon_vals no callback set 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 10 monclient: dump: 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.432 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c132040 con 0x7f497c183350 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4978012d80 con 0x7f497c183350 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(488..488 src has 112..488) ==== 3943+0+0 (unknown 3322414675 0 0) 0x7f497804da70 con 0x7f497c183350 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.427+0000 7f4980e8e640 10 monclient: get_auth_request con 0x7f494c03d720 auth_method 0 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.431+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f4978018a80 con 0x7f497c183350 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.431+0000 7f49717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.431+0000 7f49717fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f497c058a30 con 0x7f497c183350 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f49717fa640 1 -- 10.20.193.102:0/2478453489 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2505654456) 0x7f49780202a0 con 0x7f497c183350 2026-02-21T22:25:46.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f49717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:46.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f49717fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:46.550 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:46.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.178:6800/2549949439 conn(0x7f494c03d720 legacy=0x7f494c05dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 10 monclient: shutdown 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 >> v1:10.20.193.102:6790/0 conn(0x7f497c183350 legacy=0x7f497c1745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 shutdown_connections 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 >> 10.20.193.102:0/2478453489 conn(0x7f497c091a50 msgr2=0x7f497c087c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 shutdown_connections 2026-02-21T22:25:46.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.547+0000 7f4983119640 1 -- 10.20.193.102:0/2478453489 wait complete. 2026-02-21T22:25:46.553 INFO:tasks.rados.rados.0.trial178.stdout:2368: done (0 left) 2026-02-21T22:25:46.553 INFO:tasks.rados.rados.0.trial178.stdout:2369: write oid 13 current snap is 238 2026-02-21T22:25:46.553 INFO:tasks.rados.rados.0.trial178.stdout:2369: seq_num 830 ranges {749810=739147,2002878=650280,3293084=415653} 2026-02-21T22:25:46.560 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:25:46.560 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:25:46.561 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:25:46.561 INFO:tasks.rados.rados.0.trial178.stdout:2369: writing trial17818531-13 from 749810 to 1488957 tid 1 2026-02-21T22:25:46.569 INFO:tasks.rados.rados.0.trial178.stdout:2369: writing trial17818531-13 from 2002878 to 2653158 tid 2 2026-02-21T22:25:46.576 INFO:tasks.rados.rados.0.trial178.stdout:2369: writing trial17818531-13 from 3293084 to 3708737 tid 3 2026-02-21T22:25:46.576 INFO:tasks.rados.rados.0.trial178.stdout:2370: rollback oid 19 current snap is 238 2026-02-21T22:25:46.576 INFO:tasks.rados.rados.0.trial178.stdout:2369: finishing write tid 1 to trial17818531-13 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2369: finishing write tid 2 to trial17818531-13 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 233 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2371: rollback oid 15 current snap is 238 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 227 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2372: read oid 41 snap -1 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2372: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2373: read oid 12 snap -1 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2373: expect deleted 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2374: read oid 7 snap -1 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2374: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:25:46.577 INFO:tasks.rados.rados.0.trial178.stdout:2375: delete oid 39 current snap is 238 2026-02-21T22:25:46.578 INFO:tasks.rados.rados.0.trial178.stdout:2369: finishing write tid 3 to trial17818531-13 2026-02-21T22:25:46.578 INFO:tasks.rados.rados.0.trial178.stdout:2369: finishing write tid 4 to trial17818531-13 2026-02-21T22:25:46.578 INFO:tasks.rados.rados.0.trial178.stdout:2369: finishing write tid 5 to trial17818531-13 2026-02-21T22:25:46.578 INFO:tasks.rados.rados.0.trial178.stdout:2369: oid 13 updating version 0 to 1156 2026-02-21T22:25:46.579 INFO:tasks.rados.rados.0.trial178.stdout:2369: oid 13 version 1156 is already newer than 1155 2026-02-21T22:25:46.579 INFO:tasks.rados.rados.0.trial178.stdout:2369: oid 13 version 1156 is already newer than 1154 2026-02-21T22:25:46.579 INFO:tasks.rados.rados.0.trial178.stdout:2369: oid 13 version 1156 is already newer than 1153 2026-02-21T22:25:46.579 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1156 (ObjNum 830 snap 238 seq_num 830) dirty exists 2026-02-21T22:25:46.579 INFO:tasks.rados.rados.0.trial178.stdout:2369: left oid 13 (ObjNum 830 snap 238 seq_num 830) 2026-02-21T22:25:46.585 INFO:tasks.rados.rados.0.trial178.stdout:2372: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:25:46.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: get_monmap_and_config 2026-02-21T22:25:46.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: monmap: 2026-02-21T22:25:46.627 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.627 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:46.626941+0000 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:46.626941+0000 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 1 Processor -- start 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 1 -- start start 2026-02-21T22:25:46.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: init 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-a con 0x7fe76408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-b con 0x7fe76408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-c con 0x7fe76408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764086a10 con 0x7fe76408ce00 2026-02-21T22:25:46.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76408b130 con 0x7fe76408d8e0 2026-02-21T22:25:46.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76408bd60 con 0x7fe76408b7f0 2026-02-21T22:25:46.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe7697a6640 10 monclient(hunting): authenticate will time out at 1755.862853s 2026-02-21T22:25:46.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe762ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fe76408ce00 0x7fe76408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41552/0 (socket says 10.20.193.102:41552) 2026-02-21T22:25:46.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.623+0000 7fe762ffd640 1 -- 10.20.193.102:0/1038011112 learned_addr learned my addr 10.20.193.102:0/1038011112 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe762ffd640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.102:6789/0 conn(0x7fe76408ce00 legacy=0x7fe76408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7627fc640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.102:6790/0 conn(0x7fe76408b7f0 legacy=0x7fe764181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7637fe640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.178:6789/0 conn(0x7fe76408d8e0 legacy=0x7fe7641a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340771207 0 0) 0x7fe764086a10 con 0x7fe76408ce00 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.637 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): my global_id is 34473 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe750003620 con 0x7fe76408ce00 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002487597 0 0) 0x7fe76408b130 con 0x7fe76408d8e0 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient(hunting): my global_id is 54280 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe764086a10 con 0x7fe76408d8e0 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 317632391 0 0) 0x7fe750003620 con 0x7fe76408ce00 2026-02-21T22:25:46.638 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.102:6790/0 conn(0x7fe76408b7f0 legacy=0x7fe764181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.178:6789/0 conn(0x7fe76408d8e0 legacy=0x7fe7641a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 monclient: found mon.noname-a 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76408b370 con 0x7fe76408ce00 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 5 monclient: authenticate success, global_id 34473 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe758003110 con 0x7fe76408ce00 2026-02-21T22:25:46.639 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: dump: 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.640 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe7580033e0 con 0x7fe76408ce00 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 4 set_mon_vals no callback set 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 1 -- 10.20.193.102:0/1038011112 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe758004b50 con 0x7fe76408ce00 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe761ffb640 10 monclient: dump: 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.641 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.642 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: shutdown 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- 10.20.193.102:0/1038011112 >> v1:10.20.193.102:6789/0 conn(0x7fe76408ce00 legacy=0x7fe76408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- 10.20.193.102:0/1038011112 shutdown_connections 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- 10.20.193.102:0/1038011112 >> 10.20.193.102:0/1038011112 conn(0x7fe764091a50 msgr2=0x7fe764091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- 10.20.193.102:0/1038011112 shutdown_connections 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- 10.20.193.102:0/1038011112 wait complete. 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: monmap: 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.643 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 Processor -- start 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- start start 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: init 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-a con 0x7fe76408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-c con 0x7fe76408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient(hunting): picked mon.noname-b con 0x7fe76408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76405e150 con 0x7fe76408b7f0 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764088a70 con 0x7fe76408d8e0 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe764153880 con 0x7fe76408ce00 2026-02-21T22:25:46.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe7697a6640 10 monclient(hunting): authenticate will time out at 1755.866891s 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe762ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fe76408b7f0 0x7fe764084960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41568/0 (socket says 10.20.193.102:41568) 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.627+0000 7fe762ffd640 1 -- 10.20.193.102:0/3332605183 learned_addr learned my addr 10.20.193.102:0/3332605183 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe762ffd640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe76408b7f0 legacy=0x7fe764084960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7627fc640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6790/0 conn(0x7fe76408ce00 legacy=0x7fe764194550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145338580 0 0) 0x7fe76405e150 con 0x7fe76408b7f0 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): my global_id is 34476 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe730003620 con 0x7fe76408b7f0 2026-02-21T22:25:46.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7637fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.178:6789/0 conn(0x7fe76408d8e0 legacy=0x7fe764195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3193680848 0 0) 0x7fe730003620 con 0x7fe76408b7f0 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe76405e150 con 0x7fe76408b7f0 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe758002b70 con 0x7fe76408b7f0 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): dump: 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.647 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 821613240 0 0) 0x7fe76405e150 con 0x7fe76408b7f0 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6790/0 conn(0x7fe76408ce00 legacy=0x7fe764194550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.178:6789/0 conn(0x7fe76408d8e0 legacy=0x7fe764195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 monclient: found mon.a 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640810d0 con 0x7fe76408b7f0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 5 monclient: authenticate success, global_id 34476 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 10 monclient: _renew_subs 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe764083430 con 0x7fe76408b7f0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 10 monclient: _renew_subs 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe764085070 con 0x7fe76408b7f0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe758003380 con 0x7fe76408b7f0 2026-02-21T22:25:46.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe7580052a0 con 0x7fe76408b7f0 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 10 monclient: dump: 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.650 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7617fa640 4 set_mon_vals no callback set 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe7641535a0 con 0x7fe7641531d0 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fe758011930 con 0x7fe76408b7f0 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(489..489 src has 112..489) ==== 3959+0+0 (unknown 318611910 0 0) 0x7fe75804c700 con 0x7fe76408b7f0 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.631+0000 7fe7637fe640 10 monclient: get_auth_request con 0x7fe73003d500 auth_method 0 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7627fc640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe7641531d0 legacy=0x7fe76408a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688677221 0 0) 0x7fe7641535a0 con 0x7fe7641531d0 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient(hunting): my global_id is 34479 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe76405e150 con 0x7fe7641531d0 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 929719999 0 0) 0x7fe76405e150 con 0x7fe7641531d0 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7641535a0 con 0x7fe7641531d0 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe74c0030f0 con 0x7fe7641531d0 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient: dump: 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.652 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.653 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1152686604 0 0) 0x7fe7641535a0 con 0x7fe7641531d0 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7640008e0 con 0x7fe7641531d0 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fe74c003510 con 0x7fe7641531d0 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:46.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe7641531d0 legacy=0x7fe76408a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.635+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe7641531d0 legacy=0x7fe76408a530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:46.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:46.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe764090810 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7627fc640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe76404d280 legacy=0x7fe7641983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740272123 0 0) 0x7fe764090810 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient(hunting): my global_id is 34482 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7641535a0 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1995477064 0 0) 0x7fe7641535a0 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe764090810 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe74c003ab0 con 0x7fe76404d280 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient: dump: 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1051090387 0 0) 0x7fe764090810 con 0x7fe76404d280 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe7640909e0 con 0x7fe76404d280 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 1699496738) 0x7fe74c0058c0 con 0x7fe76404d280 2026-02-21T22:25:46.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe76404d280 legacy=0x7fe7641983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7437fe640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe76404d280 legacy=0x7fe7641983a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:46.659 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 495508, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:46.660 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:46.661 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:46.662 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.663 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:46.664 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:46.665 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:46.666 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:46.667 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:46.668 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:46.669 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:46.670 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "num": 7 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:46.671 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:46.672 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:46.672 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:46.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.178:6800/2549949439 conn(0x7fe73003d500 legacy=0x7fe73005d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 10 monclient: shutdown 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 >> v1:10.20.193.102:6789/0 conn(0x7fe76408b7f0 legacy=0x7fe764084960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7637fe640 1 -- 10.20.193.102:0/3332605183 reap_dead start 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7637fe640 1 -- 10.20.193.102:0/3332605183 reap_dead start 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 shutdown_connections 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 >> 10.20.193.102:0/3332605183 conn(0x7fe764091a50 msgr2=0x7fe76405d500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 shutdown_connections 2026-02-21T22:25:46.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.639+0000 7fe7697a6640 1 -- 10.20.193.102:0/3332605183 wait complete. 2026-02-21T22:25:46.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:46.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:25:46.675 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:25:46.676 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2825671 2026-02-21T22:25:46.676 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:46.695 INFO:tasks.rados.rados.0.trial178.stdout:2371: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:25:46.695 INFO:tasks.rados.rados.0.trial178.stdout:2371: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2369: done (6 left) 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2372: done (5 left) 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2373: done (4 left) 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2375: done (3 left) 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2376: read oid 13 snap -1 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2376: expect (ObjNum 830 snap 238 seq_num 830) 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2377: write oid 40 current snap is 238 2026-02-21T22:25:46.696 INFO:tasks.rados.rados.0.trial178.stdout:2377: seq_num 832 ranges {783424=489645,2003489=653949,3231577=554550} 2026-02-21T22:25:46.704 INFO:tasks.rados.rados.0.trial178.stdout:2377: writing trial17818531-40 from 783424 to 1273069 tid 1 2026-02-21T22:25:46.713 INFO:tasks.rados.rados.0.trial178.stdout:2377: writing trial17818531-40 from 2003489 to 2657438 tid 2 2026-02-21T22:25:46.721 INFO:tasks.rados.rados.0.trial178.stdout:2377: writing trial17818531-40 from 3231577 to 3786127 tid 3 2026-02-21T22:25:46.721 INFO:tasks.rados.rados.0.trial178.stdout:2378: snap_remove snap 226 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: get_monmap_and_config 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: monmap: 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:46.738682+0000 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:46.738682+0000 2026-02-21T22:25:46.737 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.738 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.738 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.738 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:46.738 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:46.738 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 1 Processor -- start 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 1 -- start start 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: init 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-b con 0x7f6c98088d80 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-a con 0x7f6c9808a1d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-c con 0x7f6c98089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c98085920 con 0x7f6c9808a1d0 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9805e780 con 0x7f6c98089860 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c98086330 con 0x7f6c98088d80 2026-02-21T22:25:46.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9eab4640 10 monclient(hunting): authenticate will time out at 1755.974803s 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9c829640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f6c98088d80 0x7f6c98089150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49940/0 (socket says 10.20.193.102:49940) 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.735+0000 7f6c9c829640 1 -- 10.20.193.102:0/1212057219 learned_addr learned my addr 10.20.193.102:0/1212057219 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9c829640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.102:6790/0 conn(0x7f6c98088d80 legacy=0x7f6c98089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8ffff640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.102:6789/0 conn(0x7f6c9808a1d0 legacy=0x7f6c98180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9d02a640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.178:6789/0 conn(0x7f6c98089860 legacy=0x7f6c981a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2734384176 0 0) 0x7f6c98085920 con 0x7f6c9808a1d0 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): my global_id is 34485 2026-02-21T22:25:46.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c84003620 con 0x7f6c9808a1d0 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1656963904 0 0) 0x7f6c9805e780 con 0x7f6c98089860 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient(hunting): my global_id is 54286 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c98085920 con 0x7f6c98089860 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3107902945 0 0) 0x7f6c84003620 con 0x7f6c9808a1d0 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.102:6790/0 conn(0x7f6c98088d80 legacy=0x7f6c98089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.178:6789/0 conn(0x7f6c98089860 legacy=0x7f6c981a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 monclient: found mon.noname-a 2026-02-21T22:25:46.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9808c6b0 con 0x7f6c9808a1d0 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 5 monclient: authenticate success, global_id 34485 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c80003160 con 0x7f6c9808a1d0 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: dump: 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.747 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6c800034a0 con 0x7f6c9808a1d0 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 1 -- 10.20.193.102:0/1212057219 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c800038c0 con 0x7f6c9808a1d0 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c8f7fe640 10 monclient: dump: 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.749 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 4 set_mon_vals no callback set 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: shutdown 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:46.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- 10.20.193.102:0/1212057219 >> v1:10.20.193.102:6789/0 conn(0x7f6c9808a1d0 legacy=0x7f6c98180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- 10.20.193.102:0/1212057219 shutdown_connections 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- 10.20.193.102:0/1212057219 >> 10.20.193.102:0/1212057219 conn(0x7f6c98091a50 msgr2=0x7f6c98091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- 10.20.193.102:0/1212057219 shutdown_connections 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- 10.20.193.102:0/1212057219 wait complete. 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: build_initial_monmap 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: monmap: 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:46.751 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 Processor -- start 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- start start 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: init 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-a con 0x7f6c98088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-b con 0x7f6c98089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient(hunting): picked mon.noname-c con 0x7f6c9808a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c980821a0 con 0x7f6c98088d80 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c98083110 con 0x7f6c98089860 2026-02-21T22:25:46.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c980833f0 con 0x7f6c9808a1d0 2026-02-21T22:25:46.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:46.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9eab4640 10 monclient(hunting): authenticate will time out at 1755.979338s 2026-02-21T22:25:46.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9c829640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f6c98088d80 0x7f6c9812fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41602/0 (socket says 10.20.193.102:41602) 2026-02-21T22:25:46.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9c829640 1 -- 10.20.193.102:0/4086310137 learned_addr learned my addr 10.20.193.102:0/4086310137 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:46.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.739+0000 7f6c9d02a640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f6c9808a1d0 0x7f6c9811e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49948/0 (socket says 10.20.193.102:49948) 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9d02a640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9808a1d0 legacy=0x7f6c9811e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9c829640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6789/0 conn(0x7f6c98088d80 legacy=0x7f6c9812fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8ffff640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.178:6789/0 conn(0x7f6c98089860 legacy=0x7f6c98130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879359757 0 0) 0x7f6c980821a0 con 0x7f6c98088d80 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): my global_id is 34488 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c68003620 con 0x7f6c98088d80 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257510003 0 0) 0x7f6c98083110 con 0x7f6c98089860 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:46.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): my global_id is 54289 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c980821a0 con 0x7f6c98089860 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1109290069 0 0) 0x7f6c68003620 con 0x7f6c98088d80 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c98083110 con 0x7f6c98088d80 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c88003330 con 0x7f6c98088d80 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): dump: 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.761 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2768797286 0 0) 0x7f6c980821a0 con 0x7f6c98089860 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c68003620 con 0x7f6c98089860 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c80002bc0 con 0x7f6c98089860 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient(hunting): dump: 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.762 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2942014650 0 0) 0x7f6c98083110 con 0x7f6c98088d80 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9808a1d0 legacy=0x7f6c9811e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.178:6789/0 conn(0x7f6c98089860 legacy=0x7f6c98130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 monclient: found mon.a 2026-02-21T22:25:46.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c98130c70 con 0x7f6c98088d80 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 5 monclient: authenticate success, global_id 34488 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 10 monclient: _renew_subs 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6c98081ed0 con 0x7f6c98088d80 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 10 monclient: _renew_subs 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6c98088a70 con 0x7f6c98088d80 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6c88003550 con 0x7f6c98088d80 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:46.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c88004d70 con 0x7f6c98088d80 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 10 monclient: dump: 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8effd640 4 set_mon_vals no callback set 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:46.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f6c88005040 con 0x7f6c98088d80 2026-02-21T22:25:46.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6c980837c0 con 0x7f6c9808cf90 2026-02-21T22:25:46.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(489..489 src has 112..489) ==== 3959+0+0 (unknown 318611910 0 0) 0x7f6c8804c750 con 0x7f6c98088d80 2026-02-21T22:25:46.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.743+0000 7f6c9d02a640 10 monclient: get_auth_request con 0x7f6c6803d610 auth_method 0 2026-02-21T22:25:46.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:46.747+0000 7f6c8ffff640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9808cf90 legacy=0x7f6c9807b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441420610 0 0) 0x7f6c980837c0 con 0x7f6c9808cf90 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient(hunting): my global_id is 84125 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c98083110 con 0x7f6c9808cf90 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1795495012 0 0) 0x7f6c98083110 con 0x7f6c9808cf90 2026-02-21T22:25:47.151 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c980837c0 con 0x7f6c9808cf90 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c80003de0 con 0x7f6c9808cf90 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient: dump: 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.152 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3766204921 0 0) 0x7f6c980837c0 con 0x7f6c9808cf90 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c98085300 con 0x7f6c9808cf90 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f6c800058c0 con 0x7f6c9808cf90 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9808cf90 legacy=0x7f6c9807b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.147+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9808cf90 legacy=0x7f6c9807b380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:47.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.151+0000 7f6c9eab4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:47.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.151+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6c98090810 con 0x7f6c9807ba90 2026-02-21T22:25:47.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8ffff640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9807ba90 legacy=0x7f6c981286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359106779 0 0) 0x7f6c98090810 con 0x7f6c9807ba90 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient(hunting): my global_id is 84128 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c980837c0 con 0x7f6c9807ba90 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3151537066 0 0) 0x7f6c980837c0 con 0x7f6c9807ba90 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c98090810 con 0x7f6c9807ba90 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6c800041c0 con 0x7f6c9807ba90 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient: dump: 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4276865510 0 0) 0x7f6c98090810 con 0x7f6c9807ba90 2026-02-21T22:25:47.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6c980909e0 con 0x7f6c9807ba90 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 2537333858) 0x7f6c80005b40 con 0x7f6c9807ba90 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9807ba90 legacy=0x7f6c981286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c8d7fa640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6790/0 conn(0x7f6c9807ba90 legacy=0x7f6c981286e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:47.160 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 29975, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:25:47.161 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.162 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.163 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:47.164 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:47.165 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.166 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.167 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:47.168 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.169 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.170 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:47.171 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.172 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.173 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.173 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:47.173 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:47.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.178:6800/2549949439 conn(0x7f6c6803d610 legacy=0x7f6c6805dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 10 monclient: shutdown 2026-02-21T22:25:47.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 >> v1:10.20.193.102:6789/0 conn(0x7f6c98088d80 legacy=0x7f6c9812fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9d02a640 1 -- 10.20.193.102:0/4086310137 reap_dead start 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9d02a640 1 -- 10.20.193.102:0/4086310137 reap_dead start 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 shutdown_connections 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 >> 10.20.193.102:0/4086310137 conn(0x7f6c98091a50 msgr2=0x7f6c9805cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 shutdown_connections 2026-02-21T22:25:47.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.155+0000 7f6c9eab4640 1 -- 10.20.193.102:0/4086310137 wait complete. 2026-02-21T22:25:47.176 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:47.176 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:25:47.176 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:25:47.177 INFO:tasks.rados.rados.0.trial178.stdout:2378: done (5 left) 2026-02-21T22:25:47.177 INFO:tasks.rados.rados.0.trial178.stdout:2379: snap_create 2026-02-21T22:25:47.177 INFO:tasks.rados.rados.0.trial178.stdout:2371: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:25:47.177 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 853 (ObjNum 745 snap 209 seq_num 745) dirty exists 2026-02-21T22:25:47.178 INFO:tasks.rados.rados.0.trial178.stdout:2370: finishing rollback tid 0 to trial17818531-19 2026-02-21T22:25:47.178 INFO:tasks.rados.rados.0.trial178.stdout:2370: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:25:47.178 INFO:tasks.rados.rados.0.trial178.stdout:2370: finishing rollback tid 2 to trial17818531-19 2026-02-21T22:25:47.178 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1310 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:25:47.178 INFO:tasks.rados.rados.0.trial178.stdout:2374: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:25:47.187 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 107577 2026-02-21T22:25:47.240 INFO:tasks.rados.rados.0.trial178.stdout:2376: expect (ObjNum 830 snap 238 seq_num 830) 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: get_monmap_and_config 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: build_initial_monmap 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: monmap: 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:47.242598+0000 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:47.242598+0000 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 1 Processor -- start 2026-02-21T22:25:47.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 1 -- start start 2026-02-21T22:25:47.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: init 2026-02-21T22:25:47.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:47.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:25:47.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-b con 0x7f4dcc08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-c con 0x7f4dcc08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-a con 0x7f4dcc08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc086a10 con 0x7f4dcc08d8e0 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc08b130 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc08bd60 con 0x7f4dcc08ce00 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd391c640 10 monclient(hunting): authenticate will time out at 1756.478691s 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd1691640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08ce00 0x7f4dcc08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:49990/0 (socket says 10.20.193.102:49990) 2026-02-21T22:25:47.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.239+0000 7f4dd1691640 1 -- 10.20.193.102:0/2241282706 learned_addr learned my addr 10.20.193.102:0/2241282706 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd1691640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08ce00 legacy=0x7f4dcc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd1e92640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.102:6789/0 conn(0x7f4dcc08d8e0 legacy=0x7f4dcc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd0e90640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc08b7f0 legacy=0x7f4dcc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606140616 0 0) 0x7f4dcc08bd60 con 0x7f4dcc08ce00 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): my global_id is 84131 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4db4003620 con 0x7f4dcc08ce00 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018126625 0 0) 0x7f4dcc086a10 con 0x7f4dcc08d8e0 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): my global_id is 34491 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dcc08bd60 con 0x7f4dcc08d8e0 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191111948 0 0) 0x7f4dcc08b130 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient(hunting): my global_id is 54292 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dcc086a10 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2725416355 0 0) 0x7f4db4003620 con 0x7f4dcc08ce00 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc08b7f0 legacy=0x7f4dcc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.102:6789/0 conn(0x7f4dcc08d8e0 legacy=0x7f4dcc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 monclient: found mon.noname-b 2026-02-21T22:25:47.246 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc08b370 con 0x7f4dcc08ce00 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: _finish_auth 0 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dbc003110 con 0x7f4dcc08ce00 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: dump: 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 5 monclient: authenticate success, global_id 84131 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:47.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4dbc0033e0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 1 -- 10.20.193.102:0/2241282706 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dbc004b50 con 0x7f4dcc08ce00 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dbbfff640 10 monclient: dump: 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 4 set_mon_vals no callback set 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: shutdown 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- 10.20.193.102:0/2241282706 >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08ce00 legacy=0x7f4dcc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- 10.20.193.102:0/2241282706 shutdown_connections 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- 10.20.193.102:0/2241282706 >> 10.20.193.102:0/2241282706 conn(0x7f4dcc091a50 msgr2=0x7f4dcc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- 10.20.193.102:0/2241282706 shutdown_connections 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- 10.20.193.102:0/2241282706 wait complete. 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: build_initial_monmap 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: monmap: 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 Processor -- start 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- start start 2026-02-21T22:25:47.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: init 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-c con 0x7f4dcc08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-a con 0x7f4dcc08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient(hunting): picked mon.noname-b con 0x7f4dcc08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc083390 con 0x7f4dcc08ce00 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc05e380 con 0x7f4dcc08d8e0 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dcc088a70 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd391c640 10 monclient(hunting): authenticate will time out at 1756.482894s 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd1691640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08b7f0 0x7f4dcc1531d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50002/0 (socket says 10.20.193.102:50002) 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd1691640 1 -- 10.20.193.102:0/3620819118 learned_addr learned my addr 10.20.193.102:0/3620819118 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.243+0000 7f4dd0e90640 1 --1- 10.20.193.102:0/3620819118 >> v1:10.20.193.102:6789/0 conn(0x7f4dcc08ce00 0x7f4dcc0848c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41610/0 (socket says 10.20.193.102:41610) 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd0e90640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.102:6789/0 conn(0x7f4dcc08ce00 legacy=0x7f4dcc0848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd1691640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08b7f0 legacy=0x7f4dcc1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2161159496 0 0) 0x7f4dcc083390 con 0x7f4dcc08ce00 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): my global_id is 34494 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d9c003620 con 0x7f4dcc08ce00 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499687872 0 0) 0x7f4dcc088a70 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): my global_id is 84134 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dcc083390 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd1e92640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc08d8e0 legacy=0x7f4dcc195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1026635935 0 0) 0x7f4d9c003620 con 0x7f4dcc08ce00 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dcc088a70 con 0x7f4dcc08ce00 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dc00033c0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): dump: 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.253 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3488771369 0 0) 0x7f4dcc083390 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d9c003620 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dbc002b70 con 0x7f4dcc08b7f0 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient(hunting): dump: 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3664244641 0 0) 0x7f4dcc088a70 con 0x7f4dcc08ce00 2026-02-21T22:25:47.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.102:6790/0 conn(0x7f4dcc08b7f0 legacy=0x7f4dcc1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc08d8e0 legacy=0x7f4dcc195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 monclient: found mon.a 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc081030 con 0x7f4dcc08ce00 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 5 monclient: authenticate success, global_id 34494 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 10 monclient: _renew_subs 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc0830b0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 10 monclient: _renew_subs 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4dcc084fd0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4dc00039c0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dc0003d50 con 0x7f4dcc08ce00 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 10 monclient: dump: 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.255 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dbb7fe640 4 set_mon_vals no callback set 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4dcc082af0 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4dc0004020 con 0x7f4dcc08ce00 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(490..490 src has 112..490) ==== 3987+0+0 (unknown 1522834410 0 0) 0x7f4dc004c6f0 con 0x7f4dcc08ce00 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.247+0000 7f4dd1e92640 10 monclient: get_auth_request con 0x7f4d9c03d650 auth_method 0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4dd1691640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b4f0 legacy=0x7f4dcc04d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914439761 0 0) 0x7f4d9c003620 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient(hunting): my global_id is 54298 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dcc088a70 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1745278250 0 0) 0x7f4dcc088a70 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d9c003620 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dbc004e80 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient: dump: 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.256 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1372125611 0 0) 0x7f4d9c003620 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4dcc0008e0 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f4dbc007a20 con 0x7f4dcc07b4f0 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b4f0 legacy=0x7f4dcc04d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.251+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b4f0 legacy=0x7f4dcc04d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:47.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4dd391c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:47.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4dcc04d990 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4dd1691640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b8c0 legacy=0x7f4dcc197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306987923 0 0) 0x7f4dcc04d990 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 10 monclient(hunting): my global_id is 54301 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.255+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d9c003620 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1573570988 0 0) 0x7f4d9c003620 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dcc04d990 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4dbc005350 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 10 monclient: dump: 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.260 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2692113463 0 0) 0x7f4dcc04d990 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4dcc090810 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 3229493722) 0x7f4dbc007790 con 0x7f4dcc07b8c0 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:47.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b8c0 legacy=0x7f4dcc197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4db9ffb640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6789/0 conn(0x7f4dcc07b8c0 legacy=0x7f4dcc197c70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 250901, 2026-02-21T22:25:47.262 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.263 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.264 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:47.265 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:47.266 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:47.267 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.268 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.269 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:47.270 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:47.271 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.272 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:47.273 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:47.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.178:6800/2549949439 conn(0x7f4d9c03d650 legacy=0x7f4d9c05db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 10 monclient: shutdown 2026-02-21T22:25:47.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 >> v1:10.20.193.102:6789/0 conn(0x7f4dcc08ce00 legacy=0x7f4dcc0848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd1e92640 1 -- 10.20.193.102:0/3620819118 reap_dead start 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd1e92640 1 -- 10.20.193.102:0/3620819118 reap_dead start 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 shutdown_connections 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 >> 10.20.193.102:0/3620819118 conn(0x7f4dcc091a50 msgr2=0x7f4dcc05cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 shutdown_connections 2026-02-21T22:25:47.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.259+0000 7f4dd391c640 1 -- 10.20.193.102:0/3620819118 wait complete. 2026-02-21T22:25:47.275 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:47.275 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:25:47.276 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:25:47.324 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: get_monmap_and_config 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: build_initial_monmap 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: monmap: 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.336 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:47.338244+0000 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:47.338244+0000 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:47.337 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 1 Processor -- start 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 1 -- start start 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: init 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-a con 0x7f0ee808c1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:47.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-b con 0x7f0ee808cc80 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-c con 0x7f0ee8183480 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee808d780 con 0x7f0ee808c1a0 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee8085da0 con 0x7f0ee8183480 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee8085420 con 0x7f0ee808cc80 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0eed19b640 10 monclient(hunting): authenticate will time out at 1756.574134s 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0ee7fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0ee808c1a0 0x7f0ee808c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41616/0 (socket says 10.20.193.102:41616) 2026-02-21T22:25:47.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.335+0000 7f0ee7fff640 1 -- 10.20.193.102:0/4279717807 learned_addr learned my addr 10.20.193.102:0/4279717807 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:47.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee7fff640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.102:6789/0 conn(0x7f0ee808c1a0 legacy=0x7f0ee808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee77fe640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.102:6790/0 conn(0x7f0ee808cc80 legacy=0x7f0ee8181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eec99a640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.178:6789/0 conn(0x7f0ee8183480 legacy=0x7f0ee81a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1083163533 0 0) 0x7f0ed4002c60 con 0x7f0ee808c1a0 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): my global_id is 34497 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ecc003620 con 0x7f0ee808c1a0 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230342165 0 0) 0x7f0ee8085420 con 0x7f0ee808cc80 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): my global_id is 84137 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ed4002c60 con 0x7f0ee808cc80 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297682414 0 0) 0x7f0ee8085da0 con 0x7f0ee8183480 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient(hunting): my global_id is 54304 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee8085420 con 0x7f0ee8183480 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2365322268 0 0) 0x7f0ecc003620 con 0x7f0ee808c1a0 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.102:6790/0 conn(0x7f0ee808cc80 legacy=0x7f0ee8181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.178:6789/0 conn(0x7f0ee8183480 legacy=0x7f0ee81a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:25:47.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 monclient: found mon.noname-a 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee8086640 con 0x7f0ee808c1a0 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 5 monclient: authenticate success, global_id 34497 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0ed40032e0 con 0x7f0ee808c1a0 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: dump: 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.345 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0ed40049a0 con 0x7f0ee808c1a0 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 1 -- 10.20.193.102:0/4279717807 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0ed4004d30 con 0x7f0ee808c1a0 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0ee6ffd640 10 monclient: dump: 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 4 set_mon_vals no callback set 2026-02-21T22:25:47.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: shutdown 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- 10.20.193.102:0/4279717807 >> v1:10.20.193.102:6789/0 conn(0x7f0ee808c1a0 legacy=0x7f0ee808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- 10.20.193.102:0/4279717807 shutdown_connections 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- 10.20.193.102:0/4279717807 >> 10.20.193.102:0/4279717807 conn(0x7f0ee808f6a0 msgr2=0x7f0ee808faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- 10.20.193.102:0/4279717807 shutdown_connections 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- 10.20.193.102:0/4279717807 wait complete. 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: build_initial_monmap 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: monmap: 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:47.348 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 Processor -- start 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- start start 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: init 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-b con 0x7f0ee808c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-a con 0x7f0ee808cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient(hunting): picked mon.noname-c con 0x7f0ee8183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:47.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee8058890 con 0x7f0ee808cc80 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee805e0a0 con 0x7f0ee808c1a0 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee8088990 con 0x7f0ee8183480 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eed19b640 10 monclient(hunting): authenticate will time out at 1756.578684s 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eec99a640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0ee8183480 0x7f0ee81753b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50032/0 (socket says 10.20.193.102:50032) 2026-02-21T22:25:47.350 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.339+0000 7f0eec99a640 1 -- 10.20.193.102:0/3882625230 learned_addr learned my addr 10.20.193.102:0/3882625230 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eec99a640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.102:6790/0 conn(0x7f0ee8183480 legacy=0x7f0ee81753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee77fe640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.102:6789/0 conn(0x7f0ee808cc80 legacy=0x7f0ee8084810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944701092 0 0) 0x7f0ee8085420 con 0x7f0ee8183480 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.353 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): my global_id is 84140 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0eb8003620 con 0x7f0ee8183480 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404600138 0 0) 0x7f0ed8003450 con 0x7f0ee808cc80 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): my global_id is 34500 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee8085420 con 0x7f0ee808cc80 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee7fff640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.178:6789/0 conn(0x7f0ee808c1a0 legacy=0x7f0ee8153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2341834031 0 0) 0x7f0eb8003620 con 0x7f0ee8183480 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ed8003450 con 0x7f0ee8183480 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0edc004980 con 0x7f0ee8183480 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:47.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): dump: 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2661335184 0 0) 0x7f0ee8085420 con 0x7f0ee808cc80 2026-02-21T22:25:47.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0eb8003620 con 0x7f0ee808cc80 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0ed80038c0 con 0x7f0ee808cc80 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient(hunting): dump: 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.356 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3442495137 0 0) 0x7f0ed8003450 con 0x7f0ee8183480 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.178:6789/0 conn(0x7f0ee808c1a0 legacy=0x7f0ee8153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.102:6789/0 conn(0x7f0ee808cc80 legacy=0x7f0ee8084810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 monclient: found mon.c 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee8083000 con 0x7f0ee8183480 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: _finish_auth 0 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 5 monclient: authenticate success, global_id 84140 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _renew_subs 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee8082d20 con 0x7f0ee8183480 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _renew_subs 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f0ee8084f20 con 0x7f0ee8183480 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0edc004ed0 con 0x7f0ee8183480 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0edc0052e0 con 0x7f0ee8183480 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee67fc640 4 set_mon_vals no callback set 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 10 monclient: dump: 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.358 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0edc005500 con 0x7f0ee8183480 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee80008e0 con 0x7f0ee8183480 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(490..490 src has 112..490) ==== 3987+0+0 (unknown 1522834410 0 0) 0x7f0edc04dce0 con 0x7f0ee8183480 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.343+0000 7f0ee7fff640 10 monclient: get_auth_request con 0x7f0eb803d650 auth_method 0 2026-02-21T22:25:47.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.347+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0ee8088990 con 0x7f0ee8183480 2026-02-21T22:25:47.360 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.347+0000 7f0ee4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:47.360 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.347+0000 7f0ee4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:47.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:25:47.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:25:47.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:47.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0ee80857a0 con 0x7f0ee8183480 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=13,kv=11,logm=58} crc {auth=444507208,config=1618278320,health=702036140,kv=3765542339,logm=3889511952}) 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=14,config=2,health=11,kv=12,logm=61} crc {auth=3175223690,config=1618278320,health=1122601069,kv=514132712,logm=2991568725}) 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=13,kv=11,logm=58} crc {auth=444507208,config=1618278320,health=702036140,kv=3765542339,logm=3889511952}) 2026-02-21T22:25:47.466 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=13,kv=11,logm=59} crc {auth=1365056518,config=1618278320,health=702036140,kv=1097538409,logm=2865942908}) 2026-02-21T22:25:47.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0ee4ff9640 1 -- 10.20.193.102:0/3882625230 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f0edc018c40 con 0x7f0ee8183480 2026-02-21T22:25:47.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0ee4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:25:47.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0ee4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1716221217}) 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1184789482}) 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1716221217}) 2026-02-21T22:25:47.468 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.463+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2307017498}) 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0eb803d650 legacy=0x7f0eb805db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 10 monclient: shutdown 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 >> v1:10.20.193.102:6790/0 conn(0x7f0ee8183480 legacy=0x7f0ee81753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 shutdown_connections 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 >> 10.20.193.102:0/3882625230 conn(0x7f0ee808f6a0 msgr2=0x7f0ee80574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:47.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.463+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 shutdown_connections 2026-02-21T22:25:47.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:47.467+0000 7f0eed19b640 1 -- 10.20.193.102:0/3882625230 wait complete. 2026-02-21T22:25:47.473 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.473 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=340863839}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3971785950}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=340863839}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4167194655}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1132216683}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4186579495}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1132216683}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2938290039}) 2026-02-21T22:25:47.474 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409299597}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1972317946}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1409299597}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2247482132}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3155156033}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=57135549}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3155156033}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2161770946}) 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.475 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2119530249}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3334687949}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2119530249}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.467+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=357597939}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=931997983}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088659754}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=931997983}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2522838231}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977024490}) 2026-02-21T22:25:47.476 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1434434421}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3977024490}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1210957240}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=312744766}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=564476703}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=312744766}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3014883128}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847918249}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=610245803}) 2026-02-21T22:25:47.477 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.478 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2847918249}) 2026-02-21T22:25:47.478 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=722836038}) 2026-02-21T22:25:47.479 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.479 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3340369927}) 2026-02-21T22:25:47.479 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3114744288}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3340369927}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.471+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=491312892}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3659733680}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1493050590}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3659733680}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=520564139}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4155793804}) 2026-02-21T22:25:47.480 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=630041132}) 2026-02-21T22:25:47.481 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.481 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4155793804}) 2026-02-21T22:25:47.481 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2031163064}) 2026-02-21T22:25:47.482 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:25:47.487 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284445322}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3881593860}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3284445322}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2044784461}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1968488884}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3526900308}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1968488884}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4177159011}) 2026-02-21T22:25:47.488 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=352372901}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3719970082}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=352372901}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=197705491}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3952220246}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3537178727}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3952220246}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.475+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=644704698}) 2026-02-21T22:25:47.489 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2834772003}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1947026254}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2834772003}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3895755372}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3990319463}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3004267758}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3990319463}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1118607780}) 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.490 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936652216}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4025195270}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1936652216}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1022526257}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2659290951}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3106456717}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2659290951}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2126343684}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15} crc {logm=206747219,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2652785122,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=460548319,osd_snap=2913090133}) 2026-02-21T22:25:47.491 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=44,mdsmap=2,mgr=8,mgr_command_descs=1,mgrstat=9,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=14} crc {logm=4034736279,mdsmap=4097168772,mgr=4278827987,mgr_command_descs=2141687499,mgrstat=2061972184,mon_config_key=2427692996,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=460548319,osd_snap=1739422894}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=42,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=15} crc {logm=206747219,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2652785122,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=460548319,osd_snap=2913090133}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.479+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=43,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=12} crc {logm=2966243969,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708307,mgrstat=3708335383,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=460548319,osd_snap=3193845409}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=876961718,osdmap=482275016}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=77,osdmap=23} crc {osd_snap=3672874771,osdmap=581020221}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=72,osdmap=28} crc {osd_snap=876961718,osdmap=482275016}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=77,osdmap=23} crc {osd_snap=329182895,osdmap=3335516475}) 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.492 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=116416145}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2825597960}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=116416145}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=52642383}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=750574254}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3796209838}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=750574254}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3042835330}) 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.493 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=281838761}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2741473862}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=281838761}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2550073364}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=814100196}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1543028821}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.483+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=814100196}) 2026-02-21T22:25:47.494 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2945368137}) 2026-02-21T22:25:47.495 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2411334504}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1578185036}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2411334504}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=666061106}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=35} crc {osdmap=2694601076}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=34} crc {osdmap=2321439464}) 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:25:47.496 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=35} crc {osdmap=2694601076}) 2026-02-21T22:25:47.497 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:25:47.487+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=34} crc {osdmap=924026131}) 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: finishing write tid 1 to trial17818531-40 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: finishing write tid 2 to trial17818531-40 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: finishing write tid 3 to trial17818531-40 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: finishing write tid 4 to trial17818531-40 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: finishing write tid 5 to trial17818531-40 2026-02-21T22:25:47.499 INFO:tasks.rados.rados.0.trial178.stdout:2377: oid 40 updating version 0 to 1108 2026-02-21T22:25:47.500 INFO:tasks.rados.rados.0.trial178.stdout:2377: oid 40 version 1108 is already newer than 1107 2026-02-21T22:25:47.500 INFO:tasks.rados.rados.0.trial178.stdout:2377: oid 40 version 1108 is already newer than 1105 2026-02-21T22:25:47.500 INFO:tasks.rados.rados.0.trial178.stdout:2377: oid 40 version 1108 is already newer than 1106 2026-02-21T22:25:47.500 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1108 (ObjNum 832 snap 238 seq_num 832) dirty exists 2026-02-21T22:25:47.500 INFO:tasks.rados.rados.0.trial178.stdout:2377: left oid 40 (ObjNum 832 snap 238 seq_num 832) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2370: done (5 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2371: done (4 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2374: done (3 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2376: done (2 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2377: done (1 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2379: done (0 left) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2380: read oid 37 snap -1 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2380: expect (ObjNum 808 snap 226 seq_num 808) 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2381: write oid 28 current snap is 239 2026-02-21T22:25:48.164 INFO:tasks.rados.rados.0.trial178.stdout:2381: seq_num 833 ranges {400231=564894,1661971=338260} 2026-02-21T22:25:48.170 INFO:tasks.rados.rados.0.trial178.stdout:2381: writing trial17818531-28 from 400231 to 965125 tid 1 2026-02-21T22:25:48.177 INFO:tasks.rados.rados.0.trial178.stdout:2381: writing trial17818531-28 from 1661971 to 2000231 tid 2 2026-02-21T22:25:48.177 INFO:tasks.rados.rados.0.trial178.stdout:2382: delete oid 43 current snap is 239 2026-02-21T22:25:48.177 INFO:tasks.rados.rados.0.trial178.stdout:2381: finishing write tid 1 to trial17818531-28 2026-02-21T22:25:48.177 INFO:tasks.rados.rados.0.trial178.stdout:2380: expect (ObjNum 808 snap 226 seq_num 808) 2026-02-21T22:25:48.225 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 405608 2026-02-21T22:25:48.276 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:25:48.332 INFO:tasks.rados.rados.0.trial178.stdout:2381: finishing write tid 2 to trial17818531-28 2026-02-21T22:25:48.332 INFO:tasks.rados.rados.0.trial178.stdout:2381: finishing write tid 3 to trial17818531-28 2026-02-21T22:25:48.333 INFO:tasks.rados.rados.0.trial178.stdout:2381: finishing write tid 4 to trial17818531-28 2026-02-21T22:25:48.333 INFO:tasks.rados.rados.0.trial178.stdout:2381: oid 28 updating version 0 to 841 2026-02-21T22:25:48.333 INFO:tasks.rados.rados.0.trial178.stdout:2381: oid 28 version 841 is already newer than 839 2026-02-21T22:25:48.333 INFO:tasks.rados.rados.0.trial178.stdout:2381: oid 28 version 841 is already newer than 840 2026-02-21T22:25:48.333 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 841 (ObjNum 833 snap 239 seq_num 833) dirty exists 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2381: left oid 28 (ObjNum 833 snap 239 seq_num 833) 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2380: done (2 left) 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2381: done (1 left) 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2382: done (0 left) 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2383: read oid 48 snap 224 2026-02-21T22:25:48.334 INFO:tasks.rados.rados.0.trial178.stdout:2383: expect deleted 2026-02-21T22:25:48.335 INFO:tasks.rados.rados.0.trial178.stdout:2384: snap_remove snap 238 2026-02-21T22:25:48.483 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:25:48.483 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:48.542 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 10 monclient: get_monmap_and_config 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 10 monclient: monmap: 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:48.544018+0000 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:48.544018+0000 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.543 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 1 Processor -- start 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 1 -- start start 2026-02-21T22:25:48.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.539+0000 7fd7fd0d4640 10 monclient: init 2026-02-21T22:25:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-a con 0x7fd7f808cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-b con 0x7fd7f808d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-c con 0x7fd7f808b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f80890e0 con 0x7fd7f808cd00 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f8088ea0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f808aea0 con 0x7fd7f808d7e0 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient(hunting): authenticate will time out at 1757.780155s 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f6d76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd7f808cd00 0x7fd7f808d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41636/0 (socket says 10.20.193.102:41636) 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f6d76640 1 -- 10.20.193.102:0/3858283060 learned_addr learned my addr 10.20.193.102:0/3858283060 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f6d76640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.102:6789/0 conn(0x7fd7f808cd00 legacy=0x7fd7f808d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f6575640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.102:6790/0 conn(0x7fd7f808d7e0 legacy=0x7fd7f81840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127847759 0 0) 0x7fd7f80890e0 con 0x7fd7f808cd00 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): my global_id is 34503 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7e4003620 con 0x7fd7f808cd00 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041377626 0 0) 0x7fd7f808aea0 con 0x7fd7f808d7e0 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient(hunting): my global_id is 84143 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7f80890e0 con 0x7fd7f808d7e0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f7577640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.178:6789/0 conn(0x7fd7f808b8d0 legacy=0x7fd7f81a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2671244185 0 0) 0x7fd7f80890e0 con 0x7fd7f808d7e0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.178:6789/0 conn(0x7fd7f808b8d0 legacy=0x7fd7f81a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.102:6789/0 conn(0x7fd7f808cd00 legacy=0x7fd7f808d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 monclient: found mon.noname-b 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f808a600 con 0x7fd7f808d7e0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.551 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd7e0003160 con 0x7fd7f808d7e0 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 5 monclient: authenticate success, global_id 84143 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: dump: 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.552 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd7e0003430 con 0x7fd7f808d7e0 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 1 -- 10.20.193.102:0/3858283060 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd7e00038c0 con 0x7fd7f808d7e0 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7f5d74640 10 monclient: dump: 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:48.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 4 set_mon_vals no callback set 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: shutdown 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/3858283060 >> v1:10.20.193.102:6790/0 conn(0x7fd7f808d7e0 legacy=0x7fd7f81840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/3858283060 shutdown_connections 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/3858283060 >> 10.20.193.102:0/3858283060 conn(0x7fd7f8091a50 msgr2=0x7fd7f8091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/3858283060 shutdown_connections 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/3858283060 wait complete. 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.543+0000 7fd7fd0d4640 10 monclient: monmap: 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:48.555 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 Processor -- start 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- start start 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: init 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-c con 0x7fd7f808b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-a con 0x7fd7f808cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient(hunting): picked mon.noname-b con 0x7fd7f808d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f805d4b0 con 0x7fd7f808cd00 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f8088a70 con 0x7fd7f808d7e0 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f8153b60 con 0x7fd7f808b8d0 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient(hunting): authenticate will time out at 1757.784626s 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f6d76640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd7f808b8d0 0x7fd7f80838e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50054/0 (socket says 10.20.193.102:50054) 2026-02-21T22:25:48.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f6d76640 1 -- 10.20.193.102:0/198565373 learned_addr learned my addr 10.20.193.102:0/198565373 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f6d76640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.102:6790/0 conn(0x7fd7f808b8d0 legacy=0x7fd7f80838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f6575640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.102:6789/0 conn(0x7fd7f808cd00 legacy=0x7fd7f8194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712920459 0 0) 0x7fd7f8153b60 con 0x7fd7f808b8d0 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): my global_id is 84146 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7c4003620 con 0x7fd7f808b8d0 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f7577640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.178:6789/0 conn(0x7fd7f808d7e0 legacy=0x7fd7f8195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2889547748 0 0) 0x7fd7f805d4b0 con 0x7fd7f808cd00 2026-02-21T22:25:48.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): my global_id is 34506 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7f8153b60 con 0x7fd7f808cd00 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4280042270 0 0) 0x7fd7c4003620 con 0x7fd7f808b8d0 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd7f805d4b0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512478358 0 0) 0x7fd7f8088a70 con 0x7fd7f808d7e0 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): my global_id is 54310 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7c4003620 con 0x7fd7f808d7e0 2026-02-21T22:25:48.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd7ec0033a0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): dump: 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.563 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4125557 0 0) 0x7fd7f8153b60 con 0x7fd7f808cd00 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd7f8088a70 con 0x7fd7f808cd00 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd7e0003710 con 0x7fd7f808cd00 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient(hunting): dump: 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.564 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2310690623 0 0) 0x7fd7f805d4b0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.178:6789/0 conn(0x7fd7f808d7e0 legacy=0x7fd7f8195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.102:6789/0 conn(0x7fd7f808cd00 legacy=0x7fd7f8194810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 monclient: found mon.c 2026-02-21T22:25:48.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8080050 con 0x7fd7f808b8d0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 5 monclient: authenticate success, global_id 84146 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _renew_subs 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f80823b0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _renew_subs 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd7f8083ff0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd7ec004b90 con 0x7fd7f808b8d0 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd7ec004f20 con 0x7fd7f808b8d0 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7d77fe640 10 monclient: dump: 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.567 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7f5573640 4 set_mon_vals no callback set 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.547+0000 7fd7fd0d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7fd0d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7fd0d4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f805dff0 con 0x7fd7f808b8d0 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd7ec005100 con 0x7fd7f808b8d0 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7fd7ec04c700 con 0x7fd7f808b8d0 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7f6575640 10 monclient: get_auth_request con 0x7fd7c403d850 auth_method 0 2026-02-21T22:25:48.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fd7ec017640 con 0x7fd7f808b8d0 2026-02-21T22:25:48.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7d77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.551+0000 7fd7d77fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:48.569 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:48.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd7f8082030 con 0x7fd7f808b8d0 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7d77fe640 1 -- 10.20.193.102:0/198565373 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2915026968) 0x7fd7ec01c960 con 0x7fd7f808b8d0 2026-02-21T22:25:48.676 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7d77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:48.677 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7d77fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:48.677 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd7c403d850 legacy=0x7fd7c405dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 10 monclient: shutdown 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 >> v1:10.20.193.102:6790/0 conn(0x7fd7f808b8d0 legacy=0x7fd7f80838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 shutdown_connections 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 >> 10.20.193.102:0/198565373 conn(0x7fd7f8091a50 msgr2=0x7fd7f8084c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.678 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 shutdown_connections 2026-02-21T22:25:48.679 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.675+0000 7fd7fd0d4640 1 -- 10.20.193.102:0/198565373 wait complete. 2026-02-21T22:25:48.686 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:25:48.686 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:25:48.687 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:25:48.687 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:25:48.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: get_monmap_and_config 2026-02-21T22:25:48.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: monmap: 2026-02-21T22:25:48.758 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:48.758944+0000 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:48.758944+0000 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 1 Processor -- start 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 1 -- start start 2026-02-21T22:25:48.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: init 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-a con 0x7fa66c087b60 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-c con 0x7fa66c08d2b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-b con 0x7fa66c185370 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c08a200 con 0x7fa66c087b60 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c08a4e0 con 0x7fa66c08d2b0 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c088640 con 0x7fa66c185370 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa670edc640 10 monclient(hunting): authenticate will time out at 1757.994843s 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa66b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa66c087b60 0x7fa66c087f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41646/0 (socket says 10.20.193.102:41646) 2026-02-21T22:25:48.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.755+0000 7fa66b7fe640 1 -- 10.20.193.102:0/1546327728 learned_addr learned my addr 10.20.193.102:0/1546327728 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66b7fe640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.102:6789/0 conn(0x7fa66c087b60 legacy=0x7fa66c087f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66bfff640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.102:6790/0 conn(0x7fa66c185370 legacy=0x7fa66c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363096912 0 0) 0x7fa66c08a200 con 0x7fa66c087b60 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): my global_id is 34509 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa650003620 con 0x7fa66c087b60 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140072842 0 0) 0x7fa66c088640 con 0x7fa66c185370 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient(hunting): my global_id is 84149 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c08a200 con 0x7fa66c185370 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66affd640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.178:6789/0 conn(0x7fa66c08d2b0 legacy=0x7fa66c183c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3582522704 0 0) 0x7fa650003620 con 0x7fa66c087b60 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.102:6790/0 conn(0x7fa66c185370 legacy=0x7fa66c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.178:6789/0 conn(0x7fa66c08d2b0 legacy=0x7fa66c183c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 monclient: found mon.noname-a 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c08b220 con 0x7fa66c087b60 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 5 monclient: authenticate success, global_id 34509 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa658003110 con 0x7fa66c087b60 2026-02-21T22:25:48.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: dump: 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.769 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa658003450 con 0x7fa66c087b60 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 1 -- 10.20.193.102:0/1546327728 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa658004b50 con 0x7fa66c087b60 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66a7fc640 10 monclient: dump: 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.770 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 4 set_mon_vals no callback set 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: shutdown 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- 10.20.193.102:0/1546327728 >> v1:10.20.193.102:6789/0 conn(0x7fa66c087b60 legacy=0x7fa66c087f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- 10.20.193.102:0/1546327728 shutdown_connections 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- 10.20.193.102:0/1546327728 >> 10.20.193.102:0/1546327728 conn(0x7fa66c08f6a0 msgr2=0x7fa66c08faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- 10.20.193.102:0/1546327728 shutdown_connections 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- 10.20.193.102:0/1546327728 wait complete. 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: monmap: 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:48.772 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 Processor -- start 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- start start 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: init 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-c con 0x7fa66c087b60 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-a con 0x7fa66c08d2b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient(hunting): picked mon.noname-b con 0x7fa66c185370 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c05e620 con 0x7fa66c08d2b0 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c088990 con 0x7fa66c185370 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66c07fcf0 con 0x7fa66c087b60 2026-02-21T22:25:48.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa670edc640 10 monclient(hunting): authenticate will time out at 1757.998772s 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66b7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa66c087b60 0x7fa66c084590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50074/0 (socket says 10.20.193.102:50074) 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.759+0000 7fa66b7fe640 1 -- 10.20.193.102:0/1290469676 learned_addr learned my addr 10.20.193.102:0/1290469676 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa66b7fe640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6790/0 conn(0x7fa66c087b60 legacy=0x7fa66c084590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa66affd640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c08d2b0 legacy=0x7fa66c172e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928779312 0 0) 0x7fa66c07fcf0 con 0x7fa66c087b60 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): my global_id is 84152 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa63c003620 con 0x7fa66c087b60 2026-02-21T22:25:48.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744848303 0 0) 0x7fa66c05e620 con 0x7fa66c08d2b0 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): my global_id is 34512 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c07fcf0 con 0x7fa66c08d2b0 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa66bfff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.178:6789/0 conn(0x7fa66c185370 legacy=0x7fa66c174520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1048686916 0 0) 0x7fa63c003620 con 0x7fa66c087b60 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa66c05e620 con 0x7fa66c087b60 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa658002b70 con 0x7fa66c087b60 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:48.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): dump: 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3234039873 0 0) 0x7fa66c07fcf0 con 0x7fa66c08d2b0 2026-02-21T22:25:48.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa63c003620 con 0x7fa66c08d2b0 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa65c003380 con 0x7fa66c08d2b0 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): dump: 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.777 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3154074039 0 0) 0x7fa66c088990 con 0x7fa66c185370 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient(hunting): my global_id is 54316 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c07fcf0 con 0x7fa66c185370 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 595449449 0 0) 0x7fa66c05e620 con 0x7fa66c087b60 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.178:6789/0 conn(0x7fa66c185370 legacy=0x7fa66c174520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c08d2b0 legacy=0x7fa66c172e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 monclient: found mon.c 2026-02-21T22:25:48.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c082050 con 0x7fa66c087b60 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 5 monclient: authenticate success, global_id 84152 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 10 monclient: _renew_subs 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c081810 con 0x7fa66c087b60 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 10 monclient: _renew_subs 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa66c084e70 con 0x7fa66c087b60 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa658003290 con 0x7fa66c087b60 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa658005170 con 0x7fa66c087b60 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 10 monclient: dump: 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.780 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa669ffb640 4 set_mon_vals no callback set 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa66c085040 con 0x7fa66c131ce0 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa658011800 con 0x7fa66c087b60 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7fa65804c6d0 con 0x7fa66c087b60 2026-02-21T22:25:48.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.763+0000 7fa66bfff640 10 monclient: get_auth_request con 0x7fa63c03d890 auth_method 0 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa66affd640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c131ce0 legacy=0x7fa66c07a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575986220 0 0) 0x7fa63c003620 con 0x7fa66c131ce0 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient(hunting): my global_id is 34515 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa66c05e620 con 0x7fa66c131ce0 2026-02-21T22:25:48.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 16373005 0 0) 0x7fa66c05e620 con 0x7fa66c131ce0 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa63c003620 con 0x7fa66c131ce0 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa65c003ab0 con 0x7fa66c131ce0 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient: dump: 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.785 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1744130234 0 0) 0x7fa63c003620 con 0x7fa66c131ce0 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa66c08b890 con 0x7fa66c131ce0 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fa65c0068f0 con 0x7fa66c131ce0 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c131ce0 legacy=0x7fa66c07a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.767+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c131ce0 legacy=0x7fa66c07a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:48.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa66c03ad60 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa66affd640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c07aa80 legacy=0x7fa66c039640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081205235 0 0) 0x7fa66c03ad60 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient(hunting): my global_id is 34518 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa63c003620 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3523463371 0 0) 0x7fa63c003620 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa66c03ad60 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa65c004230 con 0x7fa66c07aa80 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient: dump: 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2392695743 0 0) 0x7fa66c03ad60 con 0x7fa66c07aa80 2026-02-21T22:25:48.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa66c062ce0 con 0x7fa66c07aa80 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3386039409) 0x7fa65c004530 con 0x7fa66c07aa80 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c07aa80 legacy=0x7fa66c039640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa647fff640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6789/0 conn(0x7fa66c07aa80 legacy=0x7fa66c039640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:48.789 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 497640, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 3, 2026-02-21T22:25:48.790 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.791 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.792 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:48.793 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.794 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:48.795 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:48.796 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.797 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.798 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:48.799 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.800 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:48.801 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:48.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa63c03d890 legacy=0x7fa63c05dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 10 monclient: shutdown 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 >> v1:10.20.193.102:6790/0 conn(0x7fa66c087b60 legacy=0x7fa66c084590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa66bfff640 1 -- 10.20.193.102:0/1290469676 reap_dead start 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa66bfff640 1 -- 10.20.193.102:0/1290469676 reap_dead start 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 shutdown_connections 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 >> 10.20.193.102:0/1290469676 conn(0x7fa66c08f6a0 msgr2=0x7fa66c1a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 shutdown_connections 2026-02-21T22:25:48.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.771+0000 7fa670edc640 1 -- 10.20.193.102:0/1290469676 wait complete. 2026-02-21T22:25:48.803 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:48.803 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:25:48.804 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: get_monmap_and_config 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: monmap: 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:48.874484+0000 2026-02-21T22:25:48.874 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:48.874484+0000 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 1 Processor -- start 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 1 -- start start 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: init 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-a con 0x7f8e5008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-c con 0x7f8e5008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-b con 0x7f8e5008d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e50086a10 con 0x7f8e5008ce00 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5008b130 con 0x7f8e5008b7f0 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5008bd60 con 0x7f8e5008d8e0 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e586b8640 10 monclient(hunting): authenticate will time out at 1758.110392s 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e5642d640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f8e5008ce00 0x7f8e5008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41700/0 (socket says 10.20.193.102:41700) 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e5642d640 1 -- 10.20.193.102:0/668576601 learned_addr learned my addr 10.20.193.102:0/668576601 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.871+0000 7f8e56c2e640 1 --1- 10.20.193.102:0/668576601 >> v1:10.20.193.102:6790/0 conn(0x7f8e5008d8e0 0x7f8e501a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50082/0 (socket says 10.20.193.102:50082) 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e56c2e640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.102:6790/0 conn(0x7f8e5008d8e0 legacy=0x7f8e501a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5642d640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.102:6789/0 conn(0x7f8e5008ce00 legacy=0x7f8e5008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e55c2c640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.178:6789/0 conn(0x7f8e5008b7f0 legacy=0x7f8e50181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939953710 0 0) 0x7f8e5008bd60 con 0x7f8e5008d8e0 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): my global_id is 84155 2026-02-21T22:25:48.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e38003620 con 0x7f8e5008d8e0 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225442334 0 0) 0x7f8e50086a10 con 0x7f8e5008ce00 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): my global_id is 34521 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e5008bd60 con 0x7f8e5008ce00 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431267011 0 0) 0x7f8e5008b130 con 0x7f8e5008b7f0 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient(hunting): my global_id is 54319 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e50086a10 con 0x7f8e5008b7f0 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3714409287 0 0) 0x7f8e38003620 con 0x7f8e5008d8e0 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.178:6789/0 conn(0x7f8e5008b7f0 legacy=0x7f8e50181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.102:6789/0 conn(0x7f8e5008ce00 legacy=0x7f8e5008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 monclient: found mon.noname-b 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5008b370 con 0x7f8e5008d8e0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e4c004620 con 0x7f8e5008d8e0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: dump: 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.884 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 5 monclient: authenticate success, global_id 84155 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e4c0048f0 con 0x7f8e5008d8e0 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 1 -- 10.20.193.102:0/668576601 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e4c005fa0 con 0x7f8e5008d8e0 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e5542b640 10 monclient: dump: 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 4 set_mon_vals no callback set 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: shutdown 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- 10.20.193.102:0/668576601 >> v1:10.20.193.102:6790/0 conn(0x7f8e5008d8e0 legacy=0x7f8e501a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- 10.20.193.102:0/668576601 shutdown_connections 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- 10.20.193.102:0/668576601 >> 10.20.193.102:0/668576601 conn(0x7f8e50091a50 msgr2=0x7f8e50091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- 10.20.193.102:0/668576601 shutdown_connections 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- 10.20.193.102:0/668576601 wait complete. 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: monmap: 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 Processor -- start 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- start start 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: init 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-b con 0x7f8e5008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-c con 0x7f8e5008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient(hunting): picked mon.noname-a con 0x7f8e5008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e50089ba0 con 0x7f8e5008d8e0 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5007fba0 con 0x7f8e5008b7f0 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5005da10 con 0x7f8e5008ce00 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e586b8640 10 monclient(hunting): authenticate will time out at 1758.114343s 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e55c2c640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f8e5008ce00 0x7f8e50089490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50084/0 (socket says 10.20.193.102:50084) 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.875+0000 7f8e55c2c640 1 -- 10.20.193.102:0/3220054084 learned_addr learned my addr 10.20.193.102:0/3220054084 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e55c2c640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e5008ce00 legacy=0x7f8e50089490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e56c2e640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6789/0 conn(0x7f8e5008d8e0 legacy=0x7f8e501a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977251534 0 0) 0x7f8e5005da10 con 0x7f8e5008ce00 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): my global_id is 84158 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e20003620 con 0x7f8e5008ce00 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564366077 0 0) 0x7f8e50089ba0 con 0x7f8e5008d8e0 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): my global_id is 34524 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e5005da10 con 0x7f8e5008d8e0 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e5642d640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.178:6789/0 conn(0x7f8e5008b7f0 legacy=0x7f8e50088d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3275947031 0 0) 0x7f8e20003620 con 0x7f8e5008ce00 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e50089ba0 con 0x7f8e5008ce00 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e44003490 con 0x7f8e5008ce00 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): dump: 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.890 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1169779166 0 0) 0x7f8e5005da10 con 0x7f8e5008d8e0 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e20003620 con 0x7f8e5008d8e0 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e4c005df0 con 0x7f8e5008d8e0 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient(hunting): dump: 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.891 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 141485102 0 0) 0x7f8e50089ba0 con 0x7f8e5008ce00 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.178:6789/0 conn(0x7f8e5008b7f0 legacy=0x7f8e50088d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6789/0 conn(0x7f8e5008d8e0 legacy=0x7f8e501a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 monclient: found mon.c 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e50058d70 con 0x7f8e5008ce00 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 5 monclient: authenticate success, global_id 84158 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 10 monclient: _renew_subs 2026-02-21T22:25:48.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8e50088a70 con 0x7f8e5008ce00 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 10 monclient: _renew_subs 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f8e5008a6a0 con 0x7f8e5008ce00 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e44003c30 con 0x7f8e5008ce00 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e44003fc0 con 0x7f8e5008ce00 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 10 monclient: dump: 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.893 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e54c2a640 4 set_mon_vals no callback set 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e500008e0 con 0x7f8e501a4580 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8e44004210 con 0x7f8e5008ce00 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7f8e4404c790 con 0x7f8e5008ce00 2026-02-21T22:25:48.894 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.879+0000 7f8e56c2e640 10 monclient: get_auth_request con 0x7f8e2003d650 auth_method 0 2026-02-21T22:25:48.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e5642d640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4580 legacy=0x7f8e50123e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498072655 0 0) 0x7f8e500008e0 con 0x7f8e501a4580 2026-02-21T22:25:48.896 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient(hunting): my global_id is 84161 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e50089ba0 con 0x7f8e501a4580 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1851659135 0 0) 0x7f8e50089ba0 con 0x7f8e501a4580 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e500008e0 con 0x7f8e501a4580 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e40002e70 con 0x7f8e501a4580 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient: dump: 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.897 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 259004424 0 0) 0x7f8e500008e0 con 0x7f8e501a4580 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e50090810 con 0x7f8e501a4580 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f8e400048e0 con 0x7f8e501a4580 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4580 legacy=0x7f8e50123e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.883+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4580 legacy=0x7f8e50123e60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:48.898 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e500909c0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e5642d640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4950 legacy=0x7f8e50124570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291768168 0 0) 0x7f8e500909c0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient(hunting): my global_id is 84164 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e500008e0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2729051120 0 0) 0x7f8e500008e0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e500909c0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e40004eb0 con 0x7f8e501a4950 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient: dump: 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.899 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2452859262 0 0) 0x7f8e500909c0 con 0x7f8e501a4950 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e501a4d20 con 0x7f8e501a4950 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 378362211) 0x7f8e40006380 con 0x7f8e501a4950 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4950 legacy=0x7f8e50124570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e36ffd640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e501a4950 legacy=0x7f8e50124570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 31707, 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:48.901 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 3, 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.902 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.903 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:48.904 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.905 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.906 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.907 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:48.908 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:48.909 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:48.910 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:48.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.178:6800/2549949439 conn(0x7f8e2003d650 legacy=0x7f8e2005db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 10 monclient: shutdown 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 >> v1:10.20.193.102:6790/0 conn(0x7f8e5008ce00 legacy=0x7f8e50089490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e56c2e640 1 -- 10.20.193.102:0/3220054084 reap_dead start 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e56c2e640 1 -- 10.20.193.102:0/3220054084 reap_dead start 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 shutdown_connections 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 >> 10.20.193.102:0/3220054084 conn(0x7f8e50091a50 msgr2=0x7f8e50125f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 shutdown_connections 2026-02-21T22:25:48.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.887+0000 7f8e586b8640 1 -- 10.20.193.102:0/3220054084 wait complete. 2026-02-21T22:25:48.912 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:48.912 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:25:48.913 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:25:48.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: get_monmap_and_config 2026-02-21T22:25:48.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: monmap: 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:48.974897+0000 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:48.974897+0000 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 1 Processor -- start 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 1 -- start start 2026-02-21T22:25:48.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: init 2026-02-21T22:25:48.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-a con 0x7f74c4085970 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-b con 0x7f74c4085d40 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-c con 0x7f74c41854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c408df30 con 0x7f74c4085970 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c408d900 con 0x7f74c41854c0 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c4056dd0 con 0x7f74c4085d40 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c9f58640 10 monclient(hunting): authenticate will time out at 1758.210985s 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c37fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f74c4085970 0x7f74c408e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41728/0 (socket says 10.20.193.102:41728) 2026-02-21T22:25:48.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.971+0000 7f74c37fe640 1 -- 10.20.193.102:0/1699883671 learned_addr learned my addr 10.20.193.102:0/1699883671 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c37fe640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.102:6789/0 conn(0x7f74c4085970 legacy=0x7f74c408e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c2ffd640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.102:6790/0 conn(0x7f74c4085d40 legacy=0x7f74c4183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363019528 0 0) 0x7f74c4056dd0 con 0x7f74c4085d40 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): my global_id is 84167 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a8003620 con 0x7f74c4085d40 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320352334 0 0) 0x7f74c408df30 con 0x7f74c4085970 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c3fff640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.178:6789/0 conn(0x7f74c41854c0 legacy=0x7f74c41a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient(hunting): my global_id is 34527 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c4056dd0 con 0x7f74c4085970 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3836252852 0 0) 0x7f74c4056dd0 con 0x7f74c4085970 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.102:6790/0 conn(0x7f74c4085d40 legacy=0x7f74c4183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.178:6789/0 conn(0x7f74c41854c0 legacy=0x7f74c41a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 monclient: found mon.noname-a 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c408d620 con 0x7f74c4085970 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b0003110 con 0x7f74c4085970 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: dump: 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 5 monclient: authenticate success, global_id 34527 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f74b00032b0 con 0x7f74c4085970 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 1 -- 10.20.193.102:0/1699883671 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b00049a0 con 0x7f74c4085970 2026-02-21T22:25:48.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c27fc640 10 monclient: dump: 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 4 set_mon_vals no callback set 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: shutdown 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:48.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- 10.20.193.102:0/1699883671 >> v1:10.20.193.102:6789/0 conn(0x7f74c4085970 legacy=0x7f74c408e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- 10.20.193.102:0/1699883671 shutdown_connections 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- 10.20.193.102:0/1699883671 >> 10.20.193.102:0/1699883671 conn(0x7f74c4054e40 msgr2=0x7f74c4092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- 10.20.193.102:0/1699883671 shutdown_connections 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- 10.20.193.102:0/1699883671 wait complete. 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: build_initial_monmap 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: monmap: 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 Processor -- start 2026-02-21T22:25:48.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- start start 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: init 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-b con 0x7f74c4085970 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-a con 0x7f74c4085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient(hunting): picked mon.noname-c con 0x7f74c41854c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c407f8f0 con 0x7f74c4085d40 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c407c8f0 con 0x7f74c4085970 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c4084e10 con 0x7f74c41854c0 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c9f58640 10 monclient(hunting): authenticate will time out at 1758.215641s 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c2ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f74c4085d40 0x7f74c407c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41744/0 (socket says 10.20.193.102:41744) 2026-02-21T22:25:48.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.975+0000 7f74c2ffd640 1 -- 10.20.193.102:0/353101167 learned_addr learned my addr 10.20.193.102:0/353101167 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c2ffd640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.102:6789/0 conn(0x7f74c4085d40 legacy=0x7f74c407c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c3fff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.102:6790/0 conn(0x7f74c41854c0 legacy=0x7f74c4184ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1691311688 0 0) 0x7f74c4084e10 con 0x7f74c41854c0 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): my global_id is 84170 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7494003620 con 0x7f74c41854c0 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c37fe640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c4085970 legacy=0x7f74c407bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660292313 0 0) 0x7f74c407f8f0 con 0x7f74c4085d40 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): my global_id is 34530 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c4084e10 con 0x7f74c4085d40 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2532045842 0 0) 0x7f7494003620 con 0x7f74c41854c0 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74c407f8f0 con 0x7f74c41854c0 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b8004910 con 0x7f74c41854c0 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): dump: 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.989 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738951788 0 0) 0x7f74c407c8f0 con 0x7f74c4085970 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): my global_id is 54328 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7494003620 con 0x7f74c4085970 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 20729083 0 0) 0x7f74c4084e10 con 0x7f74c4085d40 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74c407c8f0 con 0x7f74c4085d40 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b4002f00 con 0x7f74c4085d40 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient(hunting): dump: 2026-02-21T22:25:48.990 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 695291959 0 0) 0x7f74c407f8f0 con 0x7f74c41854c0 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c4085970 legacy=0x7f74c407bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.102:6789/0 conn(0x7f74c4085d40 legacy=0x7f74c407c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 monclient: found mon.c 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c4089a80 con 0x7f74c41854c0 2026-02-21T22:25:48.991 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: _finish_auth 0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 5 monclient: authenticate success, global_id 84170 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 10 monclient: _renew_subs 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f74c41852b0 con 0x7f74c41854c0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 10 monclient: _renew_subs 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f74c4084180 con 0x7f74c41854c0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f74b8004ed0 con 0x7f74c41854c0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b8005260 con 0x7f74c41854c0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c1ffb640 4 set_mon_vals no callback set 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 10 monclient: dump: 2026-02-21T22:25:48.992 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f74c407e4c0 con 0x7f74c407e0f0 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f74b8005400 con 0x7f74c41854c0 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7f74b804da80 con 0x7f74c41854c0 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.979+0000 7f74c2ffd640 10 monclient: get_auth_request con 0x7f749403d830 auth_method 0 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f74c37fe640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c407e0f0 legacy=0x7f74c41a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19493776 0 0) 0x7f74c407e4c0 con 0x7f74c407e0f0 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient(hunting): my global_id is 54331 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c407f8f0 con 0x7f74c407e0f0 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4149376066 0 0) 0x7f74c407f8f0 con 0x7f74c407e0f0 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74c407e4c0 con 0x7f74c407e0f0 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b0003110 con 0x7f74c407e0f0 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient: dump: 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.994 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1094052353 0 0) 0x7f74c407e4c0 con 0x7f74c407e0f0 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74c407f1e0 con 0x7f74c407e0f0 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f74b00033e0 con 0x7f74c407e0f0 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c407e0f0 legacy=0x7f74c41a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.983+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c407e0f0 legacy=0x7f74c41a58a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.987+0000 7f74c9f58640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.987+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f74c408f0e0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c37fe640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c41a5fb0 legacy=0x7f74c4134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458763225 0 0) 0x7f74c408f0e0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:48.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient(hunting): my global_id is 54334 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c407e4c0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1704206490 0 0) 0x7f74c407e4c0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74c408f0e0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74b00053a0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient: dump: 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:48.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1951571360 0 0) 0x7f74c408f0e0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f74c4062ce0 con 0x7f74c41a5fb0 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 3955724002) 0x7f74b0007580 con 0x7f74c41a5fb0 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c41a5fb0 legacy=0x7f74c4134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f749ffff640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6789/0 conn(0x7f74c41a5fb0 legacy=0x7f74c4134b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 40, 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 252633, 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:25:48.998 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 3, 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:48.999 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:25:49.000 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:25:49.001 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:49.002 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:49.003 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.004 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:25:49.005 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:25:49.006 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:25:49.006 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:25:49.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.178:6800/2549949439 conn(0x7f749403d830 legacy=0x7f749405dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 10 monclient: shutdown 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 >> v1:10.20.193.102:6790/0 conn(0x7f74c41854c0 legacy=0x7f74c4184ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c3fff640 1 -- 10.20.193.102:0/353101167 reap_dead start 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c3fff640 1 -- 10.20.193.102:0/353101167 reap_dead start 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 shutdown_connections 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 >> 10.20.193.102:0/353101167 conn(0x7f74c4054e40 msgr2=0x7f74c40891d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 shutdown_connections 2026-02-21T22:25:49.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:48.991+0000 7f74c9f58640 1 -- 10.20.193.102:0/353101167 wait complete. 2026-02-21T22:25:49.007 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:25:49.007 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2026-02-21T22:25:49.007 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:25:49.007 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2026-02-21T22:25:49.007 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2026-02-21T22:25:49.008 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-21T22:25:49.008 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:25:49.018 INFO:tasks.ceph.mon.c:Stopped 2026-02-21T22:25:49.019 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:25:49.019 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: get_monmap_and_config 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: build_initial_monmap 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: monmap: 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:49.127679+0000 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:49.127679+0000 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:49.127 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 1 Processor -- start 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 1 -- start start 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: init 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-b con 0x7f9ae008d290 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:49.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-a con 0x7f9ae008d660 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-c con 0x7f9ae0182eb0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae008e550 con 0x7f9ae008d660 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae008ef40 con 0x7f9ae0182eb0 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae008a750 con 0x7f9ae008d290 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae786c640 10 monclient(hunting): authenticate will time out at 1758.363537s 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae55e1640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f9ae008d290 legacy=0x7f9ae008b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae4de0640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ae008d660 0x7f9ae008da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41758/0 (socket says 10.20.193.102:41758) 2026-02-21T22:25:49.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.123+0000 7f9ae4de0640 1 -- 10.20.193.102:0/1675096891 learned_addr learned my addr 10.20.193.102:0/1675096891 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:49.134 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae4de0640 1 -- 10.20.193.102:0/1675096891 >> v1:10.20.193.102:6789/0 conn(0x7f9ae008d660 legacy=0x7f9ae008da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae5de2640 1 -- 10.20.193.102:0/1675096891 >> v1:10.20.193.178:6789/0 conn(0x7f9ae0182eb0 legacy=0x7f9ae01a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170601759 0 0) 0x7f9ae008e550 con 0x7f9ae008d660 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): my global_id is 34533 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac8003620 con 0x7f9ae008d660 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 891460908 0 0) 0x7f9ae008ef40 con 0x7f9ae0182eb0 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient(hunting): my global_id is 54337 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ae008e550 con 0x7f9ae0182eb0 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603093924 0 0) 0x7f9ac8003620 con 0x7f9ae008d660 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 >> v1:10.20.193.102:6790/0 conn(0x7f9ae008d290 legacy=0x7f9ae008b130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:25:49.135 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 >> v1:10.20.193.178:6789/0 conn(0x7f9ae0182eb0 legacy=0x7f9ae01a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 monclient: found mon.noname-a 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae008a470 con 0x7f9ae008d660 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: _finish_auth 0 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ad4003110 con 0x7f9ae008d660 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: dump: 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:49.136 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 5 monclient: authenticate success, global_id 34533 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9ad40033e0 con 0x7f9ae008d660 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 1 -- 10.20.193.102:0/1675096891 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9ad4004b50 con 0x7f9ae008d660 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9acffff640 10 monclient: dump: 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:49.137 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 4 set_mon_vals no callback set 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: shutdown 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- 10.20.193.102:0/1675096891 >> v1:10.20.193.102:6789/0 conn(0x7f9ae008d660 legacy=0x7f9ae008da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- 10.20.193.102:0/1675096891 shutdown_connections 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- 10.20.193.102:0/1675096891 >> 10.20.193.102:0/1675096891 conn(0x7f9ae0092a60 msgr2=0x7f9ae0092e60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- 10.20.193.102:0/1675096891 shutdown_connections 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- 10.20.193.102:0/1675096891 wait complete. 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: build_initial_monmap 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: monmap: 2026-02-21T22:25:49.138 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 Processor -- start 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- start start 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: init 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-b con 0x7f9ae008d290 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-c con 0x7f9ae008d660 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient(hunting): picked mon.noname-a con 0x7f9ae0182eb0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae0089a80 con 0x7f9ae0182eb0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae007ec80 con 0x7f9ae008d290 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae007ef60 con 0x7f9ae008d660 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae786c640 10 monclient(hunting): authenticate will time out at 1758.367692s 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae4de0640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f9ae008d660 legacy=0x7f9ae007bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae5de2640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ae0182eb0 0x7f9ae011fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41774/0 (socket says 10.20.193.102:41774) 2026-02-21T22:25:49.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.127+0000 7f9ae5de2640 1 -- 10.20.193.102:0/1498672196 learned_addr learned my addr 10.20.193.102:0/1498672196 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae5de2640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.102:6789/0 conn(0x7f9ae0182eb0 legacy=0x7f9ae011fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935077235 0 0) 0x7f9ae008e550 con 0x7f9ae0182eb0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): _negotiate 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): my global_id is 34536 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ab0003620 con 0x7f9ae0182eb0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 legacy=0x7f9ae007b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 legacy=0x7f9ae007b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 legacy=0x7f9ae007b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 1 --1- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 0x7f9ae007b420 :-1 s=OPENED pgs=375 gs=1 cs=1 l=1).handle_message read tag failed 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 1 --1- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 0x7f9ae007b420 :-1 s=OPENED pgs=375 gs=1 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.178:6789/0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2480023698 0 0) 0x7f9ab0003620 con 0x7f9ae0182eb0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ae008e550 con 0x7f9ae0182eb0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9adc0049a0 con 0x7f9ae0182eb0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): dump: 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:49.140 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 873725143 0 0) 0x7f9ae008e550 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.102:6790/0 conn(0x7f9ae008d660 legacy=0x7f9ae007bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6789/0 conn(0x7f9ae008d290 legacy=0x7f9ae007b420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: _finish_hunting 0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 monclient: found mon.a 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae007c240 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: _finish_auth 0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: _check_auth_tickets 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 5 monclient: authenticate success, global_id 34536 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _renew_subs 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae007da40 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _renew_subs 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9ae0085ce0 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9adc004c20 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9adc005270 con 0x7f9ae0182eb0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:49.141 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 10 monclient: dump: 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acf7fe640 4 set_mon_vals no callback set 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae005cec0 con 0x7f9ae0182eb0 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9adc005530 con 0x7f9ae0182eb0 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7f9adc04db70 con 0x7f9ae0182eb0 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.131+0000 7f9ae55e1640 10 monclient: get_auth_request con 0x7f9ab003d320 auth_method 0 2026-02-21T22:25:49.142 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.135+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9ae0089a80 con 0x7f9ae0182eb0 2026-02-21T22:25:49.143 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.135+0000 7f9acdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:49.143 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.135+0000 7f9acdffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:49.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:49.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:49.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:49.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9ae007cee0 con 0x7f9ae0182eb0 2026-02-21T22:25:49.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9acdffb640 1 -- 10.20.193.102:0/1498672196 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2915026968) 0x7f9adc018b80 con 0x7f9ae0182eb0 2026-02-21T22:25:49.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9acdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:49.255 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9acdffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:49.255 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:49.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9ab003d320 legacy=0x7f9ab005d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 10 monclient: shutdown 2026-02-21T22:25:49.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:49.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 >> v1:10.20.193.102:6789/0 conn(0x7f9ae0182eb0 legacy=0x7f9ae011fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:49.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 shutdown_connections 2026-02-21T22:25:49.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 >> 10.20.193.102:0/1498672196 conn(0x7f9ae0092a60 msgr2=0x7f9ae01a3290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:49.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 shutdown_connections 2026-02-21T22:25:49.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:49.251+0000 7f9ae786c640 1 -- 10.20.193.102:0/1498672196 wait complete. 2026-02-21T22:25:52.265 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:52.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.323+0000 7f2171b06640 10 monclient: get_monmap_and_config 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.323+0000 7f2171b06640 10 monclient: build_initial_monmap 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.323+0000 7f2171b06640 10 monclient: monmap: 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:52.328376+0000 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:52.328376+0000 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:52.327 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:52.328 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:52.328 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.323+0000 7f2171b06640 1 Processor -- start 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- start start 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient: init 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-c con 0x7f216c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:52.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-b con 0x7f216c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-a con 0x7f216c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c086a10 con 0x7f216c08d8e0 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c08b130 con 0x7f216c08ce00 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c08bd60 con 0x7f216c08b7f0 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216affd640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f216c08b7f0 legacy=0x7f216c181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient(hunting): authenticate will time out at 1761.564501s 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 0x7f216c1a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:45690/0 (socket says 10.20.193.102:45690) 2026-02-21T22:25:52.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216bfff640 1 -- 10.20.193.102:0/2420873390 learned_addr learned my addr 10.20.193.102:0/2420873390 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216bfff640 1 -- 10.20.193.102:0/2420873390 >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 legacy=0x7f216c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216b7fe640 1 -- 10.20.193.102:0/2420873390 >> v1:10.20.193.178:6789/0 conn(0x7f216c08ce00 legacy=0x7f216c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133198770 0 0) 0x7f216c086a10 con 0x7f216c08d8e0 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient(hunting): my global_id is 34539 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2150003620 con 0x7f216c08d8e0 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1052779799 0 0) 0x7f2150003620 con 0x7f216c08d8e0 2026-02-21T22:25:52.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 >> v1:10.20.193.102:6790/0 conn(0x7f216c08b7f0 legacy=0x7f216c181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 >> v1:10.20.193.178:6789/0 conn(0x7f216c08ce00 legacy=0x7f216c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 monclient: found mon.noname-a 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c08b370 con 0x7f216c08d8e0 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 5 monclient: authenticate success, global_id 34539 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f21600046a0 con 0x7f216c08d8e0 2026-02-21T22:25:52.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: dump: 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:52.337 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2160004930 con 0x7f216c08d8e0 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 1 -- 10.20.193.102:0/2420873390 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2160006060 con 0x7f216c08d8e0 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f216a7fc640 10 monclient: dump: 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.338 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient: get_monmap_and_config success 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 4 set_mon_vals no callback set 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 10 monclient: shutdown 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- 10.20.193.102:0/2420873390 >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 legacy=0x7f216c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- 10.20.193.102:0/2420873390 shutdown_connections 2026-02-21T22:25:52.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- 10.20.193.102:0/2420873390 >> 10.20.193.102:0/2420873390 conn(0x7f216c091a50 msgr2=0x7f216c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- 10.20.193.102:0/2420873390 shutdown_connections 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.327+0000 7f2171b06640 1 -- 10.20.193.102:0/2420873390 wait complete. 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: build_initial_monmap 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: monmap: 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:25:52.340 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 Processor -- start 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- start start 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: init 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-b con 0x7f216c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-c con 0x7f216c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient(hunting): picked mon.noname-a con 0x7f216c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c0852e0 con 0x7f216c08d8e0 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c080ca0 con 0x7f216c08b7f0 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f216c05d990 con 0x7f216c08ce00 2026-02-21T22:25:52.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:52.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient(hunting): authenticate will time out at 1761.568701s 2026-02-21T22:25:52.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f216affd640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f216c08ce00 legacy=0x7f216c192150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:52.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f216bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 0x7f216c193870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:45698/0 (socket says 10.20.193.102:45698) 2026-02-21T22:25:52.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f216bfff640 1 -- 10.20.193.102:0/1831363381 learned_addr learned my addr 10.20.193.102:0/1831363381 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:52.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f216bfff640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 legacy=0x7f216c193870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694043341 0 0) 0x7f216c08b370 con 0x7f216c08d8e0 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): _negotiate 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f216b7fe640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.178:6789/0 conn(0x7f216c08b7f0 legacy=0x7f216c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): my global_id is 34542 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c003620 con 0x7f216c08d8e0 2026-02-21T22:25:52.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3531619104 0 0) 0x7f213c003620 con 0x7f216c08d8e0 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f216c08b370 con 0x7f216c08d8e0 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f21600046a0 con 0x7f216c08d8e0 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): dump: 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:52.346 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 244360170 0 0) 0x7f216c08b370 con 0x7f216c08d8e0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.102:6790/0 conn(0x7f216c08ce00 legacy=0x7f216c192150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.178:6789/0 conn(0x7f216c08b7f0 legacy=0x7f216c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: _finish_hunting 0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 monclient: found mon.a 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c058d10 con 0x7f216c08d8e0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: _finish_auth 0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: _check_auth_tickets 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 5 monclient: authenticate success, global_id 34542 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _renew_subs 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f216c0809c0 con 0x7f216c08d8e0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _renew_subs 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f216c151bd0 con 0x7f216c08d8e0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f21600048c0 con 0x7f216c08d8e0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2160006680 con 0x7f216c08d8e0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2147fff640 10 monclient: dump: 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:52.348 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.331+0000 7f2169ffb640 4 set_mon_vals no callback set 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2171b06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2171b06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2171b06640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2160012d10 con 0x7f216c08d8e0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216c0008e0 con 0x7f216c08d8e0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(491..491 src has 112..491) ==== 3931+0+0 (unknown 748918637 0 0) 0x7f216004dba0 con 0x7f216c08d8e0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f216b7fe640 10 monclient: get_auth_request con 0x7f213c03d390 auth_method 0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f216c0852e0 con 0x7f216c08d8e0 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2147fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:25:52.349 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.335+0000 7f2147fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:25:52.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:25:52.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:52.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:25:52.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f216c090810 con 0x7f216c08d8e0 2026-02-21T22:25:52.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2147fff640 1 -- 10.20.193.102:0/1831363381 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3834925560) 0x7f2160018ba0 con 0x7f216c08d8e0 2026-02-21T22:25:52.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2147fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:25:52.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2147fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:25:52.458 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:25:52.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.178:6800/2549949439 conn(0x7f213c03d390 legacy=0x7f213c05d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:52.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 10 monclient: shutdown 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 >> v1:10.20.193.102:6789/0 conn(0x7f216c08d8e0 legacy=0x7f216c193870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 shutdown_connections 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 >> 10.20.193.102:0/1831363381 conn(0x7f216c091a50 msgr2=0x7f216c05a060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 shutdown_connections 2026-02-21T22:25:52.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:52.455+0000 7f2171b06640 1 -- 10.20.193.102:0/1831363381 wait complete. 2026-02-21T22:25:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:25:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:25:53.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:53.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:55.468 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:25:55.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: get_monmap_and_config 2026-02-21T22:25:55.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: build_initial_monmap 2026-02-21T22:25:55.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: monmap: 2026-02-21T22:25:55.532 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:25:55.532 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:25:55.534501+0000 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:25:55.534501+0000 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 1 Processor -- start 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 1 -- start start 2026-02-21T22:25:55.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: init 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: _reopen_session rank -1 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-c con 0x7f555c08e5b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-b con 0x7f555c059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-a con 0x7f555c08f090 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c057ed0 con 0x7f555c08f090 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c0581b0 con 0x7f555c08e5b0 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5561569640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c183fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c058490 con 0x7f555c059150 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f5563ff5640 10 monclient(hunting): authenticate will time out at 1764.770749s 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f556256b640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f555c08f090 0x7f555c1a5710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:45702/0 (socket says 10.20.193.102:45702) 2026-02-21T22:25:55.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.531+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 learned_addr learned my addr 10.20.193.102:0/2055773662 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:25:55.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.535+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f555c08f090 legacy=0x7f555c1a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:55.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.535+0000 7f5561d6a640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f555c08e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:55.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:55.735+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c183fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:56.139 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:56.139+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c183fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:56.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:56.943+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c183fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): tick 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 monclient(hunting): continuing hunt 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c183fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f555c08e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f555c08f090 legacy=0x7f555c1a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:25:58.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f55400038c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f555c08e5b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f555c059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c05e000 con 0x7f55400038c0 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c058490 con 0x7f555c08e5b0 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554006a5b0 con 0x7f555c059150 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f554b7fe640 10 monclient(hunting): _renew_subs 2026-02-21T22:25:58.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.531+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:58.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.535+0000 7f5561d6a640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f55400038c0 legacy=0x7f5540023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:58.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.535+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f5540045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:25:58.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:25:58.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:25:58.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:58.735+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:59.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:59.139+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:25:59.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:25:59.943+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): tick 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 monclient(hunting): continuing hunt 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f5540045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:01.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f55400038c0 legacy=0x7f5540023ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f554006a8f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f555c059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f555c08e5b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554006b850 con 0x7f554006a8f0 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554006a5b0 con 0x7f555c08e5b0 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5540003cd0 con 0x7f555c059150 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f554b7fe640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:01.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.531+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:01.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.535+0000 7f5561d6a640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f554006a8f0 legacy=0x7f554006acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:01.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.535+0000 7f556256b640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f5540045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:01.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:01.735+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:02.140 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:02.139+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:02.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:02.943+0000 7f5561569640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648576401 0 0) 0x7f554006a5b0 con 0x7f555c08e5b0 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): my global_id is 54343 2026-02-21T22:26:03.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5544003620 con 0x7f555c08e5b0 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681066713 0 0) 0x7f554006b850 con 0x7f554006a8f0 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient(hunting): my global_id is 34545 2026-02-21T22:26:03.347 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f554006a5b0 con 0x7f554006a8f0 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2171582648 0 0) 0x7f554006a5b0 con 0x7f554006a8f0 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f5540066d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.178:6789/0 conn(0x7f555c08e5b0 legacy=0x7f5540045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 monclient: found mon.noname-a 2026-02-21T22:26:03.351 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540004eb0 con 0x7f554006a8f0 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f554c0030c0 con 0x7f554006a8f0 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5563ff5640 5 monclient: authenticate success, global_id 34545 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5563ff5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.352 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5560d68640 10 monclient: dump: 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.353 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.343+0000 7f5563ff5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f554c003310 con 0x7f554006a8f0 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: get_monmap_and_config success 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 4 set_mon_vals no callback set 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 1 -- 10.20.193.102:0/2055773662 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f554c003710 con 0x7f554006a8f0 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5560d68640 10 monclient: dump: 2026-02-21T22:26:03.354 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: shutdown 2026-02-21T22:26:03.355 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- 10.20.193.102:0/2055773662 >> v1:10.20.193.102:6789/0 conn(0x7f554006a8f0 legacy=0x7f554006acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- 10.20.193.102:0/2055773662 shutdown_connections 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- 10.20.193.102:0/2055773662 >> 10.20.193.102:0/2055773662 conn(0x7f555c056d40 msgr2=0x7f555c093bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- 10.20.193.102:0/2055773662 shutdown_connections 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- 10.20.193.102:0/2055773662 wait complete. 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: build_initial_monmap 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: monmap: 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:26:03.356 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 Processor -- start 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- start start 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: init 2026-02-21T22:26:03.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: _reopen_session rank -1 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-b con 0x7f554006a8f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-c con 0x7f555c059150 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient(hunting): picked mon.noname-a con 0x7f555c08e5b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c140b50 con 0x7f555c08e5b0 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c05e900 con 0x7f554006a8f0 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555c12cf60 con 0x7f555c059150 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5563ff5640 10 monclient(hunting): authenticate will time out at 1772.586157s 2026-02-21T22:26:03.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f5561569640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c21b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f556256b640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f555c08e5b0 0x7f555c23cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60502/0 (socket says 10.20.193.102:60502) 2026-02-21T22:26:03.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.347+0000 7f556256b640 1 -- 10.20.193.102:0/1692214776 learned_addr learned my addr 10.20.193.102:0/1692214776 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:26:03.360 INFO:tasks.rados.rados.0.trial178.stdout:2384: done (1 left) 2026-02-21T22:26:03.360 INFO:tasks.rados.rados.0.trial178.stdout:2385: write oid 29 current snap is 239 2026-02-21T22:26:03.360 INFO:tasks.rados.rados.0.trial178.stdout:2385: seq_num 835 ranges {433845=715392,1662582=371263} 2026-02-21T22:26:03.360 INFO:tasks.rados.rados.0.trial178.stdout:2385: writing trial17818531-29 from 433845 to 1149237 tid 1 2026-02-21T22:26:03.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f556256b640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.102:6789/0 conn(0x7f555c08e5b0 legacy=0x7f555c23cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116538254 0 0) 0x7f555c140b50 con 0x7f555c08e5b0 2026-02-21T22:26:03.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.364 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): my global_id is 34548 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5561d6a640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.178:6789/0 conn(0x7f554006a8f0 legacy=0x7f555c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f552c003620 con 0x7f555c08e5b0 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1555408108 0 0) 0x7f552c003620 con 0x7f555c08e5b0 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f555c140b50 con 0x7f555c08e5b0 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5558004490 con 0x7f555c08e5b0 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.365 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): dump: 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.366 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556380135 0 0) 0x7f555c05e900 con 0x7f554006a8f0 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient(hunting): my global_id is 54346 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f552c003620 con 0x7f554006a8f0 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2230227820 0 0) 0x7f555c140b50 con 0x7f555c08e5b0 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.102:6790/0 conn(0x7f555c059150 legacy=0x7f555c21b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.178:6789/0 conn(0x7f554006a8f0 legacy=0x7f555c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.367 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 monclient: found mon.a 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c12e0f0 con 0x7f555c08e5b0 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 5 monclient: authenticate success, global_id 34548 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _renew_subs 2026-02-21T22:26:03.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f555c12d850 con 0x7f555c08e5b0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _renew_subs 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f555c13d030 con 0x7f555c08e5b0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f55580060a0 con 0x7f555c08e5b0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5558006430 con 0x7f555c08e5b0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554bfff640 4 set_mon_vals no callback set 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 10 monclient: dump: 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.370 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c0008e0 con 0x7f555c08e5b0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f55580066c0 con 0x7f555c08e5b0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(492..492 src has 112..492) ==== 3975+0+0 (unknown 336452335 0 0) 0x7f555804dad0 con 0x7f555c08e5b0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.351+0000 7f5561d6a640 10 monclient: get_auth_request con 0x7f552c03d4d0 auth_method 0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.355+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5558018ab0 con 0x7f555c08e5b0 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.355+0000 7f554a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.371 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.355+0000 7f554a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:26:03.372 INFO:tasks.rados.rados.0.trial178.stdout:2385: writing trial17818531-29 from 1662582 to 2033845 tid 2 2026-02-21T22:26:03.373 INFO:tasks.rados.rados.0.trial178.stdout:2386: write (excl) oid 9 current snap is 239 2026-02-21T22:26:03.373 INFO:tasks.rados.rados.0.trial178.stdout:2386: seq_num 836 ranges {450652=790641,1721064=329588} 2026-02-21T22:26:03.373 INFO:tasks.rados.rados.0.trial178.stdout:2386: writing trial17818531-9 from 450652 to 1241293 tid 1 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2386: writing trial17818531-9 from 1721064 to 2050652 tid 2 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2387: delete oid 34 current snap is 239 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2385: finishing write tid 1 to trial17818531-29 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2385: finishing write tid 2 to trial17818531-29 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2385: finishing write tid 3 to trial17818531-29 2026-02-21T22:26:03.376 INFO:tasks.rados.rados.0.trial178.stdout:2385: finishing write tid 4 to trial17818531-29 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:2385: oid 29 updating version 0 to 796 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:2385: oid 29 version 796 is already newer than 795 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:2385: oid 29 updating version 796 to 797 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 797 (ObjNum 835 snap 239 seq_num 835) dirty exists 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:2385: left oid 29 (ObjNum 835 snap 239 seq_num 835) 2026-02-21T22:26:03.377 INFO:tasks.rados.rados.0.trial178.stdout:2386: finishing write tid 1 to trial17818531-9 2026-02-21T22:26:03.378 INFO:tasks.rados.rados.0.trial178.stdout:2386: finishing write tid 2 to trial17818531-9 2026-02-21T22:26:03.378 INFO:tasks.rados.rados.0.trial178.stdout:2383: done (3 left) 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2385: done (2 left) 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2387: done (1 left) 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2388: read oid 39 snap -1 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2388: expect deleted 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2389: rollback oid 32 current snap is 239 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 227 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2390: read oid 27 snap -1 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2390: expect (ObjNum 822 snap 232 seq_num 822) 2026-02-21T22:26:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2391: snap_remove snap 229 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f555c092c10 con 0x7f555c08e5b0 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f554a7fc640 1 -- 10.20.193.102:0/1692214776 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 425301407) 0x7f555801dd00 con 0x7f555c08e5b0 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f554a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f554a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:26:03.472 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.178:6800/2549949439 conn(0x7f552c03d4d0 legacy=0x7f552c05d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 10 monclient: shutdown 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 >> v1:10.20.193.102:6789/0 conn(0x7f555c08e5b0 legacy=0x7f555c23cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 shutdown_connections 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 >> 10.20.193.102:0/1692214776 conn(0x7f555c056d40 msgr2=0x7f555c141cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 shutdown_connections 2026-02-21T22:26:03.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.471+0000 7f5563ff5640 1 -- 10.20.193.102:0/1692214776 wait complete. 2026-02-21T22:26:03.484 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:26:03.484 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:26:03.484 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:26:03.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: get_monmap_and_config 2026-02-21T22:26:03.553 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: build_initial_monmap 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: monmap: 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:26:03.555386+0000 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:26:03.555386+0000 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:26:03.554 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 1 Processor -- start 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 1 -- start start 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: init 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: _reopen_session rank -1 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-a con 0x7ffa7c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-c con 0x7ffa7c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-b con 0x7ffa7c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c086a10 con 0x7ffa7c08ce00 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c08b130 con 0x7ffa7c08b7f0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c08bd60 con 0x7ffa7c08d8e0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa8226e640 10 monclient(hunting): authenticate will time out at 1772.791441s 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa7bfff640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7ffa7c08d8e0 legacy=0x7ffa7c1a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa7b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 0x7ffa7c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60508/0 (socket says 10.20.193.102:60508) 2026-02-21T22:26:03.555 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.551+0000 7ffa7b7fe640 1 -- 10.20.193.102:0/1276142604 learned_addr learned my addr 10.20.193.102:0/1276142604 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:26:03.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7b7fe640 1 -- 10.20.193.102:0/1276142604 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 legacy=0x7ffa7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181040052 0 0) 0x7ffa7c086a10 con 0x7ffa7c08ce00 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient(hunting): my global_id is 34551 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa60003620 con 0x7ffa7c08ce00 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7affd640 1 -- 10.20.193.102:0/1276142604 >> v1:10.20.193.178:6789/0 conn(0x7ffa7c08b7f0 legacy=0x7ffa7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2797669618 0 0) 0x7ffa60003620 con 0x7ffa7c08ce00 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 >> v1:10.20.193.102:6790/0 conn(0x7ffa7c08d8e0 legacy=0x7ffa7c1a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 >> v1:10.20.193.178:6789/0 conn(0x7ffa7c08b7f0 legacy=0x7ffa7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 monclient: found mon.noname-a 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c08b370 con 0x7ffa7c08ce00 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa68003190 con 0x7ffa7c08ce00 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 5 monclient: authenticate success, global_id 34551 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: dump: 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffa68003420 con 0x7ffa7c08ce00 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 1 -- 10.20.193.102:0/1276142604 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa68004b50 con 0x7ffa7c08ce00 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7a7fc640 10 monclient: dump: 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: get_monmap_and_config success 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 4 set_mon_vals no callback set 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: shutdown 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- 10.20.193.102:0/1276142604 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 legacy=0x7ffa7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- 10.20.193.102:0/1276142604 shutdown_connections 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- 10.20.193.102:0/1276142604 >> 10.20.193.102:0/1276142604 conn(0x7ffa7c091a50 msgr2=0x7ffa7c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- 10.20.193.102:0/1276142604 shutdown_connections 2026-02-21T22:26:03.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- 10.20.193.102:0/1276142604 wait complete. 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: build_initial_monmap 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: monmap: 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 Processor -- start 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- start start 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: init 2026-02-21T22:26:03.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: _reopen_session rank -1 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-b con 0x7ffa7c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-a con 0x7ffa7c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient(hunting): picked mon.noname-c con 0x7ffa7c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c082c70 con 0x7ffa7c08ce00 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c082f50 con 0x7ffa7c08b7f0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c05d9e0 con 0x7ffa7c08d8e0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa8226e640 10 monclient(hunting): authenticate will time out at 1772.795859s 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7bfff640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7ffa7c08d8e0 legacy=0x7ffa7c1953a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7affd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 0x7ffa7c084740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60514/0 (socket says 10.20.193.102:60514) 2026-02-21T22:26:03.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.555+0000 7ffa7affd640 1 -- 10.20.193.102:0/1897847669 learned_addr learned my addr 10.20.193.102:0/1897847669 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:26:03.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa7affd640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 legacy=0x7ffa7c084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa7b7fe640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.178:6789/0 conn(0x7ffa7c08b7f0 legacy=0x7ffa7c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869564360 0 0) 0x7ffa7c082c70 con 0x7ffa7c08ce00 2026-02-21T22:26:03.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): my global_id is 34554 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa4c003620 con 0x7ffa7c08ce00 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2336837351 0 0) 0x7ffa7c082f50 con 0x7ffa7c08b7f0 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): my global_id is 54352 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa7c082c70 con 0x7ffa7c08b7f0 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1370025372 0 0) 0x7ffa4c003620 con 0x7ffa7c08ce00 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa7c082f50 con 0x7ffa7c08ce00 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa6c003340 con 0x7ffa7c08ce00 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): dump: 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.563 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3391917053 0 0) 0x7ffa7c082f50 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6790/0 conn(0x7ffa7c08d8e0 legacy=0x7ffa7c1953a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.178:6789/0 conn(0x7ffa7c08b7f0 legacy=0x7ffa7c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 monclient: found mon.a 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c081780 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 5 monclient: authenticate success, global_id 34554 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 10 monclient: _renew_subs 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c084e50 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 10 monclient: _renew_subs 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ffa7c196c90 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffa6c0039c0 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa6c003d50 con 0x7ffa7c08ce00 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 10 monclient: dump: 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.564 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa79ffb640 4 set_mon_vals no callback set 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffa7c0008e0 con 0x7ffa7c07a370 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ffa6c003fa0 con 0x7ffa7c08ce00 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(492..492 src has 112..492) ==== 3975+0+0 (unknown 336452335 0 0) 0x7ffa6c04c6b0 con 0x7ffa7c08ce00 2026-02-21T22:26:03.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.559+0000 7ffa7bfff640 10 monclient: get_auth_request con 0x7ffa4c03d240 auth_method 0 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa7b7fe640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07a370 legacy=0x7ffa7c07a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704567119 0 0) 0x7ffa7c0008e0 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient(hunting): my global_id is 34557 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa7c082f50 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1163813018 0 0) 0x7ffa7c082f50 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa7c0008e0 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa68005000 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient: dump: 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2573311881 0 0) 0x7ffa7c0008e0 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa7c083190 con 0x7ffa7c07a370 2026-02-21T22:26:03.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ffa68005090 con 0x7ffa7c07a370 2026-02-21T22:26:03.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:26:03.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07a370 legacy=0x7ffa7c07a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.563+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07a370 legacy=0x7ffa7c07a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:26:03.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.567+0000 7ffa8226e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:26:03.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.567+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffa7c08a870 con 0x7ffa7c07ae50 2026-02-21T22:26:03.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:03.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:03.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.567+0000 7ffa7b7fe640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07ae50 legacy=0x7ffa7c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295025082 0 0) 0x7ffa7c08a870 con 0x7ffa7c07ae50 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient(hunting): my global_id is 34560 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa7c0008e0 con 0x7ffa7c07ae50 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2285335003 0 0) 0x7ffa7c0008e0 con 0x7ffa7c07ae50 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffa7c08a870 con 0x7ffa7c07ae50 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ffa68005740 con 0x7ffa7c07ae50 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient: dump: 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.571 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3910628160 0 0) 0x7ffa7c08a870 con 0x7ffa7c07ae50 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffa7c07b250 con 0x7ffa7c07ae50 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 2393581487) 0x7ffa68008ae0 con 0x7ffa7c07ae50 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07ae50 legacy=0x7ffa7c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa57fff640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c07ae50 legacy=0x7ffa7c197f90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:26:03.572 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 44, 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 512440, 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.573 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:26:03.574 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.575 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.576 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.577 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:26:03.578 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.178:6800/2549949439 conn(0x7ffa4c03d240 legacy=0x7ffa4c05d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 10 monclient: shutdown 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 >> v1:10.20.193.102:6789/0 conn(0x7ffa7c08ce00 legacy=0x7ffa7c084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa7bfff640 1 -- 10.20.193.102:0/1897847669 reap_dead start 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 shutdown_connections 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 >> 10.20.193.102:0/1897847669 conn(0x7ffa7c091a50 msgr2=0x7ffa7c05a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 shutdown_connections 2026-02-21T22:26:03.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.571+0000 7ffa8226e640 1 -- 10.20.193.102:0/1897847669 wait complete. 2026-02-21T22:26:03.580 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:26:03.580 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2026-02-21T22:26:03.580 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: get_monmap_and_config 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: build_initial_monmap 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: monmap: 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:26:03.643211+0000 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:26:03.643211+0000 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.641 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 1 Processor -- start 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 1 -- start start 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: init 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: _reopen_session rank -1 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-c con 0x7f8140091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-a con 0x7f81400909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-b con 0x7f81401854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8140057a20 con 0x7f81400909a0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8140091d20 con 0x7f8140091240 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8140057740 con 0x7f81401854b0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f81484ee640 10 monclient(hunting): authenticate will time out at 1772.879140s 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f8146a64640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f81401854b0 legacy=0x7f81401a5890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.642 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f8145a62640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 0x7f8140183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60548/0 (socket says 10.20.193.102:60548) 2026-02-21T22:26:03.643 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.639+0000 7f8145a62640 1 -- 10.20.193.102:0/1480950698 learned_addr learned my addr 10.20.193.102:0/1480950698 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145a62640 1 -- 10.20.193.102:0/1480950698 >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 legacy=0x7f8140183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8146263640 1 -- 10.20.193.102:0/1480950698 >> v1:10.20.193.178:6789/0 conn(0x7f8140091240 legacy=0x7f8140091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495775208 0 0) 0x7f8140057a20 con 0x7f81400909a0 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): my global_id is 34563 2026-02-21T22:26:03.644 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8128003620 con 0x7f81400909a0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101721267 0 0) 0x7f8140091d20 con 0x7f8140091240 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient(hunting): my global_id is 54355 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8140057a20 con 0x7f8140091240 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443843035 0 0) 0x7f8128003620 con 0x7f81400909a0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 >> v1:10.20.193.102:6790/0 conn(0x7f81401854b0 legacy=0x7f81401a5890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 >> v1:10.20.193.178:6789/0 conn(0x7f8140091240 legacy=0x7f8140091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 monclient: found mon.noname-a 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140090dc0 con 0x7f81400909a0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8134003110 con 0x7f81400909a0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.645 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: dump: 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 5 monclient: authenticate success, global_id 34563 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f81340033e0 con 0x7f81400909a0 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 1 -- 10.20.193.102:0/1480950698 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8134004b50 con 0x7f81400909a0 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8145261640 10 monclient: dump: 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: get_monmap_and_config success 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 4 set_mon_vals no callback set 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: shutdown 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- 10.20.193.102:0/1480950698 >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 legacy=0x7f8140183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- 10.20.193.102:0/1480950698 shutdown_connections 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- 10.20.193.102:0/1480950698 >> 10.20.193.102:0/1480950698 conn(0x7f814005e660 msgr2=0x7f8140095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- 10.20.193.102:0/1480950698 shutdown_connections 2026-02-21T22:26:03.646 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- 10.20.193.102:0/1480950698 wait complete. 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: build_initial_monmap 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: monmap: 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 Processor -- start 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- start start 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: init 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: _reopen_session rank -1 2026-02-21T22:26:03.647 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-a con 0x7f81400909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-b con 0x7f8140091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient(hunting): picked mon.noname-c con 0x7f81401854b0 addr v1:10.20.193.102:6790/0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f814012c1a0 con 0x7f81400909a0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81401133f0 con 0x7f8140091240 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8140112580 con 0x7f81401854b0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient(hunting): _renew_subs 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f81484ee640 10 monclient(hunting): authenticate will time out at 1772.883672s 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8146a64640 1 -- >> v1:10.20.193.102:6790/0 conn(0x7f81401854b0 legacy=0x7f8140174570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.102:6790/0 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8146263640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 0x7f814012ba90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60558/0 (socket says 10.20.193.102:60558) 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.643+0000 7f8146263640 1 -- 10.20.193.102:0/108304876 learned_addr learned my addr 10.20.193.102:0/108304876 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:26:03.648 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8146263640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 legacy=0x7f814012ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8145a62640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f8140091240 legacy=0x7f8140172e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260022982 0 0) 0x7f814012c1a0 con 0x7f81400909a0 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): my global_id is 34566 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8110003620 con 0x7f81400909a0 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2057249559 0 0) 0x7f8110003620 con 0x7f81400909a0 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f814012c1a0 con 0x7f81400909a0 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954646709 0 0) 0x7f81401133f0 con 0x7f8140091240 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.649 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): my global_id is 54358 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8110003620 con 0x7f8140091240 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8130003490 con 0x7f81400909a0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): dump: 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 709719432 0 0) 0x7f814012c1a0 con 0x7f81400909a0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.102:6790/0 conn(0x7f81401854b0 legacy=0x7f8140174570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f8140091240 legacy=0x7f8140172e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 monclient: found mon.a 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140111090 con 0x7f81400909a0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: _finish_auth 0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 5 monclient: authenticate success, global_id 34566 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 10 monclient: _renew_subs 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8140112b50 con 0x7f81400909a0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 10 monclient: _renew_subs 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8140111bf0 con 0x7f81400909a0 2026-02-21T22:26:03.650 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8130003790 con 0x7f81400909a0 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8130003de0 con 0x7f81400909a0 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 10 monclient: dump: 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8144a60640 4 set_mon_vals no callback set 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f81300040a0 con 0x7f81400909a0 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f814008f2d0 con 0x7f814012ea80 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(492..492 src has 112..492) ==== 3975+0+0 (unknown 336452335 0 0) 0x7f813004c7f0 con 0x7f81400909a0 2026-02-21T22:26:03.651 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.647+0000 7f8146a64640 10 monclient: get_auth_request con 0x7f811003d490 auth_method 0 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8145a62640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814012ea80 legacy=0x7f814008d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895594305 0 0) 0x7f814008f2d0 con 0x7f814012ea80 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient(hunting): my global_id is 54361 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f814012c1a0 con 0x7f814012ea80 2026-02-21T22:26:03.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3854746227 0 0) 0x7f814012c1a0 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f814008f2d0 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8134003540 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient: dump: 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3270995016 0 0) 0x7f814008f2d0 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f814012ee50 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f8134004aa0 con 0x7f814012ea80 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814012ea80 legacy=0x7f814008d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.651+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814012ea80 legacy=0x7f814008d190 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:26:03.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.655+0000 7f81484ee640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:26:03.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.655+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f814005eb70 con 0x7f814008d8a0 2026-02-21T22:26:03.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8145a62640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814008d8a0 legacy=0x7f8140176ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506817884 0 0) 0x7f814005eb70 con 0x7f814008d8a0 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient(hunting): my global_id is 54364 2026-02-21T22:26:03.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f814008f2d0 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3893204833 0 0) 0x7f814008f2d0 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f814005eb70 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8134005000 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient: dump: 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2775254931 0 0) 0x7f814005eb70 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8140111ee0 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4503 (unknown 0 0 1344615979) 0x7f8134007260 con 0x7f814008d8a0 2026-02-21T22:26:03.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814008d8a0 legacy=0x7f8140176ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8126ffd640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6789/0 conn(0x7f814008d8a0 legacy=0x7f8140176ca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 44, 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 267308, 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: 1 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.663 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:26:03.664 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.665 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.666 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.667 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.178:6800/2549949439 conn(0x7f811003d490 legacy=0x7f811005d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 10 monclient: shutdown 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 >> v1:10.20.193.102:6789/0 conn(0x7f81400909a0 legacy=0x7f814012ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8146a64640 1 -- 10.20.193.102:0/108304876 reap_dead start 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f8146a64640 1 -- 10.20.193.102:0/108304876 reap_dead start 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 shutdown_connections 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 >> 10.20.193.102:0/108304876 conn(0x7f814005e660 msgr2=0x7f814008b870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 shutdown_connections 2026-02-21T22:26:03.668 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:26:03.659+0000 7f81484ee640 1 -- 10.20.193.102:0/108304876 wait complete. 2026-02-21T22:26:03.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:26:03.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2391: done (4 left) 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2392: read oid 49 snap -1 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2392: expect (ObjNum 804 snap 225 seq_num 804) 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2393: write (excl) oid 10 current snap is 239 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2386: finishing write tid 3 to trial17818531-9 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2386: finishing write tid 4 to trial17818531-9 2026-02-21T22:26:04.349 INFO:tasks.rados.rados.0.trial178.stdout:2386: oid 9 updating version 0 to 1138 2026-02-21T22:26:04.350 INFO:tasks.rados.rados.0.trial178.stdout:2386: oid 9 version 1138 is already newer than 1137 2026-02-21T22:26:04.350 INFO:tasks.rados.rados.0.trial178.stdout:2386: oid 9 version 1138 is already newer than 1136 2026-02-21T22:26:04.350 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1138 (ObjNum 836 snap 239 seq_num 836) dirty exists 2026-02-21T22:26:04.350 INFO:tasks.rados.rados.0.trial178.stdout:2386: left oid 9 (ObjNum 836 snap 239 seq_num 836) 2026-02-21T22:26:04.350 INFO:tasks.rados.rados.0.trial178.stdout:2393: seq_num 838 ranges {484266=657492,1554381=529885} 2026-02-21T22:26:04.356 INFO:tasks.rados.rados.0.trial178.stdout:2393: writing trial17818531-10 from 484266 to 1141758 tid 1 2026-02-21T22:26:04.363 INFO:tasks.rados.rados.0.trial178.stdout:2393: writing trial17818531-10 from 1554381 to 2084266 tid 2 2026-02-21T22:26:04.363 INFO:tasks.rados.rados.0.trial178.stdout:2386: done (5 left) 2026-02-21T22:26:04.363 INFO:tasks.rados.rados.0.trial178.stdout:2388: done (4 left) 2026-02-21T22:26:04.363 INFO:tasks.rados.rados.0.trial178.stdout:2394: copy_from oid 37 from oid 5 current snap is 239 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:2389: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:2389: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:2389: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1159 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:2390: expect (ObjNum 822 snap 232 seq_num 822) 2026-02-21T22:26:04.364 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 332079 2026-02-21T22:26:04.427 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1047588 2026-02-21T22:26:04.450 INFO:tasks.rados.rados.0.trial178.stdout:2392: expect (ObjNum 804 snap 225 seq_num 804) 2026-02-21T22:26:04.457 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2814173 2026-02-21T22:26:04.492 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2393: finishing write tid 1 to trial17818531-10 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2393: finishing write tid 2 to trial17818531-10 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2393: finishing write tid 3 to trial17818531-10 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2389: done (4 left) 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2390: done (3 left) 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2392: done (2 left) 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2395: delete oid 47 current snap is 239 2026-02-21T22:26:04.519 INFO:tasks.rados.rados.0.trial178.stdout:2393: finishing write tid 4 to trial17818531-10 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2393: oid 10 updating version 0 to 1143 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2393: oid 10 updating version 1143 to 1144 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2393: oid 10 updating version 1144 to 1145 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1145 (ObjNum 838 snap 239 seq_num 838) dirty exists 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2393: left oid 10 (ObjNum 838 snap 239 seq_num 838) 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2394: finishing copy_from racing read to trial17818531-37 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:2394: finishing copy_from to trial17818531-37 2026-02-21T22:26:04.520 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1138 (ObjNum 791 snap 220 seq_num 791) dirty exists 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2393: done (2 left) 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2394: done (1 left) 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2395: done (0 left) 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2396: read oid 2 snap 227 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2396: expect (ObjNum 809 snap 226 seq_num 809) 2026-02-21T22:26:04.521 INFO:tasks.rados.rados.0.trial178.stdout:2397: snap_remove snap 224 2026-02-21T22:26:05.353 INFO:tasks.rados.rados.0.trial178.stdout:2397: done (1 left) 2026-02-21T22:26:05.353 INFO:tasks.rados.rados.0.trial178.stdout:2398: copy_from oid 27 from oid 7 current snap is 239 2026-02-21T22:26:05.353 INFO:tasks.rados.rados.0.trial178.stdout:2396: expect (ObjNum 809 snap 226 seq_num 809) 2026-02-21T22:26:05.353 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:05.491 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3199574 2026-02-21T22:26:05.493 INFO:tasks.rados.rados.0.trial178.stdout:2398: finishing copy_from racing read to trial17818531-27 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:2398: finishing copy_from to trial17818531-27 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1169 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:2396: done (1 left) 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:2398: done (0 left) 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:2399: write (excl) oid 9 current snap is 239 2026-02-21T22:26:05.494 INFO:tasks.rados.rados.0.trial178.stdout:2399: seq_num 840 ranges {517880=407990,1554992=562888} 2026-02-21T22:26:05.500 INFO:tasks.rados.rados.0.trial178.stdout:2399: writing trial17818531-9 from 517880 to 925870 tid 1 2026-02-21T22:26:05.508 INFO:tasks.rados.rados.0.trial178.stdout:2399: writing trial17818531-9 from 1554992 to 2117880 tid 2 2026-02-21T22:26:05.508 INFO:tasks.rados.rados.0.trial178.stdout:2400: snap_create 2026-02-21T22:26:05.508 INFO:tasks.rados.rados.0.trial178.stdout:2399: finishing write tid 1 to trial17818531-9 2026-02-21T22:26:05.511 INFO:tasks.rados.rados.0.trial178.stdout:2399: finishing write tid 2 to trial17818531-9 2026-02-21T22:26:05.511 INFO:tasks.rados.rados.0.trial178.stdout:2399: finishing write tid 3 to trial17818531-9 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:2399: finishing write tid 4 to trial17818531-9 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:2399: oid 9 updating version 0 to 1172 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:2399: oid 9 version 1172 is already newer than 1171 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:2399: oid 9 version 1172 is already newer than 1170 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1172 (ObjNum 840 snap 239 seq_num 840) dirty exists 2026-02-21T22:26:05.512 INFO:tasks.rados.rados.0.trial178.stdout:2399: left oid 9 (ObjNum 840 snap 239 seq_num 840) 2026-02-21T22:26:06.516 INFO:tasks.rados.rados.0.trial178.stdout:2399: done (1 left) 2026-02-21T22:26:06.516 INFO:tasks.rados.rados.0.trial178.stdout:2400: done (0 left) 2026-02-21T22:26:06.516 INFO:tasks.rados.rados.0.trial178.stdout:2401: delete oid 23 current snap is 240 2026-02-21T22:26:06.517 INFO:tasks.rados.rados.0.trial178.stdout:2401: done (0 left) 2026-02-21T22:26:06.518 INFO:tasks.rados.rados.0.trial178.stdout:2402: snap_create 2026-02-21T22:26:07.154 INFO:tasks.rados.rados.0.trial178.stdout:2402: done (0 left) 2026-02-21T22:26:07.154 INFO:tasks.rados.rados.0.trial178.stdout:2403: write (excl) oid 26 current snap is 241 2026-02-21T22:26:07.154 INFO:tasks.rados.rados.0.trial178.stdout:2403: seq_num 842 ranges {551494=558488,1671956=479538} 2026-02-21T22:26:07.161 INFO:tasks.rados.rados.0.trial178.stdout:2403: writing trial17818531-26 from 551494 to 1109982 tid 1 2026-02-21T22:26:07.169 INFO:tasks.rados.rados.0.trial178.stdout:2403: writing trial17818531-26 from 1671956 to 2151494 tid 2 2026-02-21T22:26:07.169 INFO:tasks.rados.rados.0.trial178.stdout:2404: snap_create 2026-02-21T22:26:07.169 INFO:tasks.rados.rados.0.trial178.stdout:2403: finishing write tid 1 to trial17818531-26 2026-02-21T22:26:07.171 INFO:tasks.rados.rados.0.trial178.stdout:2403: finishing write tid 2 to trial17818531-26 2026-02-21T22:26:07.172 INFO:tasks.rados.rados.0.trial178.stdout:2403: finishing write tid 3 to trial17818531-26 2026-02-21T22:26:07.172 INFO:tasks.rados.rados.0.trial178.stdout:2403: finishing write tid 4 to trial17818531-26 2026-02-21T22:26:07.172 INFO:tasks.rados.rados.0.trial178.stdout:2403: oid 26 updating version 0 to 1147 2026-02-21T22:26:07.173 INFO:tasks.rados.rados.0.trial178.stdout:2403: oid 26 updating version 1147 to 1148 2026-02-21T22:26:07.173 INFO:tasks.rados.rados.0.trial178.stdout:2403: oid 26 version 1148 is already newer than 1146 2026-02-21T22:26:07.173 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1148 (ObjNum 842 snap 241 seq_num 842) dirty exists 2026-02-21T22:26:07.173 INFO:tasks.rados.rados.0.trial178.stdout:2403: left oid 26 (ObjNum 842 snap 241 seq_num 842) 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:2403: done (1 left) 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:2404: done (0 left) 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:2405: rollback oid 23 current snap is 242 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 239 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:2406: rollback oid 17 current snap is 242 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 227 2026-02-21T22:26:08.160 INFO:tasks.rados.rados.0.trial178.stdout:2407: snap_remove snap 241 2026-02-21T22:26:08.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:08.570 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2407: done (2 left) 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2408: delete oid 3 current snap is 242 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2405: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 0 (ObjNum 81 snap 0 seq_num 2885557536) dirty dne 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2406: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2406: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:2406: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:26:09.158 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 869 (ObjNum 797 snap 224 seq_num 797) dirty exists 2026-02-21T22:26:09.164 INFO:tasks.rados.rados.0.trial178.stdout:2405: done (2 left) 2026-02-21T22:26:09.164 INFO:tasks.rados.rados.0.trial178.stdout:2406: done (1 left) 2026-02-21T22:26:09.164 INFO:tasks.rados.rados.0.trial178.stdout:2408: done (0 left) 2026-02-21T22:26:09.164 INFO:tasks.rados.rados.0.trial178.stdout:2409: snap_remove snap 236 2026-02-21T22:26:10.165 INFO:tasks.rados.rados.0.trial178.stdout:2409: done (0 left) 2026-02-21T22:26:10.166 INFO:tasks.rados.rados.0.trial178.stdout:2410: rollback oid 15 current snap is 242 2026-02-21T22:26:10.166 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 230 2026-02-21T22:26:10.166 INFO:tasks.rados.rados.0.trial178.stdout:2411: delete oid 11 current snap is 242 2026-02-21T22:26:10.169 INFO:tasks.rados.rados.0.trial178.stdout:2411: done (1 left) 2026-02-21T22:26:10.169 INFO:tasks.rados.rados.0.trial178.stdout:2412: delete oid 46 current snap is 242 2026-02-21T22:26:10.170 INFO:tasks.rados.rados.0.trial178.stdout:2410: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:26:10.171 INFO:tasks.rados.rados.0.trial178.stdout:2412: done (1 left) 2026-02-21T22:26:10.171 INFO:tasks.rados.rados.0.trial178.stdout:2413: write (excl) oid 1 current snap is 242 2026-02-21T22:26:10.171 INFO:tasks.rados.rados.0.trial178.stdout:2413: seq_num 846 ranges {618722=575837,1905884=312838} 2026-02-21T22:26:10.179 INFO:tasks.rados.rados.0.trial178.stdout:2413: writing trial17818531-1 from 618722 to 1194559 tid 1 2026-02-21T22:26:10.186 INFO:tasks.rados.rados.0.trial178.stdout:2413: writing trial17818531-1 from 1905884 to 2218722 tid 2 2026-02-21T22:26:10.186 INFO:tasks.rados.rados.0.trial178.stdout:2414: write (excl) oid 18 current snap is 242 2026-02-21T22:26:10.186 INFO:tasks.rados.rados.0.trial178.stdout:2414: seq_num 847 ranges {635529=651086,1848013=387516} 2026-02-21T22:26:10.194 INFO:tasks.rados.rados.0.trial178.stdout:2414: writing trial17818531-18 from 635529 to 1286615 tid 1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2414: writing trial17818531-18 from 1848013 to 2235529 tid 2 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2415: read oid 20 snap -1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2415: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2416: rollback oid 39 current snap is 242 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 239 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2410: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2410: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 875 (ObjNum 745 snap 209 seq_num 745) dirty exists 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: finishing write tid 1 to trial17818531-1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: finishing write tid 2 to trial17818531-1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: finishing write tid 3 to trial17818531-1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: finishing write tid 4 to trial17818531-1 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: oid 1 updating version 0 to 1274 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: oid 1 version 1274 is already newer than 1272 2026-02-21T22:26:10.201 INFO:tasks.rados.rados.0.trial178.stdout:2413: oid 1 version 1274 is already newer than 1273 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1274 (ObjNum 846 snap 242 seq_num 846) dirty exists 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2413: left oid 1 (ObjNum 846 snap 242 seq_num 846) 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2414: finishing write tid 1 to trial17818531-18 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2410: done (4 left) 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2413: done (3 left) 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2417: rollback oid 31 current snap is 242 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 223 2026-02-21T22:26:10.202 INFO:tasks.rados.rados.0.trial178.stdout:2418: snap_create 2026-02-21T22:26:10.204 INFO:tasks.rados.rados.0.trial178.stdout:2414: finishing write tid 2 to trial17818531-18 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: finishing write tid 3 to trial17818531-18 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: finishing write tid 4 to trial17818531-18 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: oid 18 updating version 0 to 1143 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: oid 18 updating version 1143 to 1145 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: oid 18 version 1145 is already newer than 1144 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1145 (ObjNum 847 snap 242 seq_num 847) dirty exists 2026-02-21T22:26:10.205 INFO:tasks.rados.rados.0.trial178.stdout:2414: left oid 18 (ObjNum 847 snap 242 seq_num 847) 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:2417: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:2416: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1135 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:2417: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:2417: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:26:10.207 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1152 (ObjNum 784 snap 219 seq_num 784) dirty exists 2026-02-21T22:26:10.208 INFO:tasks.rados.rados.0.trial178.stdout:2415: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:26:10.298 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:11.169 INFO:tasks.rados.rados.0.trial178.stdout:2414: done (4 left) 2026-02-21T22:26:11.169 INFO:tasks.rados.rados.0.trial178.stdout:2415: done (3 left) 2026-02-21T22:26:11.169 INFO:tasks.rados.rados.0.trial178.stdout:2416: done (2 left) 2026-02-21T22:26:11.169 INFO:tasks.rados.rados.0.trial178.stdout:2417: done (1 left) 2026-02-21T22:26:11.170 INFO:tasks.rados.rados.0.trial178.stdout:2418: done (0 left) 2026-02-21T22:26:11.170 INFO:tasks.rados.rados.0.trial178.stdout:2419: write oid 27 current snap is 243 2026-02-21T22:26:11.170 INFO:tasks.rados.rados.0.trial178.stdout:2419: seq_num 848 ranges {652336=726335,1906495=345841} 2026-02-21T22:26:11.178 INFO:tasks.rados.rados.0.trial178.stdout:2419: writing trial17818531-27 from 652336 to 1378671 tid 1 2026-02-21T22:26:11.184 INFO:tasks.rados.rados.0.trial178.stdout:2419: writing trial17818531-27 from 1906495 to 2252336 tid 2 2026-02-21T22:26:11.184 INFO:tasks.rados.rados.0.trial178.stdout:2420: write (excl) oid 48 current snap is 243 2026-02-21T22:26:11.184 INFO:tasks.rados.rados.0.trial178.stdout:2420: seq_num 849 ranges {669143=401584,1564977=426959} 2026-02-21T22:26:11.191 INFO:tasks.rados.rados.0.trial178.stdout:2420: writing trial17818531-48 from 669143 to 1070727 tid 1 2026-02-21T22:26:11.197 INFO:tasks.rados.rados.0.trial178.stdout:2420: writing trial17818531-48 from 1564977 to 1991936 tid 2 2026-02-21T22:26:11.197 INFO:tasks.rados.rados.0.trial178.stdout:2421: write oid 35 current snap is 243 2026-02-21T22:26:11.197 INFO:tasks.rados.rados.0.trial178.stdout:2421: seq_num 850 ranges {685950=476833,1623459=570617} 2026-02-21T22:26:11.205 INFO:tasks.rados.rados.0.trial178.stdout:2421: writing trial17818531-35 from 685950 to 1162783 tid 1 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2421: writing trial17818531-35 from 1623459 to 2194076 tid 2 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2422: copy_from oid 31 from oid 44 current snap is 243 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: finishing write tid 1 to trial17818531-27 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: finishing write tid 2 to trial17818531-27 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: finishing write tid 3 to trial17818531-27 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: finishing write tid 4 to trial17818531-27 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: oid 27 updating version 0 to 1173 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: oid 27 updating version 1173 to 1174 2026-02-21T22:26:11.213 INFO:tasks.rados.rados.0.trial178.stdout:2419: oid 27 updating version 1174 to 1175 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1175 (ObjNum 848 snap 243 seq_num 848) dirty exists 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2419: left oid 27 (ObjNum 848 snap 243 seq_num 848) 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: finishing write tid 1 to trial17818531-48 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: finishing write tid 2 to trial17818531-48 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: finishing write tid 3 to trial17818531-48 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: finishing write tid 4 to trial17818531-48 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: oid 48 updating version 0 to 1277 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: oid 48 version 1277 is already newer than 1276 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: oid 48 version 1277 is already newer than 1275 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1277 (ObjNum 849 snap 243 seq_num 849) dirty exists 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2420: left oid 48 (ObjNum 849 snap 243 seq_num 849) 2026-02-21T22:26:11.214 INFO:tasks.rados.rados.0.trial178.stdout:2421: finishing write tid 1 to trial17818531-35 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2419: done (3 left) 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2420: done (2 left) 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2423: delete oid 46 current snap is 243 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2422: finishing copy_from to trial17818531-31 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2422: got expected ENOENT (src dne) 2026-02-21T22:26:11.215 INFO:tasks.rados.rados.0.trial178.stdout:2422: finishing copy_from racing read to trial17818531-31 2026-02-21T22:26:11.216 INFO:tasks.rados.rados.0.trial178.stdout:2422: done (2 left) 2026-02-21T22:26:11.216 INFO:tasks.rados.rados.0.trial178.stdout:2423: done (1 left) 2026-02-21T22:26:11.216 INFO:tasks.rados.rados.0.trial178.stdout:2424: write (excl) oid 25 current snap is 243 2026-02-21T22:26:11.216 INFO:tasks.rados.rados.0.trial178.stdout:2424: seq_num 852 ranges {719564=743684,2140423=179141} 2026-02-21T22:26:11.220 INFO:tasks.rados.rados.0.trial178.stdout:2424: writing trial17818531-25 from 719564 to 1463248 tid 1 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2424: writing trial17818531-25 from 2140423 to 2319564 tid 2 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2425: rollback oid 3 current snap is 243 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 223 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: finishing write tid 2 to trial17818531-35 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: finishing write tid 3 to trial17818531-35 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: finishing write tid 4 to trial17818531-35 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: oid 35 updating version 0 to 1163 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: oid 35 version 1163 is already newer than 1161 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: oid 35 version 1163 is already newer than 1162 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1163 (ObjNum 850 snap 243 seq_num 850) dirty exists 2026-02-21T22:26:11.227 INFO:tasks.rados.rados.0.trial178.stdout:2421: left oid 35 (ObjNum 850 snap 243 seq_num 850) 2026-02-21T22:26:11.228 INFO:tasks.rados.rados.0.trial178.stdout:2424: finishing write tid 1 to trial17818531-25 2026-02-21T22:26:11.228 INFO:tasks.rados.rados.0.trial178.stdout:2421: done (2 left) 2026-02-21T22:26:11.228 INFO:tasks.rados.rados.0.trial178.stdout:2426: snap_remove snap 237 2026-02-21T22:26:12.156 INFO:tasks.rados.rados.0.trial178.stdout:2426: done (2 left) 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2427: delete oid 49 current snap is 243 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2424: finishing write tid 2 to trial17818531-25 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2424: finishing write tid 3 to trial17818531-25 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2425: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2425: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1165 (ObjNum 759 snap 214 seq_num 759) dirty exists 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2424: finishing write tid 4 to trial17818531-25 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2424: oid 25 updating version 0 to 1153 2026-02-21T22:26:12.157 INFO:tasks.rados.rados.0.trial178.stdout:2424: oid 25 updating version 1153 to 1154 2026-02-21T22:26:12.158 INFO:tasks.rados.rados.0.trial178.stdout:2424: oid 25 updating version 1154 to 1155 2026-02-21T22:26:12.158 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1155 (ObjNum 852 snap 243 seq_num 852) dirty exists 2026-02-21T22:26:12.158 INFO:tasks.rados.rados.0.trial178.stdout:2424: left oid 25 (ObjNum 852 snap 243 seq_num 852) 2026-02-21T22:26:12.161 INFO:tasks.rados.rados.0.trial178.stdout:2424: done (2 left) 2026-02-21T22:26:12.161 INFO:tasks.rados.rados.0.trial178.stdout:2425: done (1 left) 2026-02-21T22:26:12.161 INFO:tasks.rados.rados.0.trial178.stdout:2427: done (0 left) 2026-02-21T22:26:12.161 INFO:tasks.rados.rados.0.trial178.stdout:2428: snap_remove snap 227 2026-02-21T22:26:13.156 INFO:tasks.rados.rados.0.trial178.stdout:2428: done (0 left) 2026-02-21T22:26:13.156 INFO:tasks.rados.rados.0.trial178.stdout:2429: write (excl) oid 18 current snap is 243 2026-02-21T22:26:13.156 INFO:tasks.rados.rados.0.trial178.stdout:2429: seq_num 854 ranges {753178=494182,1857387=495791} 2026-02-21T22:26:13.162 INFO:tasks.rados.rados.0.trial178.stdout:2429: writing trial17818531-18 from 753178 to 1247360 tid 1 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2429: writing trial17818531-18 from 1857387 to 2353178 tid 2 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2430: copy_from oid 49 from oid 31 current snap is 243 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2429: finishing write tid 1 to trial17818531-18 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2431: read oid 25 snap 240 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2431: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2432: write (excl) oid 30 current snap is 243 2026-02-21T22:26:13.167 INFO:tasks.rados.rados.0.trial178.stdout:2432: seq_num 855 ranges {769985=569431,1915869=437966} 2026-02-21T22:26:13.172 INFO:tasks.rados.rados.0.trial178.stdout:2432: writing trial17818531-30 from 769985 to 1339416 tid 1 2026-02-21T22:26:13.178 INFO:tasks.rados.rados.0.trial178.stdout:2432: writing trial17818531-30 from 1915869 to 2353835 tid 2 2026-02-21T22:26:13.178 INFO:tasks.rados.rados.0.trial178.stdout:2433: rollback oid 9 current snap is 243 2026-02-21T22:26:13.178 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 223 2026-02-21T22:26:13.178 INFO:tasks.rados.rados.0.trial178.stdout:2431: expect (ObjNum 754 snap 213 seq_num 754) 2026-02-21T22:26:13.270 INFO:tasks.rados.rados.0.trial178.stdout:2429: finishing write tid 2 to trial17818531-18 2026-02-21T22:26:13.270 INFO:tasks.rados.rados.0.trial178.stdout:2429: finishing write tid 3 to trial17818531-18 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2429: finishing write tid 4 to trial17818531-18 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2429: oid 18 updating version 0 to 1148 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2429: oid 18 version 1148 is already newer than 1147 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2429: oid 18 version 1148 is already newer than 1146 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1148 (ObjNum 854 snap 243 seq_num 854) dirty exists 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2429: left oid 18 (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: finishing write tid 1 to trial17818531-30 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: finishing write tid 2 to trial17818531-30 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: finishing write tid 3 to trial17818531-30 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: finishing write tid 4 to trial17818531-30 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: oid 30 updating version 0 to 1149 2026-02-21T22:26:13.271 INFO:tasks.rados.rados.0.trial178.stdout:2432: oid 30 updating version 1149 to 1150 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2432: oid 30 updating version 1150 to 1151 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1151 (ObjNum 855 snap 243 seq_num 855) dirty exists 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2432: left oid 30 (ObjNum 855 snap 243 seq_num 855) 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2433: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2433: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2433: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1178 (ObjNum 761 snap 214 seq_num 761) dirty exists 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2430: finishing copy_from racing read to trial17818531-49 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2430: finishing copy_from to trial17818531-49 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1152 (ObjNum 784 snap 219 seq_num 784) dirty exists 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2429: done (4 left) 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2430: done (3 left) 2026-02-21T22:26:13.272 INFO:tasks.rados.rados.0.trial178.stdout:2431: done (2 left) 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2432: done (1 left) 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2433: done (0 left) 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2434: read oid 14 snap 233 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2434: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2435: write oid 11 current snap is 243 2026-02-21T22:26:13.273 INFO:tasks.rados.rados.0.trial178.stdout:2435: seq_num 856 ranges {786792=644680,1857998=528794} 2026-02-21T22:26:13.279 INFO:tasks.rados.rados.0.trial178.stdout:2435: writing trial17818531-11 from 786792 to 1431472 tid 1 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2435: writing trial17818531-11 from 1857998 to 2386792 tid 2 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2436: read oid 38 snap -1 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2436: expect deleted 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2437: read oid 47 snap -1 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2437: expect deleted 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2438: write oid 10 current snap is 243 2026-02-21T22:26:13.286 INFO:tasks.rados.rados.0.trial178.stdout:2438: seq_num 857 ranges {403599=719929,1916480=441635} 2026-02-21T22:26:13.293 INFO:tasks.rados.rados.0.trial178.stdout:2438: writing trial17818531-10 from 403599 to 1123528 tid 1 2026-02-21T22:26:13.301 INFO:tasks.rados.rados.0.trial178.stdout:2438: writing trial17818531-10 from 1916480 to 2358115 tid 2 2026-02-21T22:26:13.301 INFO:tasks.rados.rados.0.trial178.stdout:2439: snap_create 2026-02-21T22:26:13.302 INFO:tasks.rados.rados.0.trial178.stdout:2434: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:26:13.350 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2253449 2026-02-21T22:26:13.360 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2202068 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: finishing write tid 1 to trial17818531-11 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: finishing write tid 2 to trial17818531-11 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: finishing write tid 3 to trial17818531-11 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: finishing write tid 4 to trial17818531-11 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: oid 11 updating version 0 to 1167 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: oid 11 version 1167 is already newer than 1166 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:2435: oid 11 updating version 1167 to 1168 2026-02-21T22:26:13.363 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1168 (ObjNum 856 snap 243 seq_num 856) dirty exists 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2435: left oid 11 (ObjNum 856 snap 243 seq_num 856) 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: finishing write tid 1 to trial17818531-10 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: finishing write tid 2 to trial17818531-10 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: finishing write tid 3 to trial17818531-10 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: finishing write tid 4 to trial17818531-10 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: oid 10 updating version 0 to 1150 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: oid 10 version 1150 is already newer than 1149 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: oid 10 updating version 1150 to 1151 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1151 (ObjNum 857 snap 243 seq_num 857) dirty exists 2026-02-21T22:26:13.364 INFO:tasks.rados.rados.0.trial178.stdout:2438: left oid 10 (ObjNum 857 snap 243 seq_num 857) 2026-02-21T22:26:13.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:13.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2434: done (5 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2435: done (4 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2436: done (3 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2437: done (2 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2438: done (1 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2439: done (0 left) 2026-02-21T22:26:14.180 INFO:tasks.rados.rados.0.trial178.stdout:2440: snap_remove snap 240 2026-02-21T22:26:15.183 INFO:tasks.rados.rados.0.trial178.stdout:2440: done (0 left) 2026-02-21T22:26:15.184 INFO:tasks.rados.rados.0.trial178.stdout:2441: snap_create 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:2441: done (0 left) 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:2442: read oid 21 snap -1 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:2442: expect (ObjNum 782 snap 219 seq_num 782) 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:2443: rollback oid 33 current snap is 245 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 33 to 231 2026-02-21T22:26:16.185 INFO:tasks.rados.rados.0.trial178.stdout:2444: read oid 48 snap -1 2026-02-21T22:26:16.186 INFO:tasks.rados.rados.0.trial178.stdout:2444: expect (ObjNum 849 snap 243 seq_num 849) 2026-02-21T22:26:16.186 INFO:tasks.rados.rados.0.trial178.stdout:2445: write oid 38 current snap is 245 2026-02-21T22:26:16.186 INFO:tasks.rados.rados.0.trial178.stdout:2445: seq_num 858 ranges {420406=795178,1974962=445444} 2026-02-21T22:26:16.193 INFO:tasks.rados.rados.0.trial178.stdout:2445: writing trial17818531-38 from 420406 to 1215584 tid 1 2026-02-21T22:26:16.201 INFO:tasks.rados.rados.0.trial178.stdout:2445: writing trial17818531-38 from 1974962 to 2420406 tid 2 2026-02-21T22:26:16.201 INFO:tasks.rados.rados.0.trial178.stdout:2446: copy_from oid 4 from oid 12 current snap is 245 2026-02-21T22:26:16.201 INFO:tasks.rados.rados.0.trial178.stdout:2444: expect (ObjNum 849 snap 243 seq_num 849) 2026-02-21T22:26:16.223 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1720174 2026-02-21T22:26:16.263 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2050204 2026-02-21T22:26:16.301 INFO:tasks.rados.rados.0.trial178.stdout:2443: finishing rollback tid 0 to trial17818531-33 2026-02-21T22:26:16.301 INFO:tasks.rados.rados.0.trial178.stdout:2443: finishing rollback tid 1 to trial17818531-33 2026-02-21T22:26:16.301 INFO:tasks.rados.rados.0.trial178.stdout:2443: finishing rollback tid 2 to trial17818531-33 2026-02-21T22:26:16.301 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 844 (ObjNum 807 snap 226 seq_num 807) dirty exists 2026-02-21T22:26:16.301 INFO:tasks.rados.rados.0.trial178.stdout:2442: expect (ObjNum 782 snap 219 seq_num 782) 2026-02-21T22:26:16.355 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 821515 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: finishing write tid 1 to trial17818531-38 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2446: finishing copy_from to trial17818531-4 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2446: got expected ENOENT (src dne) 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2446: finishing copy_from racing read to trial17818531-4 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: finishing write tid 2 to trial17818531-38 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: finishing write tid 3 to trial17818531-38 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: finishing write tid 4 to trial17818531-38 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: oid 38 updating version 0 to 847 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: oid 38 version 847 is already newer than 845 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: oid 38 version 847 is already newer than 846 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 847 (ObjNum 858 snap 245 seq_num 858) dirty exists 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2445: left oid 38 (ObjNum 858 snap 245 seq_num 858) 2026-02-21T22:26:16.426 INFO:tasks.rados.rados.0.trial178.stdout:2442: done (4 left) 2026-02-21T22:26:16.427 INFO:tasks.rados.rados.0.trial178.stdout:2443: done (3 left) 2026-02-21T22:26:16.427 INFO:tasks.rados.rados.0.trial178.stdout:2444: done (2 left) 2026-02-21T22:26:16.427 INFO:tasks.rados.rados.0.trial178.stdout:2445: done (1 left) 2026-02-21T22:26:16.427 INFO:tasks.rados.rados.0.trial178.stdout:2446: done (0 left) 2026-02-21T22:26:16.427 INFO:tasks.rados.rados.0.trial178.stdout:2447: snap_create 2026-02-21T22:26:17.156 INFO:tasks.rados.rados.0.trial178.stdout:2447: done (0 left) 2026-02-21T22:26:17.156 INFO:tasks.rados.rados.0.trial178.stdout:2448: snap_remove snap 230 2026-02-21T22:26:18.161 INFO:tasks.rados.rados.0.trial178.stdout:2448: done (0 left) 2026-02-21T22:26:18.161 INFO:tasks.rados.rados.0.trial178.stdout:2449: copy_from oid 48 from oid 20 current snap is 246 2026-02-21T22:26:18.161 INFO:tasks.rados.rados.0.trial178.stdout:2450: read oid 36 snap -1 2026-02-21T22:26:18.161 INFO:tasks.rados.rados.0.trial178.stdout:2450: expect (ObjNum 825 snap 232 seq_num 825) 2026-02-21T22:26:18.161 INFO:tasks.rados.rados.0.trial178.stdout:2451: snap_remove snap 235 2026-02-21T22:26:18.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:18.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:19.163 INFO:tasks.rados.rados.0.trial178.stdout:2451: done (2 left) 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:2452: snap_create 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:2449: finishing copy_from racing read to trial17818531-48 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:2449: finishing copy_from to trial17818531-48 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 847 (ObjNum 724 snap 204 seq_num 724) dirty exists 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:2450: expect (ObjNum 825 snap 232 seq_num 825) 2026-02-21T22:26:19.164 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:19.279 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:20.165 INFO:tasks.rados.rados.0.trial178.stdout:2449: done (2 left) 2026-02-21T22:26:20.166 INFO:tasks.rados.rados.0.trial178.stdout:2450: done (1 left) 2026-02-21T22:26:20.166 INFO:tasks.rados.rados.0.trial178.stdout:2452: done (0 left) 2026-02-21T22:26:20.166 INFO:tasks.rados.rados.0.trial178.stdout:2453: snap_remove snap 223 2026-02-21T22:26:21.171 INFO:tasks.rados.rados.0.trial178.stdout:2453: done (0 left) 2026-02-21T22:26:21.172 INFO:tasks.rados.rados.0.trial178.stdout:2454: read oid 19 snap 246 2026-02-21T22:26:21.172 INFO:tasks.rados.rados.0.trial178.stdout:2454: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:26:21.172 INFO:tasks.rados.rados.0.trial178.stdout:2455: snap_create 2026-02-21T22:26:21.184 INFO:tasks.rados.rados.0.trial178.stdout:2454: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:26:21.203 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2736417 2026-02-21T22:26:21.219 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 813946 2026-02-21T22:26:22.157 INFO:tasks.rados.rados.0.trial178.stdout:2454: done (1 left) 2026-02-21T22:26:22.157 INFO:tasks.rados.rados.0.trial178.stdout:2455: done (0 left) 2026-02-21T22:26:22.157 INFO:tasks.rados.rados.0.trial178.stdout:2456: snap_create 2026-02-21T22:26:23.158 INFO:tasks.rados.rados.0.trial178.stdout:2456: done (0 left) 2026-02-21T22:26:23.158 INFO:tasks.rados.rados.0.trial178.stdout:2457: rollback oid 41 current snap is 249 2026-02-21T22:26:23.159 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 248 2026-02-21T22:26:23.159 INFO:tasks.rados.rados.0.trial178.stdout:2458: delete oid 35 current snap is 249 2026-02-21T22:26:23.162 INFO:tasks.rados.rados.0.trial178.stdout:2458: done (1 left) 2026-02-21T22:26:23.162 INFO:tasks.rados.rados.0.trial178.stdout:2459: read oid 26 snap 233 2026-02-21T22:26:23.162 INFO:tasks.rados.rados.0.trial178.stdout:2459: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:26:23.162 INFO:tasks.rados.rados.0.trial178.stdout:2460: copy_from oid 22 from oid 32 current snap is 249 2026-02-21T22:26:23.162 INFO:tasks.rados.rados.0.trial178.stdout:2461: snap_remove snap 246 2026-02-21T22:26:23.531 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:26:23] 2026-02-21T22:26:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:26:23, mode upmap, max misplaced 0.050000 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', '.mgr', 'unique_pool_0'] 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:26:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:26:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:23.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:26:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:26:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:26:23.558 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:26:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:26:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:26:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:26:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:26:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:26:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:26:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:26:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:26:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:26:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:26:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:26:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "512", 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:26:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:26:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:26:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:26:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 512, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 255, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:26:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:26:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:26:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:26:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:26:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:26:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:26:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:26:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:26:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:26:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0005197595666956019 of space, bias 1.0, pg target 0.15592787000868055 quantized to 32 (current 16) 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:26:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:26:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:26:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:26:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:24.159 INFO:tasks.rados.rados.0.trial178.stdout:2461: done (3 left) 2026-02-21T22:26:24.159 INFO:tasks.rados.rados.0.trial178.stdout:2462: delete oid 13 current snap is 249 2026-02-21T22:26:24.159 INFO:tasks.rados.rados.0.trial178.stdout:2457: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:26:24.159 INFO:tasks.rados.rados.0.trial178.stdout:2457: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:26:24.159 INFO:tasks.rados.rados.0.trial178.stdout:2457: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:26:24.160 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 911 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:26:24.160 INFO:tasks.rados.rados.0.trial178.stdout:2459: expect (ObjNum 745 snap 209 seq_num 745) 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2460: finishing copy_from racing read to trial17818531-22 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2460: finishing copy_from to trial17818531-22 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1159 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2457: done (3 left) 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2459: done (2 left) 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2460: done (1 left) 2026-02-21T22:26:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2462: done (0 left) 2026-02-21T22:26:24.224 INFO:tasks.rados.rados.0.trial178.stdout:2463: copy_from oid 47 from oid 18 current snap is 249 2026-02-21T22:26:24.224 INFO:tasks.rados.rados.0.trial178.stdout:2464: read oid 44 snap 247 2026-02-21T22:26:24.224 INFO:tasks.rados.rados.0.trial178.stdout:2464: expect deleted 2026-02-21T22:26:24.224 INFO:tasks.rados.rados.0.trial178.stdout:2465: write oid 1 current snap is 249 2026-02-21T22:26:24.224 INFO:tasks.rados.rados.0.trial178.stdout:2465: seq_num 861 ranges {470827=737278,1750408=720419} 2026-02-21T22:26:24.231 INFO:tasks.rados.rados.0.trial178.stdout:2465: writing trial17818531-1 from 470827 to 1208105 tid 1 2026-02-21T22:26:24.239 INFO:tasks.rados.rados.0.trial178.stdout:2465: writing trial17818531-1 from 1750408 to 2470827 tid 2 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2466: read oid 8 snap 245 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2466: expect (ObjNum 826 snap 235 seq_num 826) 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2467: read oid 45 snap -1 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2467: expect deleted 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2468: write oid 27 current snap is 249 2026-02-21T22:26:24.240 INFO:tasks.rados.rados.0.trial178.stdout:2468: seq_num 862 ranges {487634=412527,1408890=592631} 2026-02-21T22:26:24.245 INFO:tasks.rados.rados.0.trial178.stdout:2468: writing trial17818531-27 from 487634 to 900161 tid 1 2026-02-21T22:26:24.253 INFO:tasks.rados.rados.0.trial178.stdout:2468: writing trial17818531-27 from 1408890 to 2001521 tid 2 2026-02-21T22:26:24.253 INFO:tasks.rados.rados.0.trial178.stdout:2469: rollback oid 31 current snap is 249 2026-02-21T22:26:24.253 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 247 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2463: finishing copy_from racing read to trial17818531-47 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2463: finishing copy_from to trial17818531-47 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1148 (ObjNum 854 snap 243 seq_num 854) dirty exists 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: finishing write tid 1 to trial17818531-1 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: finishing write tid 2 to trial17818531-1 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: finishing write tid 3 to trial17818531-1 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: finishing write tid 4 to trial17818531-1 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: oid 1 updating version 0 to 1278 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: oid 1 updating version 1278 to 1280 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:2465: oid 1 version 1280 is already newer than 1279 2026-02-21T22:26:24.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1280 (ObjNum 861 snap 249 seq_num 861) dirty exists 2026-02-21T22:26:24.255 INFO:tasks.rados.rados.0.trial178.stdout:2465: left oid 1 (ObjNum 861 snap 249 seq_num 861) 2026-02-21T22:26:24.255 INFO:tasks.rados.rados.0.trial178.stdout:2466: expect (ObjNum 826 snap 235 seq_num 826) 2026-02-21T22:26:24.335 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2240716 2026-02-21T22:26:24.392 INFO:tasks.rados.rados.0.trial178.stdout:2468: finishing write tid 1 to trial17818531-27 2026-02-21T22:26:24.392 INFO:tasks.rados.rados.0.trial178.stdout:2469: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:26:24.392 INFO:tasks.rados.rados.0.trial178.stdout:2469: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2469: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1159 (ObjNum 784 snap 219 seq_num 784) dirty exists 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: finishing write tid 2 to trial17818531-27 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: finishing write tid 3 to trial17818531-27 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: finishing write tid 4 to trial17818531-27 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: oid 27 updating version 0 to 1181 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: oid 27 version 1181 is already newer than 1180 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: oid 27 version 1181 is already newer than 1179 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1181 (ObjNum 862 snap 249 seq_num 862) dirty exists 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2468: left oid 27 (ObjNum 862 snap 249 seq_num 862) 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2463: done (6 left) 2026-02-21T22:26:24.393 INFO:tasks.rados.rados.0.trial178.stdout:2464: done (5 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2465: done (4 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2466: done (3 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2467: done (2 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2468: done (1 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2469: done (0 left) 2026-02-21T22:26:24.394 INFO:tasks.rados.rados.0.trial178.stdout:2470: snap_create 2026-02-21T22:26:25.178 INFO:tasks.rados.rados.0.trial178.stdout:2470: done (0 left) 2026-02-21T22:26:25.178 INFO:tasks.rados.rados.0.trial178.stdout:2471: rollback oid 41 current snap is 250 2026-02-21T22:26:25.178 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 249 2026-02-21T22:26:25.178 INFO:tasks.rados.rados.0.trial178.stdout:2472: snap_remove snap 243 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2472: done (1 left) 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2473: read oid 39 snap -1 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2473: expect deleted 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2474: write oid 12 current snap is 250 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2471: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:26:26.180 INFO:tasks.rados.rados.0.trial178.stdout:2471: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:26:26.181 INFO:tasks.rados.rados.0.trial178.stdout:2471: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:26:26.181 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 917 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:26:26.181 INFO:tasks.rados.rados.0.trial178.stdout:2474: seq_num 863 ranges {504441=487776,1467372=736289} 2026-02-21T22:26:26.186 INFO:tasks.rados.rados.0.trial178.stdout:2474: writing trial17818531-12 from 504441 to 992217 tid 1 2026-02-21T22:26:26.195 INFO:tasks.rados.rados.0.trial178.stdout:2474: writing trial17818531-12 from 1467372 to 2203661 tid 2 2026-02-21T22:26:26.195 INFO:tasks.rados.rados.0.trial178.stdout:2471: done (2 left) 2026-02-21T22:26:26.195 INFO:tasks.rados.rados.0.trial178.stdout:2475: read oid 34 snap -1 2026-02-21T22:26:26.195 INFO:tasks.rados.rados.0.trial178.stdout:2475: expect deleted 2026-02-21T22:26:26.195 INFO:tasks.rados.rados.0.trial178.stdout:2476: snap_remove snap 231 2026-02-21T22:26:27.158 INFO:tasks.rados.rados.0.trial178.stdout:2476: done (3 left) 2026-02-21T22:26:27.158 INFO:tasks.rados.rados.0.trial178.stdout:2477: snap_create 2026-02-21T22:26:27.158 INFO:tasks.rados.rados.0.trial178.stdout:2474: finishing write tid 1 to trial17818531-12 2026-02-21T22:26:27.158 INFO:tasks.rados.rados.0.trial178.stdout:2474: finishing write tid 2 to trial17818531-12 2026-02-21T22:26:27.158 INFO:tasks.rados.rados.0.trial178.stdout:2474: finishing write tid 3 to trial17818531-12 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:2474: finishing write tid 4 to trial17818531-12 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:2474: oid 12 updating version 0 to 822 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:2474: oid 12 version 822 is already newer than 821 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:2474: oid 12 updating version 822 to 823 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 823 (ObjNum 863 snap 250 seq_num 863) dirty exists 2026-02-21T22:26:27.159 INFO:tasks.rados.rados.0.trial178.stdout:2474: left oid 12 (ObjNum 863 snap 250 seq_num 863) 2026-02-21T22:26:28.161 INFO:tasks.rados.rados.0.trial178.stdout:2473: done (3 left) 2026-02-21T22:26:28.161 INFO:tasks.rados.rados.0.trial178.stdout:2474: done (2 left) 2026-02-21T22:26:28.162 INFO:tasks.rados.rados.0.trial178.stdout:2475: done (1 left) 2026-02-21T22:26:28.162 INFO:tasks.rados.rados.0.trial178.stdout:2477: done (0 left) 2026-02-21T22:26:28.162 INFO:tasks.rados.rados.0.trial178.stdout:2478: snap_remove snap 233 2026-02-21T22:26:28.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:28.572 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:29.162 INFO:tasks.rados.rados.0.trial178.stdout:2478: done (0 left) 2026-02-21T22:26:29.162 INFO:tasks.rados.rados.0.trial178.stdout:2479: delete oid 25 current snap is 251 2026-02-21T22:26:29.168 INFO:tasks.rados.rados.0.trial178.stdout:2479: done (0 left) 2026-02-21T22:26:29.168 INFO:tasks.rados.rados.0.trial178.stdout:2480: write oid 43 current snap is 251 2026-02-21T22:26:29.168 INFO:tasks.rados.rados.0.trial178.stdout:2480: seq_num 865 ranges {538055=638274,1867983=670072} 2026-02-21T22:26:29.175 INFO:tasks.rados.rados.0.trial178.stdout:2480: writing trial17818531-43 from 538055 to 1176329 tid 1 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2480: writing trial17818531-43 from 1867983 to 2538055 tid 2 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2481: read oid 22 snap 250 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2481: expect (ObjNum 781 snap 219 seq_num 781) 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2482: write (excl) oid 20 current snap is 251 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2480: finishing write tid 1 to trial17818531-43 2026-02-21T22:26:29.185 INFO:tasks.rados.rados.0.trial178.stdout:2482: seq_num 866 ranges {554862=713523,1926465=599969} 2026-02-21T22:26:29.193 INFO:tasks.rados.rados.0.trial178.stdout:2482: writing trial17818531-20 from 554862 to 1268385 tid 1 2026-02-21T22:26:29.202 INFO:tasks.rados.rados.0.trial178.stdout:2482: writing trial17818531-20 from 1926465 to 2526434 tid 2 2026-02-21T22:26:29.202 INFO:tasks.rados.rados.0.trial178.stdout:2483: write oid 12 current snap is 251 2026-02-21T22:26:29.202 INFO:tasks.rados.rados.0.trial178.stdout:2483: seq_num 867 ranges {571669=505125,1701300=743627} 2026-02-21T22:26:29.210 INFO:tasks.rados.rados.0.trial178.stdout:2483: writing trial17818531-12 from 571669 to 1076794 tid 1 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2483: writing trial17818531-12 from 1701300 to 2444927 tid 2 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2484: rollback oid 4 current snap is 251 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 244 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: finishing write tid 2 to trial17818531-43 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: finishing write tid 3 to trial17818531-43 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: finishing write tid 4 to trial17818531-43 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: oid 43 updating version 0 to 1141 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: oid 43 updating version 1141 to 1142 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:2480: oid 43 version 1142 is already newer than 1140 2026-02-21T22:26:29.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1142 (ObjNum 865 snap 251 seq_num 865) dirty exists 2026-02-21T22:26:29.220 INFO:tasks.rados.rados.0.trial178.stdout:2480: left oid 43 (ObjNum 865 snap 251 seq_num 865) 2026-02-21T22:26:29.220 INFO:tasks.rados.rados.0.trial178.stdout:2481: expect (ObjNum 781 snap 219 seq_num 781) 2026-02-21T22:26:29.220 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:29.399 INFO:tasks.rados.rados.0.trial178.stdout:2482: finishing write tid 1 to trial17818531-20 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: finishing write tid 2 to trial17818531-20 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: finishing write tid 3 to trial17818531-20 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: finishing write tid 4 to trial17818531-20 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: oid 20 updating version 0 to 935 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: oid 20 version 935 is already newer than 933 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: oid 20 version 935 is already newer than 934 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 935 (ObjNum 866 snap 251 seq_num 866) dirty exists 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2482: left oid 20 (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2483: finishing write tid 1 to trial17818531-12 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2483: finishing write tid 2 to trial17818531-12 2026-02-21T22:26:29.400 INFO:tasks.rados.rados.0.trial178.stdout:2483: finishing write tid 3 to trial17818531-12 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2483: finishing write tid 4 to trial17818531-12 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2483: oid 12 updating version 0 to 826 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2483: oid 12 version 826 is already newer than 825 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2483: oid 12 version 826 is already newer than 824 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 826 (ObjNum 867 snap 251 seq_num 867) dirty exists 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2483: left oid 12 (ObjNum 867 snap 251 seq_num 867) 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2484: finishing rollback tid 0 to trial17818531-4 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2484: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2484: finishing rollback tid 2 to trial17818531-4 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 829 (ObjNum 816 snap 230 seq_num 816) dirty exists 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2480: done (4 left) 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2481: done (3 left) 2026-02-21T22:26:29.401 INFO:tasks.rados.rados.0.trial178.stdout:2482: done (2 left) 2026-02-21T22:26:29.402 INFO:tasks.rados.rados.0.trial178.stdout:2483: done (1 left) 2026-02-21T22:26:29.402 INFO:tasks.rados.rados.0.trial178.stdout:2484: done (0 left) 2026-02-21T22:26:29.402 INFO:tasks.rados.rados.0.trial178.stdout:2485: rollback oid 41 current snap is 251 2026-02-21T22:26:29.402 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 250 2026-02-21T22:26:29.402 INFO:tasks.rados.rados.0.trial178.stdout:2486: snap_create 2026-02-21T22:26:29.403 INFO:tasks.rados.rados.0.trial178.stdout:2485: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:26:29.403 INFO:tasks.rados.rados.0.trial178.stdout:2485: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:26:29.403 INFO:tasks.rados.rados.0.trial178.stdout:2485: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:26:29.403 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:26:30.169 INFO:tasks.rados.rados.0.trial178.stdout:2485: done (1 left) 2026-02-21T22:26:30.169 INFO:tasks.rados.rados.0.trial178.stdout:2486: done (0 left) 2026-02-21T22:26:30.169 INFO:tasks.rados.rados.0.trial178.stdout:2487: rollback oid 7 current snap is 252 2026-02-21T22:26:30.169 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 7 to 239 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2488: rollback oid 11 current snap is 252 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 248 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2489: read oid 37 snap 242 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2489: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2490: copy_from oid 42 from oid 40 current snap is 252 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2491: write (excl) oid 28 current snap is 252 2026-02-21T22:26:30.170 INFO:tasks.rados.rados.0.trial178.stdout:2491: seq_num 868 ranges {588476=580374,1643429=603638} 2026-02-21T22:26:30.176 INFO:tasks.rados.rados.0.trial178.stdout:2491: writing trial17818531-28 from 588476 to 1168850 tid 1 2026-02-21T22:26:30.183 INFO:tasks.rados.rados.0.trial178.stdout:2491: writing trial17818531-28 from 1643429 to 2247067 tid 2 2026-02-21T22:26:30.184 INFO:tasks.rados.rados.0.trial178.stdout:2492: snap_remove snap 247 2026-02-21T22:26:31.172 INFO:tasks.rados.rados.0.trial178.stdout:2492: done (5 left) 2026-02-21T22:26:31.172 INFO:tasks.rados.rados.0.trial178.stdout:2493: snap_remove snap 249 2026-02-21T22:26:32.793 INFO:tasks.rados.rados.0.trial178.stdout:2493: done (5 left) 2026-02-21T22:26:32.793 INFO:tasks.rados.rados.0.trial178.stdout:2494: copy_from oid 49 from oid 10 current snap is 252 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2488: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2488: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2487: finishing rollback tid 0 to trial17818531-7 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2488: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1173 (ObjNum 856 snap 243 seq_num 856) dirty exists 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2487: finishing rollback tid 1 to trial17818531-7 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2487: finishing rollback tid 2 to trial17818531-7 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1370 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:2489: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:32.794 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:32.849 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:32.882 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:32.916 INFO:tasks.rados.rados.0.trial178.stdout:2491: finishing write tid 1 to trial17818531-28 2026-02-21T22:26:32.916 INFO:tasks.rados.rados.0.trial178.stdout:2491: finishing write tid 2 to trial17818531-28 2026-02-21T22:26:32.916 INFO:tasks.rados.rados.0.trial178.stdout:2491: finishing write tid 3 to trial17818531-28 2026-02-21T22:26:32.916 INFO:tasks.rados.rados.0.trial178.stdout:2491: finishing write tid 4 to trial17818531-28 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2491: oid 28 updating version 0 to 850 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2491: oid 28 version 850 is already newer than 849 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2491: oid 28 version 850 is already newer than 848 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 850 (ObjNum 868 snap 252 seq_num 868) dirty exists 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2491: left oid 28 (ObjNum 868 snap 252 seq_num 868) 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2490: finishing copy_from racing read to trial17818531-42 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2490: finishing copy_from to trial17818531-42 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1108 (ObjNum 832 snap 238 seq_num 832) dirty exists 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2487: done (5 left) 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2488: done (4 left) 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2489: done (3 left) 2026-02-21T22:26:32.917 INFO:tasks.rados.rados.0.trial178.stdout:2490: done (2 left) 2026-02-21T22:26:32.918 INFO:tasks.rados.rados.0.trial178.stdout:2491: done (1 left) 2026-02-21T22:26:32.918 INFO:tasks.rados.rados.0.trial178.stdout:2495: read oid 14 snap -1 2026-02-21T22:26:32.918 INFO:tasks.rados.rados.0.trial178.stdout:2495: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:26:32.918 INFO:tasks.rados.rados.0.trial178.stdout:2496: snap_remove snap 248 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2496: done (2 left) 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2497: read oid 6 snap -1 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2497: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2498: delete oid 43 current snap is 252 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2494: finishing copy_from racing read to trial17818531-49 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2494: finishing copy_from to trial17818531-49 2026-02-21T22:26:33.179 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1151 (ObjNum 857 snap 243 seq_num 857) dirty exists 2026-02-21T22:26:33.180 INFO:tasks.rados.rados.0.trial178.stdout:2495: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:26:33.180 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:33.232 INFO:tasks.rados.rados.0.trial178.stdout:2497: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:26:33.262 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1757381 2026-02-21T22:26:33.281 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1763848 2026-02-21T22:26:33.294 INFO:tasks.rados.rados.0.trial178.stdout:2494: done (3 left) 2026-02-21T22:26:33.294 INFO:tasks.rados.rados.0.trial178.stdout:2495: done (2 left) 2026-02-21T22:26:33.294 INFO:tasks.rados.rados.0.trial178.stdout:2497: done (1 left) 2026-02-21T22:26:33.294 INFO:tasks.rados.rados.0.trial178.stdout:2498: done (0 left) 2026-02-21T22:26:33.294 INFO:tasks.rados.rados.0.trial178.stdout:2499: snap_create 2026-02-21T22:26:33.572 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:33.572 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:34.183 INFO:tasks.rados.rados.0.trial178.stdout:2499: done (0 left) 2026-02-21T22:26:34.184 INFO:tasks.rados.rados.0.trial178.stdout:2500: write (excl) oid 22 current snap is 253 2026-02-21T22:26:34.184 INFO:tasks.rados.rados.0.trial178.stdout:2500: seq_num 870 ranges {622090=730872,1760393=607307} 2026-02-21T22:26:34.192 INFO:tasks.rados.rados.0.trial178.stdout:2500: writing trial17818531-22 from 622090 to 1352962 tid 1 2026-02-21T22:26:34.199 INFO:tasks.rados.rados.0.trial178.stdout:2500: writing trial17818531-22 from 1760393 to 2367700 tid 2 2026-02-21T22:26:34.199 INFO:tasks.rados.rados.0.trial178.stdout:2501: write (excl) oid 29 current snap is 253 2026-02-21T22:26:34.199 INFO:tasks.rados.rados.0.trial178.stdout:2501: seq_num 871 ranges {638897=406121,1818875=750965} 2026-02-21T22:26:34.206 INFO:tasks.rados.rados.0.trial178.stdout:2501: writing trial17818531-29 from 638897 to 1045018 tid 1 2026-02-21T22:26:34.217 INFO:tasks.rados.rados.0.trial178.stdout:2501: writing trial17818531-29 from 1818875 to 2569840 tid 2 2026-02-21T22:26:34.217 INFO:tasks.rados.rados.0.trial178.stdout:2502: write oid 16 current snap is 253 2026-02-21T22:26:34.217 INFO:tasks.rados.rados.0.trial178.stdout:2502: seq_num 872 ranges {655704=481370,1761004=494623} 2026-02-21T22:26:34.225 INFO:tasks.rados.rados.0.trial178.stdout:2502: writing trial17818531-16 from 655704 to 1137074 tid 1 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2502: writing trial17818531-16 from 1761004 to 2255627 tid 2 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2503: snap_create 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: finishing write tid 1 to trial17818531-22 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: finishing write tid 2 to trial17818531-22 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: finishing write tid 3 to trial17818531-22 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: finishing write tid 4 to trial17818531-22 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: oid 22 updating version 0 to 1162 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: oid 22 version 1162 is already newer than 1161 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: oid 22 version 1162 is already newer than 1160 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1162 (ObjNum 870 snap 253 seq_num 870) dirty exists 2026-02-21T22:26:34.232 INFO:tasks.rados.rados.0.trial178.stdout:2500: left oid 22 (ObjNum 870 snap 253 seq_num 870) 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: finishing write tid 1 to trial17818531-29 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: finishing write tid 2 to trial17818531-29 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: finishing write tid 3 to trial17818531-29 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: finishing write tid 4 to trial17818531-29 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: oid 29 updating version 0 to 798 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: oid 29 updating version 798 to 800 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: oid 29 version 800 is already newer than 799 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 800 (ObjNum 871 snap 253 seq_num 871) dirty exists 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2501: left oid 29 (ObjNum 871 snap 253 seq_num 871) 2026-02-21T22:26:34.233 INFO:tasks.rados.rados.0.trial178.stdout:2502: finishing write tid 1 to trial17818531-16 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: finishing write tid 2 to trial17818531-16 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: finishing write tid 3 to trial17818531-16 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: finishing write tid 4 to trial17818531-16 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: oid 16 updating version 0 to 832 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: oid 16 version 832 is already newer than 831 2026-02-21T22:26:34.234 INFO:tasks.rados.rados.0.trial178.stdout:2502: oid 16 version 832 is already newer than 830 2026-02-21T22:26:34.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 832 (ObjNum 872 snap 253 seq_num 872) dirty exists 2026-02-21T22:26:34.235 INFO:tasks.rados.rados.0.trial178.stdout:2502: left oid 16 (ObjNum 872 snap 253 seq_num 872) 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2500: done (3 left) 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2501: done (2 left) 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2502: done (1 left) 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2503: done (0 left) 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2504: write oid 15 current snap is 254 2026-02-21T22:26:35.187 INFO:tasks.rados.rados.0.trial178.stdout:2504: seq_num 873 ranges {672511=556619,1819486=754634} 2026-02-21T22:26:35.192 INFO:tasks.rados.rados.0.trial178.stdout:2504: writing trial17818531-15 from 672511 to 1229130 tid 1 2026-02-21T22:26:35.197 INFO:tasks.rados.rados.0.trial178.stdout:2504: writing trial17818531-15 from 1819486 to 2574120 tid 2 2026-02-21T22:26:35.198 INFO:tasks.rados.rados.0.trial178.stdout:2505: write oid 31 current snap is 254 2026-02-21T22:26:35.198 INFO:tasks.rados.rados.0.trial178.stdout:2505: seq_num 874 ranges {689318=631868,1877968=498292} 2026-02-21T22:26:35.203 INFO:tasks.rados.rados.0.trial178.stdout:2505: writing trial17818531-31 from 689318 to 1321186 tid 1 2026-02-21T22:26:35.211 INFO:tasks.rados.rados.0.trial178.stdout:2505: writing trial17818531-31 from 1877968 to 2376260 tid 2 2026-02-21T22:26:35.211 INFO:tasks.rados.rados.0.trial178.stdout:2506: read oid 46 snap 253 2026-02-21T22:26:35.211 INFO:tasks.rados.rados.0.trial178.stdout:2506: expect deleted 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2507: rollback oid 45 current snap is 254 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 245 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: finishing write tid 1 to trial17818531-15 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: finishing write tid 2 to trial17818531-15 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: finishing write tid 3 to trial17818531-15 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: finishing write tid 4 to trial17818531-15 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: oid 15 updating version 0 to 955 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: oid 15 version 955 is already newer than 954 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: oid 15 version 955 is already newer than 953 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 955 (ObjNum 873 snap 254 seq_num 873) dirty exists 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2504: left oid 15 (ObjNum 873 snap 254 seq_num 873) 2026-02-21T22:26:35.212 INFO:tasks.rados.rados.0.trial178.stdout:2505: finishing write tid 1 to trial17818531-31 2026-02-21T22:26:35.213 INFO:tasks.rados.rados.0.trial178.stdout:2504: done (3 left) 2026-02-21T22:26:35.213 INFO:tasks.rados.rados.0.trial178.stdout:2508: rollback oid 32 current snap is 254 2026-02-21T22:26:35.213 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 252 2026-02-21T22:26:35.213 INFO:tasks.rados.rados.0.trial178.stdout:2509: snap_remove snap 250 2026-02-21T22:26:36.190 INFO:tasks.rados.rados.0.trial178.stdout:2509: done (4 left) 2026-02-21T22:26:36.190 INFO:tasks.rados.rados.0.trial178.stdout:2510: snap_remove snap 251 2026-02-21T22:26:37.160 INFO:tasks.rados.rados.0.trial178.stdout:2510: done (4 left) 2026-02-21T22:26:37.161 INFO:tasks.rados.rados.0.trial178.stdout:2511: snap_remove snap 242 2026-02-21T22:26:38.164 INFO:tasks.rados.rados.0.trial178.stdout:2511: done (4 left) 2026-02-21T22:26:38.164 INFO:tasks.rados.rados.0.trial178.stdout:2512: snap_remove snap 244 2026-02-21T22:26:38.572 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:38.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2512: done (4 left) 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2513: read oid 37 snap -1 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2513: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2514: rollback oid 10 current snap is 254 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 239 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2507: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 0 (ObjNum 3 snap 0 seq_num 304) dirty dne 2026-02-21T22:26:39.166 INFO:tasks.rados.rados.0.trial178.stdout:2505: finishing write tid 2 to trial17818531-31 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2505: finishing write tid 3 to trial17818531-31 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2508: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2506: done (5 left) 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2507: done (4 left) 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2515: write (excl) oid 15 current snap is 254 2026-02-21T22:26:39.167 INFO:tasks.rados.rados.0.trial178.stdout:2515: seq_num 875 ranges {706125=423470,1652803=758303} 2026-02-21T22:26:39.173 INFO:tasks.rados.rados.0.trial178.stdout:2515: writing trial17818531-15 from 706125 to 1129595 tid 1 2026-02-21T22:26:39.182 INFO:tasks.rados.rados.0.trial178.stdout:2515: writing trial17818531-15 from 1652803 to 2411106 tid 2 2026-02-21T22:26:39.182 INFO:tasks.rados.rados.0.trial178.stdout:2516: snap_remove snap 253 2026-02-21T22:26:40.170 INFO:tasks.rados.rados.0.trial178.stdout:2516: done (5 left) 2026-02-21T22:26:40.170 INFO:tasks.rados.rados.0.trial178.stdout:2517: write oid 27 current snap is 254 2026-02-21T22:26:40.170 INFO:tasks.rados.rados.0.trial178.stdout:2517: seq_num 876 ranges {722932=498719,1994932=501961} 2026-02-21T22:26:40.178 INFO:tasks.rados.rados.0.trial178.stdout:2517: writing trial17818531-27 from 722932 to 1221651 tid 1 2026-02-21T22:26:40.187 INFO:tasks.rados.rados.0.trial178.stdout:2517: writing trial17818531-27 from 1994932 to 2496893 tid 2 2026-02-21T22:26:40.187 INFO:tasks.rados.rados.0.trial178.stdout:2518: copy_from oid 40 from oid 47 current snap is 254 2026-02-21T22:26:40.187 INFO:tasks.rados.rados.0.trial178.stdout:2505: finishing write tid 4 to trial17818531-31 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2505: oid 31 updating version 0 to 1162 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2505: oid 31 updating version 1162 to 1163 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2505: oid 31 version 1163 is already newer than 1161 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1163 (ObjNum 874 snap 254 seq_num 874) dirty exists 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2505: left oid 31 (ObjNum 874 snap 254 seq_num 874) 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2508: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2505: done (6 left) 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2519: read oid 16 snap 252 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2519: expect deleted 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2520: delete oid 11 current snap is 254 2026-02-21T22:26:40.188 INFO:tasks.rados.rados.0.trial178.stdout:2508: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:26:40.189 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1184 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:26:40.189 INFO:tasks.rados.rados.0.trial178.stdout:2513: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:40.197 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1115910 2026-02-21T22:26:40.281 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2446020 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:2514: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:2514: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:2514: finishing rollback tid 2 to trial17818531-10 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1154 (ObjNum 838 snap 239 seq_num 838) dirty exists 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:2515: finishing write tid 1 to trial17818531-15 2026-02-21T22:26:40.294 INFO:tasks.rados.rados.0.trial178.stdout:2515: finishing write tid 2 to trial17818531-15 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: finishing write tid 3 to trial17818531-15 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: finishing write tid 4 to trial17818531-15 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: oid 15 updating version 0 to 978 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: oid 15 version 978 is already newer than 976 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: oid 15 version 978 is already newer than 977 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 978 (ObjNum 875 snap 254 seq_num 875) dirty exists 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: left oid 15 (ObjNum 875 snap 254 seq_num 875) 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2517: finishing write tid 1 to trial17818531-27 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2517: finishing write tid 2 to trial17818531-27 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2508: done (7 left) 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2513: done (6 left) 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2514: done (5 left) 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2515: done (4 left) 2026-02-21T22:26:40.295 INFO:tasks.rados.rados.0.trial178.stdout:2519: done (3 left) 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2520: done (2 left) 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2521: read oid 12 snap -1 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2521: expect (ObjNum 867 snap 251 seq_num 867) 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2522: snap_create 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: finishing write tid 3 to trial17818531-27 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: finishing write tid 4 to trial17818531-27 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: oid 27 updating version 0 to 1184 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: oid 27 version 1184 is already newer than 1183 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: oid 27 version 1184 is already newer than 1182 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1184 (ObjNum 876 snap 254 seq_num 876) dirty exists 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2517: left oid 27 (ObjNum 876 snap 254 seq_num 876) 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2518: finishing copy_from racing read to trial17818531-40 2026-02-21T22:26:40.296 INFO:tasks.rados.rados.0.trial178.stdout:2518: finishing copy_from to trial17818531-40 2026-02-21T22:26:40.297 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1148 (ObjNum 854 snap 243 seq_num 854) dirty exists 2026-02-21T22:26:40.297 INFO:tasks.rados.rados.0.trial178.stdout:2521: expect (ObjNum 867 snap 251 seq_num 867) 2026-02-21T22:26:40.323 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2316078 2026-02-21T22:26:41.174 INFO:tasks.rados.rados.0.trial178.stdout:2517: done (3 left) 2026-02-21T22:26:41.175 INFO:tasks.rados.rados.0.trial178.stdout:2518: done (2 left) 2026-02-21T22:26:41.175 INFO:tasks.rados.rados.0.trial178.stdout:2521: done (1 left) 2026-02-21T22:26:41.175 INFO:tasks.rados.rados.0.trial178.stdout:2522: done (0 left) 2026-02-21T22:26:41.175 INFO:tasks.rados.rados.0.trial178.stdout:2523: snap_remove snap 254 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2523: done (0 left) 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2524: read oid 47 snap -1 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2524: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2525: read oid 5 snap -1 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2525: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2526: read oid 35 snap 252 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2526: expect deleted 2026-02-21T22:26:42.161 INFO:tasks.rados.rados.0.trial178.stdout:2527: delete oid 44 current snap is 255 2026-02-21T22:26:42.163 INFO:tasks.rados.rados.0.trial178.stdout:2526: done (3 left) 2026-02-21T22:26:42.164 INFO:tasks.rados.rados.0.trial178.stdout:2527: done (2 left) 2026-02-21T22:26:42.164 INFO:tasks.rados.rados.0.trial178.stdout:2528: rollback oid 17 current snap is 255 2026-02-21T22:26:42.164 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 252 2026-02-21T22:26:42.164 INFO:tasks.rados.rados.0.trial178.stdout:2529: write (excl) oid 2 current snap is 255 2026-02-21T22:26:42.164 INFO:tasks.rados.rados.0.trial178.stdout:2529: seq_num 879 ranges {773353=724466,2170378=602975} 2026-02-21T22:26:42.173 INFO:tasks.rados.rados.0.trial178.stdout:2529: writing trial17818531-2 from 773353 to 1497819 tid 1 2026-02-21T22:26:42.182 INFO:tasks.rados.rados.0.trial178.stdout:2529: writing trial17818531-2 from 2170378 to 2773353 tid 2 2026-02-21T22:26:42.183 INFO:tasks.rados.rados.0.trial178.stdout:2530: rollback oid 3 current snap is 255 2026-02-21T22:26:42.183 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 252 2026-02-21T22:26:42.183 INFO:tasks.rados.rados.0.trial178.stdout:2524: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:26:42.303 INFO:tasks.rados.rados.0.trial178.stdout:2528: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:26:42.303 INFO:tasks.rados.rados.0.trial178.stdout:2528: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:26:42.303 INFO:tasks.rados.rados.0.trial178.stdout:2528: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:26:42.303 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 984 (ObjNum 797 snap 224 seq_num 797) dirty exists 2026-02-21T22:26:42.303 INFO:tasks.rados.rados.0.trial178.stdout:2525: expect (ObjNum 791 snap 220 seq_num 791) 2026-02-21T22:26:42.327 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:42.389 INFO:tasks.rados.rados.0.trial178.stdout:2529: finishing write tid 1 to trial17818531-2 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: finishing write tid 2 to trial17818531-2 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: finishing write tid 3 to trial17818531-2 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2530: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2530: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: finishing write tid 4 to trial17818531-2 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: oid 2 updating version 0 to 1155 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: oid 2 updating version 1155 to 1156 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: oid 2 updating version 1156 to 1157 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1157 (ObjNum 879 snap 255 seq_num 879) dirty exists 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2529: left oid 2 (ObjNum 879 snap 255 seq_num 879) 2026-02-21T22:26:42.390 INFO:tasks.rados.rados.0.trial178.stdout:2530: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1212 (ObjNum 759 snap 214 seq_num 759) dirty exists 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2524: done (4 left) 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2525: done (3 left) 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2528: done (2 left) 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2529: done (1 left) 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2530: done (0 left) 2026-02-21T22:26:42.391 INFO:tasks.rados.rados.0.trial178.stdout:2531: snap_create 2026-02-21T22:26:43.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:43.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:43.612 INFO:tasks.rados.rados.0.trial178.stdout:2531: done (0 left) 2026-02-21T22:26:43.612 INFO:tasks.rados.rados.0.trial178.stdout:2532: write (excl) oid 38 current snap is 256 2026-02-21T22:26:43.612 INFO:tasks.rados.rados.0.trial178.stdout:2532: seq_num 880 ranges {790160=799715,2112507=509299} 2026-02-21T22:26:43.624 INFO:tasks.rados.rados.0.trial178.stdout:2532: writing trial17818531-38 from 790160 to 1589875 tid 1 2026-02-21T22:26:43.631 INFO:tasks.rados.rados.0.trial178.stdout:2532: writing trial17818531-38 from 2112507 to 2621806 tid 2 2026-02-21T22:26:43.631 INFO:tasks.rados.rados.0.trial178.stdout:2533: snap_remove snap 245 2026-02-21T22:26:44.615 INFO:tasks.rados.rados.0.trial178.stdout:2533: done (1 left) 2026-02-21T22:26:44.615 INFO:tasks.rados.rados.0.trial178.stdout:2534: snap_create 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: finishing write tid 1 to trial17818531-38 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: finishing write tid 2 to trial17818531-38 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: finishing write tid 3 to trial17818531-38 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: finishing write tid 4 to trial17818531-38 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: oid 38 updating version 0 to 851 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: oid 38 updating version 851 to 853 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: oid 38 version 853 is already newer than 852 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 853 (ObjNum 880 snap 256 seq_num 880) dirty exists 2026-02-21T22:26:44.616 INFO:tasks.rados.rados.0.trial178.stdout:2532: left oid 38 (ObjNum 880 snap 256 seq_num 880) 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:2532: done (1 left) 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:2534: done (0 left) 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:2535: rollback oid 29 current snap is 257 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 239 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:2536: write (excl) oid 47 current snap is 257 2026-02-21T22:26:45.617 INFO:tasks.rados.rados.0.trial178.stdout:2536: seq_num 881 ranges {406967=474964,1370989=769310,2677186=129781} 2026-02-21T22:26:45.622 INFO:tasks.rados.rados.0.trial178.stdout:2536: writing trial17818531-47 from 406967 to 881931 tid 1 2026-02-21T22:26:45.631 INFO:tasks.rados.rados.0.trial178.stdout:2536: writing trial17818531-47 from 1370989 to 2140299 tid 2 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2536: writing trial17818531-47 from 2677186 to 2806967 tid 3 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2537: snap_create 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2535: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2535: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2535: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 803 (ObjNum 835 snap 239 seq_num 835) dirty exists 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2536: finishing write tid 1 to trial17818531-47 2026-02-21T22:26:45.636 INFO:tasks.rados.rados.0.trial178.stdout:2536: finishing write tid 2 to trial17818531-47 2026-02-21T22:26:45.637 INFO:tasks.rados.rados.0.trial178.stdout:2536: finishing write tid 3 to trial17818531-47 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: finishing write tid 4 to trial17818531-47 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: finishing write tid 5 to trial17818531-47 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: oid 47 updating version 0 to 1409 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: oid 47 version 1409 is already newer than 1407 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: oid 47 version 1409 is already newer than 1406 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: oid 47 version 1409 is already newer than 1408 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1409 (ObjNum 881 snap 257 seq_num 881) dirty exists 2026-02-21T22:26:45.638 INFO:tasks.rados.rados.0.trial178.stdout:2536: left oid 47 (ObjNum 881 snap 257 seq_num 881) 2026-02-21T22:26:46.622 INFO:tasks.rados.rados.0.trial178.stdout:2535: done (2 left) 2026-02-21T22:26:46.623 INFO:tasks.rados.rados.0.trial178.stdout:2536: done (1 left) 2026-02-21T22:26:46.623 INFO:tasks.rados.rados.0.trial178.stdout:2537: done (0 left) 2026-02-21T22:26:46.623 INFO:tasks.rados.rados.0.trial178.stdout:2538: copy_from oid 23 from oid 7 current snap is 258 2026-02-21T22:26:46.623 INFO:tasks.rados.rados.0.trial178.stdout:2539: snap_create 2026-02-21T22:26:46.636 INFO:tasks.rados.rados.0.trial178.stdout:2538: finishing copy_from racing read to trial17818531-23 2026-02-21T22:26:46.636 INFO:tasks.rados.rados.0.trial178.stdout:2538: finishing copy_from to trial17818531-23 2026-02-21T22:26:46.636 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1370 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:26:47.310 INFO:tasks.rados.rados.0.trial178.stdout:2538: done (1 left) 2026-02-21T22:26:47.310 INFO:tasks.rados.rados.0.trial178.stdout:2539: done (0 left) 2026-02-21T22:26:47.310 INFO:tasks.rados.rados.0.trial178.stdout:2540: snap_remove snap 257 2026-02-21T22:26:48.167 INFO:tasks.rados.rados.0.trial178.stdout:2540: done (0 left) 2026-02-21T22:26:48.167 INFO:tasks.rados.rados.0.trial178.stdout:2541: rollback oid 40 current snap is 259 2026-02-21T22:26:48.167 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 256 2026-02-21T22:26:48.167 INFO:tasks.rados.rados.0.trial178.stdout:2542: snap_remove snap 252 2026-02-21T22:26:48.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:48.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:49.170 INFO:tasks.rados.rados.0.trial178.stdout:2542: done (1 left) 2026-02-21T22:26:49.170 INFO:tasks.rados.rados.0.trial178.stdout:2543: delete oid 1 current snap is 259 2026-02-21T22:26:49.171 INFO:tasks.rados.rados.0.trial178.stdout:2541: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:26:49.171 INFO:tasks.rados.rados.0.trial178.stdout:2541: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:26:49.171 INFO:tasks.rados.rados.0.trial178.stdout:2541: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:26:49.171 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1165 (ObjNum 854 snap 243 seq_num 854) dirty exists 2026-02-21T22:26:49.173 INFO:tasks.rados.rados.0.trial178.stdout:2541: done (1 left) 2026-02-21T22:26:49.173 INFO:tasks.rados.rados.0.trial178.stdout:2543: done (0 left) 2026-02-21T22:26:49.173 INFO:tasks.rados.rados.0.trial178.stdout:2544: write oid 13 current snap is 259 2026-02-21T22:26:49.173 INFO:tasks.rados.rados.0.trial178.stdout:2544: seq_num 883 ranges {440581=741815,1604306=656626,2732032=108549} 2026-02-21T22:26:49.181 INFO:tasks.rados.rados.0.trial178.stdout:2544: writing trial17818531-13 from 440581 to 1182396 tid 1 2026-02-21T22:26:49.188 INFO:tasks.rados.rados.0.trial178.stdout:2544: writing trial17818531-13 from 1604306 to 2260932 tid 2 2026-02-21T22:26:49.192 INFO:tasks.rados.rados.0.trial178.stdout:2544: writing trial17818531-13 from 2732032 to 2840581 tid 3 2026-02-21T22:26:49.192 INFO:tasks.rados.rados.0.trial178.stdout:2545: delete oid 44 current snap is 259 2026-02-21T22:26:49.192 INFO:tasks.rados.rados.0.trial178.stdout:2544: finishing write tid 1 to trial17818531-13 2026-02-21T22:26:49.192 INFO:tasks.rados.rados.0.trial178.stdout:2544: finishing write tid 2 to trial17818531-13 2026-02-21T22:26:49.193 INFO:tasks.rados.rados.0.trial178.stdout:2545: done (1 left) 2026-02-21T22:26:49.193 INFO:tasks.rados.rados.0.trial178.stdout:2546: write (excl) oid 19 current snap is 259 2026-02-21T22:26:49.193 INFO:tasks.rados.rados.0.trial178.stdout:2546: seq_num 885 ranges {474195=492313,1604917=660295,2670525=203670} 2026-02-21T22:26:49.199 INFO:tasks.rados.rados.0.trial178.stdout:2546: writing trial17818531-19 from 474195 to 966508 tid 1 2026-02-21T22:26:49.208 INFO:tasks.rados.rados.0.trial178.stdout:2546: writing trial17818531-19 from 1604917 to 2265212 tid 2 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2546: writing trial17818531-19 from 2670525 to 2874195 tid 3 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2547: delete oid 31 current snap is 259 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2544: finishing write tid 3 to trial17818531-13 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2544: finishing write tid 4 to trial17818531-13 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2544: finishing write tid 5 to trial17818531-13 2026-02-21T22:26:49.213 INFO:tasks.rados.rados.0.trial178.stdout:2544: oid 13 updating version 0 to 1231 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2544: oid 13 version 1231 is already newer than 1230 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2544: oid 13 version 1231 is already newer than 1229 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2544: oid 13 updating version 1231 to 1232 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1232 (ObjNum 883 snap 259 seq_num 883) dirty exists 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2544: left oid 13 (ObjNum 883 snap 259 seq_num 883) 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2546: finishing write tid 1 to trial17818531-19 2026-02-21T22:26:49.214 INFO:tasks.rados.rados.0.trial178.stdout:2546: finishing write tid 2 to trial17818531-19 2026-02-21T22:26:49.215 INFO:tasks.rados.rados.0.trial178.stdout:2544: done (2 left) 2026-02-21T22:26:49.215 INFO:tasks.rados.rados.0.trial178.stdout:2547: done (1 left) 2026-02-21T22:26:49.215 INFO:tasks.rados.rados.0.trial178.stdout:2548: read oid 48 snap -1 2026-02-21T22:26:49.215 INFO:tasks.rados.rados.0.trial178.stdout:2548: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:26:49.215 INFO:tasks.rados.rados.0.trial178.stdout:2549: snap_remove snap 239 2026-02-21T22:26:50.175 INFO:tasks.rados.rados.0.trial178.stdout:2549: done (2 left) 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2550: snap_create 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: finishing write tid 3 to trial17818531-19 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: finishing write tid 4 to trial17818531-19 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: finishing write tid 5 to trial17818531-19 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: oid 19 updating version 0 to 1416 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: oid 19 version 1416 is already newer than 1415 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: oid 19 version 1416 is already newer than 1413 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: oid 19 version 1416 is already newer than 1414 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1416 (ObjNum 885 snap 259 seq_num 885) dirty exists 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2546: left oid 19 (ObjNum 885 snap 259 seq_num 885) 2026-02-21T22:26:50.176 INFO:tasks.rados.rados.0.trial178.stdout:2548: expect (ObjNum 724 snap 204 seq_num 724) 2026-02-21T22:26:50.236 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:50.287 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1330657 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2546: done (2 left) 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2548: done (1 left) 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2550: done (0 left) 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2551: read oid 22 snap -1 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2551: expect (ObjNum 870 snap 253 seq_num 870) 2026-02-21T22:26:51.178 INFO:tasks.rados.rados.0.trial178.stdout:2552: delete oid 25 current snap is 260 2026-02-21T22:26:51.183 INFO:tasks.rados.rados.0.trial178.stdout:2552: done (1 left) 2026-02-21T22:26:51.183 INFO:tasks.rados.rados.0.trial178.stdout:2553: snap_create 2026-02-21T22:26:51.184 INFO:tasks.rados.rados.0.trial178.stdout:2551: expect (ObjNum 870 snap 253 seq_num 870) 2026-02-21T22:26:51.224 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:51.268 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:53.016 INFO:tasks.rados.rados.0.trial178.stdout:2551: done (1 left) 2026-02-21T22:26:53.017 INFO:tasks.rados.rados.0.trial178.stdout:2553: done (0 left) 2026-02-21T22:26:53.017 INFO:tasks.rados.rados.0.trial178.stdout:2554: snap_remove snap 259 2026-02-21T22:26:53.181 INFO:tasks.rados.rados.0.trial178.stdout:2554: done (0 left) 2026-02-21T22:26:53.181 INFO:tasks.rados.rados.0.trial178.stdout:2555: read oid 22 snap -1 2026-02-21T22:26:53.182 INFO:tasks.rados.rados.0.trial178.stdout:2555: expect (ObjNum 870 snap 253 seq_num 870) 2026-02-21T22:26:53.182 INFO:tasks.rados.rados.0.trial178.stdout:2556: write (excl) oid 7 current snap is 261 2026-02-21T22:26:53.182 INFO:tasks.rados.rados.0.trial178.stdout:2556: seq_num 888 ranges {524616=718060,1664010=523975,2752794=171822} 2026-02-21T22:26:53.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: writing trial17818531-7 from 524616 to 1242676 tid 1 2026-02-21T22:26:53.196 INFO:tasks.rados.rados.0.trial178.stdout:2556: writing trial17818531-7 from 1664010 to 2187985 tid 2 2026-02-21T22:26:53.201 INFO:tasks.rados.rados.0.trial178.stdout:2556: writing trial17818531-7 from 2752794 to 2924616 tid 3 2026-02-21T22:26:53.201 INFO:tasks.rados.rados.0.trial178.stdout:2557: write oid 9 current snap is 261 2026-02-21T22:26:53.201 INFO:tasks.rados.rados.0.trial178.stdout:2557: seq_num 889 ranges {541423=793309,2122492=667633} 2026-02-21T22:26:53.210 INFO:tasks.rados.rados.0.trial178.stdout:2557: writing trial17818531-9 from 541423 to 1334732 tid 1 2026-02-21T22:26:53.220 INFO:tasks.rados.rados.0.trial178.stdout:2557: writing trial17818531-9 from 2122492 to 2790125 tid 2 2026-02-21T22:26:53.220 INFO:tasks.rados.rados.0.trial178.stdout:2558: write oid 16 current snap is 261 2026-02-21T22:26:53.220 INFO:tasks.rados.rados.0.trial178.stdout:2558: seq_num 890 ranges {558230=584911,1897327=527644,2923993=34237} 2026-02-21T22:26:53.226 INFO:tasks.rados.rados.0.trial178.stdout:2558: writing trial17818531-16 from 558230 to 1143141 tid 1 2026-02-21T22:26:53.231 INFO:tasks.rados.rados.0.trial178.stdout:2558: writing trial17818531-16 from 1897327 to 2424971 tid 2 2026-02-21T22:26:53.233 INFO:tasks.rados.rados.0.trial178.stdout:2558: writing trial17818531-16 from 2923993 to 2958230 tid 3 2026-02-21T22:26:53.233 INFO:tasks.rados.rados.0.trial178.stdout:2559: write (excl) oid 1 current snap is 261 2026-02-21T22:26:53.234 INFO:tasks.rados.rados.0.trial178.stdout:2559: seq_num 891 ranges {575037=660160,1955809=671302} 2026-02-21T22:26:53.238 INFO:tasks.rados.rados.0.trial178.stdout:2559: writing trial17818531-1 from 575037 to 1235197 tid 1 2026-02-21T22:26:53.244 INFO:tasks.rados.rados.0.trial178.stdout:2559: writing trial17818531-1 from 1955809 to 2627111 tid 2 2026-02-21T22:26:53.244 INFO:tasks.rados.rados.0.trial178.stdout:2560: write (excl) oid 5 current snap is 261 2026-02-21T22:26:53.244 INFO:tasks.rados.rados.0.trial178.stdout:2560: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2026-02-21T22:26:53.248 INFO:tasks.rados.rados.0.trial178.stdout:2560: writing trial17818531-5 from 591844 to 1327253 tid 1 2026-02-21T22:26:53.253 INFO:tasks.rados.rados.0.trial178.stdout:2560: writing trial17818531-5 from 1897938 to 2429251 tid 2 2026-02-21T22:26:53.255 INFO:tasks.rados.rados.0.trial178.stdout:2560: writing trial17818531-5 from 2862486 to 2991844 tid 3 2026-02-21T22:26:53.255 INFO:tasks.rados.rados.0.trial178.stdout:2561: read oid 41 snap -1 2026-02-21T22:26:53.255 INFO:tasks.rados.rados.0.trial178.stdout:2561: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:26:53.256 INFO:tasks.rados.rados.0.trial178.stdout:2562: read oid 31 snap -1 2026-02-21T22:26:53.256 INFO:tasks.rados.rados.0.trial178.stdout:2562: expect deleted 2026-02-21T22:26:53.256 INFO:tasks.rados.rados.0.trial178.stdout:2563: delete oid 26 current snap is 261 2026-02-21T22:26:53.256 INFO:tasks.rados.rados.0.trial178.stdout:2555: expect (ObjNum 870 snap 253 seq_num 870) 2026-02-21T22:26:53.259 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 639331 2026-02-21T22:26:53.343 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2473755 2026-02-21T22:26:53.391 INFO:tasks.rados.rados.0.trial178.stdout:2556: finishing write tid 1 to trial17818531-7 2026-02-21T22:26:53.391 INFO:tasks.rados.rados.0.trial178.stdout:2556: finishing write tid 2 to trial17818531-7 2026-02-21T22:26:53.391 INFO:tasks.rados.rados.0.trial178.stdout:2555: done (8 left) 2026-02-21T22:26:53.391 INFO:tasks.rados.rados.0.trial178.stdout:2563: done (7 left) 2026-02-21T22:26:53.391 INFO:tasks.rados.rados.0.trial178.stdout:2564: write oid 23 current snap is 261 2026-02-21T22:26:53.392 INFO:tasks.rados.rados.0.trial178.stdout:2564: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2026-02-21T22:26:53.395 INFO:tasks.rados.rados.0.trial178.stdout:2564: writing trial17818531-23 from 625458 to 1111365 tid 1 2026-02-21T22:26:53.399 INFO:tasks.rados.rados.0.trial178.stdout:2564: writing trial17818531-23 from 1614902 to 2149884 tid 2 2026-02-21T22:26:53.402 INFO:tasks.rados.rados.0.trial178.stdout:2564: writing trial17818531-23 from 2917332 to 3025458 tid 3 2026-02-21T22:26:53.402 INFO:tasks.rados.rados.0.trial178.stdout:2565: snap_remove snap 255 2026-02-21T22:26:53.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:53.537 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:53.548 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:26:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:26:53.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:53.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:54.188 INFO:tasks.rados.rados.0.trial178.stdout:2565: done (8 left) 2026-02-21T22:26:54.188 INFO:tasks.rados.rados.0.trial178.stdout:2566: snap_create 2026-02-21T22:26:54.188 INFO:tasks.rados.rados.0.trial178.stdout:2556: finishing write tid 3 to trial17818531-7 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: finishing write tid 4 to trial17818531-7 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: finishing write tid 5 to trial17818531-7 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: oid 7 updating version 0 to 1427 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: oid 7 version 1427 is already newer than 1424 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: oid 7 version 1427 is already newer than 1425 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: oid 7 version 1427 is already newer than 1426 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1427 (ObjNum 888 snap 261 seq_num 888) dirty exists 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2556: left oid 7 (ObjNum 888 snap 261 seq_num 888) 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2557: finishing write tid 1 to trial17818531-9 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2557: finishing write tid 2 to trial17818531-9 2026-02-21T22:26:54.189 INFO:tasks.rados.rados.0.trial178.stdout:2557: finishing write tid 3 to trial17818531-9 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2557: finishing write tid 4 to trial17818531-9 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2557: oid 9 updating version 0 to 1186 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2557: oid 9 version 1186 is already newer than 1185 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2557: oid 9 updating version 1186 to 1187 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1187 (ObjNum 889 snap 261 seq_num 889) dirty exists 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2557: left oid 9 (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: finishing write tid 1 to trial17818531-16 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: finishing write tid 2 to trial17818531-16 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: finishing write tid 3 to trial17818531-16 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: finishing write tid 4 to trial17818531-16 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: finishing write tid 5 to trial17818531-16 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: oid 16 updating version 0 to 834 2026-02-21T22:26:54.190 INFO:tasks.rados.rados.0.trial178.stdout:2558: oid 16 updating version 834 to 836 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2558: oid 16 version 836 is already newer than 835 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2558: oid 16 version 836 is already newer than 833 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 836 (ObjNum 890 snap 261 seq_num 890) dirty exists 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2558: left oid 16 (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: finishing write tid 1 to trial17818531-1 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: finishing write tid 2 to trial17818531-1 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: finishing write tid 3 to trial17818531-1 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: finishing write tid 4 to trial17818531-1 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: oid 1 updating version 0 to 1283 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: oid 1 updating version 1283 to 1284 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:2559: oid 1 version 1284 is already newer than 1282 2026-02-21T22:26:54.191 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1284 (ObjNum 891 snap 261 seq_num 891) dirty exists 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2559: left oid 1 (ObjNum 891 snap 261 seq_num 891) 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: finishing write tid 1 to trial17818531-5 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: finishing write tid 2 to trial17818531-5 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: finishing write tid 3 to trial17818531-5 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: finishing write tid 4 to trial17818531-5 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: finishing write tid 5 to trial17818531-5 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: oid 5 updating version 0 to 1153 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: oid 5 version 1153 is already newer than 1152 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: oid 5 updating version 1153 to 1155 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: oid 5 version 1155 is already newer than 1154 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1155 (ObjNum 892 snap 261 seq_num 892) dirty exists 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2560: left oid 5 (ObjNum 892 snap 261 seq_num 892) 2026-02-21T22:26:54.192 INFO:tasks.rados.rados.0.trial178.stdout:2561: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:26:54.198 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1392022 2026-02-21T22:26:54.281 INFO:tasks.rados.rados.0.trial178.stdout:2564: finishing write tid 1 to trial17818531-23 2026-02-21T22:26:54.281 INFO:tasks.rados.rados.0.trial178.stdout:2564: finishing write tid 2 to trial17818531-23 2026-02-21T22:26:54.281 INFO:tasks.rados.rados.0.trial178.stdout:2564: finishing write tid 3 to trial17818531-23 2026-02-21T22:26:54.281 INFO:tasks.rados.rados.0.trial178.stdout:2564: finishing write tid 4 to trial17818531-23 2026-02-21T22:26:54.281 INFO:tasks.rados.rados.0.trial178.stdout:2564: finishing write tid 5 to trial17818531-23 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:2564: oid 23 updating version 0 to 1373 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:2564: oid 23 version 1373 is already newer than 1372 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:2564: oid 23 version 1373 is already newer than 1371 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:2564: oid 23 updating version 1373 to 1374 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1374 (ObjNum 894 snap 261 seq_num 894) dirty exists 2026-02-21T22:26:54.282 INFO:tasks.rados.rados.0.trial178.stdout:2564: left oid 23 (ObjNum 894 snap 261 seq_num 894) 2026-02-21T22:26:55.188 INFO:tasks.rados.rados.0.trial178.stdout:2556: done (8 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2557: done (7 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2558: done (6 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2559: done (5 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2560: done (4 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2561: done (3 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2562: done (2 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2564: done (1 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2566: done (0 left) 2026-02-21T22:26:55.189 INFO:tasks.rados.rados.0.trial178.stdout:2567: write oid 4 current snap is 262 2026-02-21T22:26:55.190 INFO:tasks.rados.rados.0.trial178.stdout:2567: seq_num 895 ranges {642265=561156,1673384=678640,2828402=213863} 2026-02-21T22:26:55.196 INFO:tasks.rados.rados.0.trial178.stdout:2567: writing trial17818531-4 from 642265 to 1203421 tid 1 2026-02-21T22:26:55.204 INFO:tasks.rados.rados.0.trial178.stdout:2567: writing trial17818531-4 from 1673384 to 2352024 tid 2 2026-02-21T22:26:55.209 INFO:tasks.rados.rados.0.trial178.stdout:2567: writing trial17818531-4 from 2828402 to 3042265 tid 3 2026-02-21T22:26:55.209 INFO:tasks.rados.rados.0.trial178.stdout:2568: read oid 36 snap -1 2026-02-21T22:26:55.209 INFO:tasks.rados.rados.0.trial178.stdout:2568: expect (ObjNum 825 snap 232 seq_num 825) 2026-02-21T22:26:55.209 INFO:tasks.rados.rados.0.trial178.stdout:2569: read oid 41 snap -1 2026-02-21T22:26:55.209 INFO:tasks.rados.rados.0.trial178.stdout:2569: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:26:55.210 INFO:tasks.rados.rados.0.trial178.stdout:2570: snap_remove snap 256 2026-02-21T22:26:56.195 INFO:tasks.rados.rados.0.trial178.stdout:2570: done (3 left) 2026-02-21T22:26:56.195 INFO:tasks.rados.rados.0.trial178.stdout:2571: read oid 23 snap -1 2026-02-21T22:26:56.195 INFO:tasks.rados.rados.0.trial178.stdout:2571: expect (ObjNum 894 snap 261 seq_num 894) 2026-02-21T22:26:56.195 INFO:tasks.rados.rados.0.trial178.stdout:2572: snap_remove snap 260 2026-02-21T22:26:57.173 INFO:tasks.rados.rados.0.trial178.stdout:2572: done (4 left) 2026-02-21T22:26:57.173 INFO:tasks.rados.rados.0.trial178.stdout:2573: snap_create 2026-02-21T22:26:57.173 INFO:tasks.rados.rados.0.trial178.stdout:2567: finishing write tid 1 to trial17818531-4 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: finishing write tid 2 to trial17818531-4 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: finishing write tid 3 to trial17818531-4 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: finishing write tid 4 to trial17818531-4 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: finishing write tid 5 to trial17818531-4 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: oid 4 updating version 0 to 837 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: oid 4 updating version 837 to 839 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: oid 4 version 839 is already newer than 838 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: oid 4 updating version 839 to 840 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 840 (ObjNum 895 snap 262 seq_num 895) dirty exists 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2567: left oid 4 (ObjNum 895 snap 262 seq_num 895) 2026-02-21T22:26:57.174 INFO:tasks.rados.rados.0.trial178.stdout:2569: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:26:57.175 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:57.245 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 562226 2026-02-21T22:26:57.273 INFO:tasks.rados.rados.0.trial178.stdout:2568: expect (ObjNum 825 snap 232 seq_num 825) 2026-02-21T22:26:57.321 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 987351 2026-02-21T22:26:57.359 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 5083 2026-02-21T22:26:57.407 INFO:tasks.rados.rados.0.trial178.stdout:2571: expect (ObjNum 894 snap 261 seq_num 894) 2026-02-21T22:26:57.469 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2567: done (4 left) 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2568: done (3 left) 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2569: done (2 left) 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2571: done (1 left) 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2573: done (0 left) 2026-02-21T22:26:58.177 INFO:tasks.rados.rados.0.trial178.stdout:2574: write oid 6 current snap is 263 2026-02-21T22:26:58.178 INFO:tasks.rados.rados.0.trial178.stdout:2574: seq_num 896 ranges {659072=636405,1731866=422298,2855825=203247} 2026-02-21T22:26:58.185 INFO:tasks.rados.rados.0.trial178.stdout:2574: writing trial17818531-6 from 659072 to 1295477 tid 1 2026-02-21T22:26:58.191 INFO:tasks.rados.rados.0.trial178.stdout:2574: writing trial17818531-6 from 1731866 to 2154164 tid 2 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2574: writing trial17818531-6 from 2855825 to 3059072 tid 3 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2575: read oid 38 snap -1 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2575: expect (ObjNum 880 snap 256 seq_num 880) 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2576: copy_from oid 30 from oid 19 current snap is 263 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2574: finishing write tid 1 to trial17818531-6 2026-02-21T22:26:58.198 INFO:tasks.rados.rados.0.trial178.stdout:2574: finishing write tid 2 to trial17818531-6 2026-02-21T22:26:58.199 INFO:tasks.rados.rados.0.trial178.stdout:2577: snap_create 2026-02-21T22:26:58.200 INFO:tasks.rados.rados.0.trial178.stdout:2574: finishing write tid 3 to trial17818531-6 2026-02-21T22:26:58.200 INFO:tasks.rados.rados.0.trial178.stdout:2574: finishing write tid 4 to trial17818531-6 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: finishing write tid 5 to trial17818531-6 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: oid 6 updating version 0 to 1160 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: oid 6 version 1160 is already newer than 1159 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: oid 6 updating version 1160 to 1161 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: oid 6 updating version 1161 to 1162 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1162 (ObjNum 896 snap 263 seq_num 896) dirty exists 2026-02-21T22:26:58.201 INFO:tasks.rados.rados.0.trial178.stdout:2574: left oid 6 (ObjNum 896 snap 263 seq_num 896) 2026-02-21T22:26:58.202 INFO:tasks.rados.rados.0.trial178.stdout:2575: expect (ObjNum 880 snap 256 seq_num 880) 2026-02-21T22:26:58.256 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1437412 2026-02-21T22:26:58.273 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:26:58.301 INFO:tasks.rados.rados.0.trial178.stdout:2576: finishing copy_from racing read to trial17818531-30 2026-02-21T22:26:58.301 INFO:tasks.rados.rados.0.trial178.stdout:2576: finishing copy_from to trial17818531-30 2026-02-21T22:26:58.301 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1416 (ObjNum 885 snap 259 seq_num 885) dirty exists 2026-02-21T22:26:58.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:26:58.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:26:59.197 INFO:tasks.rados.rados.0.trial178.stdout:2574: done (3 left) 2026-02-21T22:26:59.197 INFO:tasks.rados.rados.0.trial178.stdout:2575: done (2 left) 2026-02-21T22:26:59.197 INFO:tasks.rados.rados.0.trial178.stdout:2576: done (1 left) 2026-02-21T22:26:59.198 INFO:tasks.rados.rados.0.trial178.stdout:2577: done (0 left) 2026-02-21T22:26:59.198 INFO:tasks.rados.rados.0.trial178.stdout:2578: write oid 14 current snap is 264 2026-02-21T22:26:59.198 INFO:tasks.rados.rados.0.trial178.stdout:2578: seq_num 897 ranges {675879=711654,2073995=682309} 2026-02-21T22:26:59.206 INFO:tasks.rados.rados.0.trial178.stdout:2578: writing trial17818531-14 from 675879 to 1387533 tid 1 2026-02-21T22:26:59.215 INFO:tasks.rados.rados.0.trial178.stdout:2578: writing trial17818531-14 from 2073995 to 2756304 tid 2 2026-02-21T22:26:59.216 INFO:tasks.rados.rados.0.trial178.stdout:2579: write (excl) oid 48 current snap is 264 2026-02-21T22:26:59.216 INFO:tasks.rados.rados.0.trial178.stdout:2579: seq_num 898 ranges {692686=503256,1848830=425967,2910671=182015} 2026-02-21T22:26:59.224 INFO:tasks.rados.rados.0.trial178.stdout:2579: writing trial17818531-48 from 692686 to 1195942 tid 1 2026-02-21T22:26:59.231 INFO:tasks.rados.rados.0.trial178.stdout:2579: writing trial17818531-48 from 1848830 to 2274797 tid 2 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2579: writing trial17818531-48 from 2910671 to 3092686 tid 3 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2580: snap_create 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2578: finishing write tid 1 to trial17818531-14 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2578: finishing write tid 2 to trial17818531-14 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2578: finishing write tid 3 to trial17818531-14 2026-02-21T22:26:59.237 INFO:tasks.rados.rados.0.trial178.stdout:2578: finishing write tid 4 to trial17818531-14 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2578: oid 14 updating version 0 to 1189 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2578: oid 14 updating version 1189 to 1190 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2578: oid 14 version 1190 is already newer than 1188 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1190 (ObjNum 897 snap 264 seq_num 897) dirty exists 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2578: left oid 14 (ObjNum 897 snap 264 seq_num 897) 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2579: finishing write tid 1 to trial17818531-48 2026-02-21T22:26:59.238 INFO:tasks.rados.rados.0.trial178.stdout:2579: finishing write tid 2 to trial17818531-48 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: finishing write tid 3 to trial17818531-48 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: finishing write tid 4 to trial17818531-48 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: finishing write tid 5 to trial17818531-48 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: oid 48 updating version 0 to 1285 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: oid 48 updating version 1285 to 1288 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: oid 48 version 1288 is already newer than 1286 2026-02-21T22:26:59.239 INFO:tasks.rados.rados.0.trial178.stdout:2579: oid 48 version 1288 is already newer than 1287 2026-02-21T22:26:59.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1288 (ObjNum 898 snap 264 seq_num 898) dirty exists 2026-02-21T22:26:59.240 INFO:tasks.rados.rados.0.trial178.stdout:2579: left oid 48 (ObjNum 898 snap 264 seq_num 898) 2026-02-21T22:27:00.200 INFO:tasks.rados.rados.0.trial178.stdout:2578: done (2 left) 2026-02-21T22:27:00.200 INFO:tasks.rados.rados.0.trial178.stdout:2579: done (1 left) 2026-02-21T22:27:00.201 INFO:tasks.rados.rados.0.trial178.stdout:2580: done (0 left) 2026-02-21T22:27:00.201 INFO:tasks.rados.rados.0.trial178.stdout:2581: write oid 38 current snap is 265 2026-02-21T22:27:00.201 INFO:tasks.rados.rados.0.trial178.stdout:2581: seq_num 899 ranges {709493=578505,1907312=685978} 2026-02-21T22:27:00.208 INFO:tasks.rados.rados.0.trial178.stdout:2581: writing trial17818531-38 from 709493 to 1287998 tid 1 2026-02-21T22:27:00.218 INFO:tasks.rados.rados.0.trial178.stdout:2581: writing trial17818531-38 from 1907312 to 2593290 tid 2 2026-02-21T22:27:00.218 INFO:tasks.rados.rados.0.trial178.stdout:2582: write (excl) oid 2 current snap is 265 2026-02-21T22:27:00.218 INFO:tasks.rados.rados.0.trial178.stdout:2582: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2026-02-21T22:27:00.226 INFO:tasks.rados.rados.0.trial178.stdout:2582: writing trial17818531-2 from 726300 to 1380054 tid 1 2026-02-21T22:27:00.233 INFO:tasks.rados.rados.0.trial178.stdout:2582: writing trial17818531-2 from 1965794 to 2395430 tid 2 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2582: writing trial17818531-2 from 2965517 to 3126300 tid 3 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2583: snap_create 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2581: finishing write tid 1 to trial17818531-38 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2581: finishing write tid 2 to trial17818531-38 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2581: finishing write tid 3 to trial17818531-38 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2581: finishing write tid 4 to trial17818531-38 2026-02-21T22:27:00.239 INFO:tasks.rados.rados.0.trial178.stdout:2581: oid 38 updating version 0 to 856 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:2581: oid 38 version 856 is already newer than 855 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:2581: oid 38 version 856 is already newer than 854 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 856 (ObjNum 899 snap 265 seq_num 899) dirty exists 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:2581: left oid 38 (ObjNum 899 snap 265 seq_num 899) 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:2582: finishing write tid 1 to trial17818531-2 2026-02-21T22:27:00.240 INFO:tasks.rados.rados.0.trial178.stdout:2582: finishing write tid 2 to trial17818531-2 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: finishing write tid 3 to trial17818531-2 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: finishing write tid 4 to trial17818531-2 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: finishing write tid 5 to trial17818531-2 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: oid 2 updating version 0 to 1164 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: oid 2 version 1164 is already newer than 1163 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: oid 2 updating version 1164 to 1165 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:2582: oid 2 updating version 1165 to 1166 2026-02-21T22:27:00.241 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1166 (ObjNum 900 snap 265 seq_num 900) dirty exists 2026-02-21T22:27:00.242 INFO:tasks.rados.rados.0.trial178.stdout:2582: left oid 2 (ObjNum 900 snap 265 seq_num 900) 2026-02-21T22:27:01.203 INFO:tasks.rados.rados.0.trial178.stdout:2581: done (2 left) 2026-02-21T22:27:01.203 INFO:tasks.rados.rados.0.trial178.stdout:2582: done (1 left) 2026-02-21T22:27:01.204 INFO:tasks.rados.rados.0.trial178.stdout:2583: done (0 left) 2026-02-21T22:27:01.204 INFO:tasks.rados.rados.0.trial178.stdout:2584: copy_from oid 30 from oid 41 current snap is 266 2026-02-21T22:27:01.204 INFO:tasks.rados.rados.0.trial178.stdout:2585: snap_create 2026-02-21T22:27:01.214 INFO:tasks.rados.rados.0.trial178.stdout:2584: finishing copy_from racing read to trial17818531-30 2026-02-21T22:27:01.215 INFO:tasks.rados.rados.0.trial178.stdout:2584: finishing copy_from to trial17818531-30 2026-02-21T22:27:01.215 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:27:02.165 INFO:tasks.rados.rados.0.trial178.stdout:2584: done (1 left) 2026-02-21T22:27:02.166 INFO:tasks.rados.rados.0.trial178.stdout:2585: done (0 left) 2026-02-21T22:27:02.166 INFO:tasks.rados.rados.0.trial178.stdout:2586: write (excl) oid 25 current snap is 267 2026-02-21T22:27:02.166 INFO:tasks.rados.rados.0.trial178.stdout:2586: seq_num 901 ranges {743107=729003,1907923=689647} 2026-02-21T22:27:02.175 INFO:tasks.rados.rados.0.trial178.stdout:2586: writing trial17818531-25 from 743107 to 1472110 tid 1 2026-02-21T22:27:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2586: writing trial17818531-25 from 1907923 to 2597570 tid 2 2026-02-21T22:27:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2587: read oid 11 snap -1 2026-02-21T22:27:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2587: expect deleted 2026-02-21T22:27:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2588: snap_create 2026-02-21T22:27:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2586: finishing write tid 1 to trial17818531-25 2026-02-21T22:27:02.185 INFO:tasks.rados.rados.0.trial178.stdout:2586: finishing write tid 2 to trial17818531-25 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: finishing write tid 3 to trial17818531-25 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: finishing write tid 4 to trial17818531-25 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: oid 25 updating version 0 to 1166 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: oid 25 version 1166 is already newer than 1165 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: oid 25 updating version 1166 to 1167 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1167 (ObjNum 901 snap 267 seq_num 901) dirty exists 2026-02-21T22:27:02.186 INFO:tasks.rados.rados.0.trial178.stdout:2586: left oid 25 (ObjNum 901 snap 267 seq_num 901) 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2586: done (2 left) 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2587: done (1 left) 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2588: done (0 left) 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2589: rollback oid 14 current snap is 268 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 262 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2590: write oid 45 current snap is 268 2026-02-21T22:27:03.166 INFO:tasks.rados.rados.0.trial178.stdout:2590: seq_num 902 ranges {759914=404252,1566405=433305,2504010=655904} 2026-02-21T22:27:03.174 INFO:tasks.rados.rados.0.trial178.stdout:2590: writing trial17818531-45 from 759914 to 1164166 tid 1 2026-02-21T22:27:03.179 INFO:tasks.rados.rados.0.trial178.stdout:2590: writing trial17818531-45 from 1566405 to 1999710 tid 2 2026-02-21T22:27:03.187 INFO:tasks.rados.rados.0.trial178.stdout:2590: writing trial17818531-45 from 2504010 to 3159914 tid 3 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2591: write (excl) oid 24 current snap is 268 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2589: finishing rollback tid 0 to trial17818531-14 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2589: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2589: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1193 (ObjNum 746 snap 209 seq_num 746) dirty exists 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2590: finishing write tid 1 to trial17818531-45 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2590: finishing write tid 2 to trial17818531-45 2026-02-21T22:27:03.188 INFO:tasks.rados.rados.0.trial178.stdout:2591: seq_num 903 ranges {776721=479501,2024887=693316} 2026-02-21T22:27:03.196 INFO:tasks.rados.rados.0.trial178.stdout:2591: writing trial17818531-24 from 776721 to 1256222 tid 1 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2591: writing trial17818531-24 from 2024887 to 2718203 tid 2 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2589: done (2 left) 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2592: read oid 40 snap 265 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2592: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2593: write oid 48 current snap is 268 2026-02-21T22:27:03.206 INFO:tasks.rados.rados.0.trial178.stdout:2593: seq_num 904 ranges {793528=554750,2083369=436974,2958856=234672} 2026-02-21T22:27:03.215 INFO:tasks.rados.rados.0.trial178.stdout:2593: writing trial17818531-48 from 793528 to 1348278 tid 1 2026-02-21T22:27:03.220 INFO:tasks.rados.rados.0.trial178.stdout:2593: writing trial17818531-48 from 2083369 to 2520343 tid 2 2026-02-21T22:27:03.223 INFO:tasks.rados.rados.0.trial178.stdout:2593: writing trial17818531-48 from 2958856 to 3193528 tid 3 2026-02-21T22:27:03.223 INFO:tasks.rados.rados.0.trial178.stdout:2594: write oid 15 current snap is 268 2026-02-21T22:27:03.223 INFO:tasks.rados.rados.0.trial178.stdout:2594: seq_num 905 ranges {410335=746352,1741851=696985,3102632=107703} 2026-02-21T22:27:03.228 INFO:tasks.rados.rados.0.trial178.stdout:2594: writing trial17818531-15 from 410335 to 1156687 tid 1 2026-02-21T22:27:03.233 INFO:tasks.rados.rados.0.trial178.stdout:2594: writing trial17818531-15 from 1741851 to 2438836 tid 2 2026-02-21T22:27:03.236 INFO:tasks.rados.rados.0.trial178.stdout:2594: writing trial17818531-15 from 3102632 to 3210335 tid 3 2026-02-21T22:27:03.236 INFO:tasks.rados.rados.0.trial178.stdout:2595: copy_from oid 37 from oid 10 current snap is 268 2026-02-21T22:27:03.236 INFO:tasks.rados.rados.0.trial178.stdout:2590: finishing write tid 3 to trial17818531-45 2026-02-21T22:27:03.236 INFO:tasks.rados.rados.0.trial178.stdout:2590: finishing write tid 4 to trial17818531-45 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: finishing write tid 5 to trial17818531-45 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: oid 45 updating version 0 to 1290 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: oid 45 updating version 1290 to 1291 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: oid 45 updating version 1291 to 1292 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: oid 45 version 1292 is already newer than 1289 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1292 (ObjNum 902 snap 268 seq_num 902) dirty exists 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: left oid 45 (ObjNum 902 snap 268 seq_num 902) 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2591: finishing write tid 1 to trial17818531-24 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2590: done (5 left) 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2596: delete oid 13 current snap is 268 2026-02-21T22:27:03.237 INFO:tasks.rados.rados.0.trial178.stdout:2591: finishing write tid 2 to trial17818531-24 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: finishing write tid 3 to trial17818531-24 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: finishing write tid 4 to trial17818531-24 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: oid 24 updating version 0 to 1444 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: oid 24 version 1444 is already newer than 1443 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: oid 24 updating version 1444 to 1445 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1445 (ObjNum 903 snap 268 seq_num 903) dirty exists 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2591: left oid 24 (ObjNum 903 snap 268 seq_num 903) 2026-02-21T22:27:03.238 INFO:tasks.rados.rados.0.trial178.stdout:2592: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:27:03.290 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 433 2026-02-21T22:27:03.366 INFO:tasks.rados.rados.0.trial178.stdout:2593: finishing write tid 1 to trial17818531-48 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2593: finishing write tid 2 to trial17818531-48 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2593: finishing write tid 3 to trial17818531-48 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2593: finishing write tid 4 to trial17818531-48 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2591: done (5 left) 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2592: done (4 left) 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2596: done (3 left) 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2597: rollback oid 5 current snap is 268 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 265 2026-02-21T22:27:03.367 INFO:tasks.rados.rados.0.trial178.stdout:2593: finishing write tid 5 to trial17818531-48 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2593: oid 48 updating version 0 to 1293 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2593: oid 48 updating version 1293 to 1295 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2593: oid 48 updating version 1295 to 1296 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2593: oid 48 version 1296 is already newer than 1294 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1296 (ObjNum 904 snap 268 seq_num 904) dirty exists 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2593: left oid 48 (ObjNum 904 snap 268 seq_num 904) 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: finishing write tid 1 to trial17818531-15 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: finishing write tid 2 to trial17818531-15 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: finishing write tid 3 to trial17818531-15 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: finishing write tid 4 to trial17818531-15 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: finishing write tid 5 to trial17818531-15 2026-02-21T22:27:03.368 INFO:tasks.rados.rados.0.trial178.stdout:2594: oid 15 updating version 0 to 1008 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2594: oid 15 version 1008 is already newer than 1007 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2594: oid 15 version 1008 is already newer than 1006 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2594: oid 15 updating version 1008 to 1009 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1009 (ObjNum 905 snap 268 seq_num 905) dirty exists 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2594: left oid 15 (ObjNum 905 snap 268 seq_num 905) 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2595: finishing copy_from racing read to trial17818531-37 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2595: finishing copy_from to trial17818531-37 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1154 (ObjNum 838 snap 239 seq_num 838) dirty exists 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2593: done (3 left) 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2594: done (2 left) 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2595: done (1 left) 2026-02-21T22:27:03.369 INFO:tasks.rados.rados.0.trial178.stdout:2598: write (excl) oid 35 current snap is 268 2026-02-21T22:27:03.370 INFO:tasks.rados.rados.0.trial178.stdout:2598: seq_num 907 ranges {443949=496850,1458815=584301,2641125=602824} 2026-02-21T22:27:03.370 INFO:tasks.rados.rados.0.trial178.stdout:2598: writing trial17818531-35 from 443949 to 940799 tid 1 2026-02-21T22:27:03.374 INFO:tasks.rados.rados.0.trial178.stdout:2598: writing trial17818531-35 from 1458815 to 2043116 tid 2 2026-02-21T22:27:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2598: writing trial17818531-35 from 2641125 to 3243949 tid 3 2026-02-21T22:27:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2599: copy_from oid 45 from oid 16 current snap is 268 2026-02-21T22:27:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2597: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:27:03.379 INFO:tasks.rados.rados.0.trial178.stdout:2597: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2597: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1419 (ObjNum 892 snap 261 seq_num 892) dirty exists 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2598: finishing write tid 1 to trial17818531-35 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2598: finishing write tid 2 to trial17818531-35 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2597: done (2 left) 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2600: write (excl) oid 5 current snap is 268 2026-02-21T22:27:03.380 INFO:tasks.rados.rados.0.trial178.stdout:2600: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2026-02-21T22:27:03.385 INFO:tasks.rados.rados.0.trial178.stdout:2600: writing trial17818531-5 from 460756 to 1032855 tid 1 2026-02-21T22:27:03.391 INFO:tasks.rados.rados.0.trial178.stdout:2600: writing trial17818531-5 from 1517297 to 1961609 tid 2 2026-02-21T22:27:03.400 INFO:tasks.rados.rados.0.trial178.stdout:2600: writing trial17818531-5 from 2668548 to 3141418 tid 3 2026-02-21T22:27:03.400 INFO:tasks.rados.rados.0.trial178.stdout:2601: read oid 40 snap 266 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2601: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2602: read oid 13 snap 266 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2602: expect (ObjNum 883 snap 259 seq_num 883) 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2603: delete oid 10 current snap is 268 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: finishing write tid 3 to trial17818531-35 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: finishing write tid 4 to trial17818531-35 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: finishing write tid 5 to trial17818531-35 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: oid 35 updating version 0 to 1241 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: oid 35 updating version 1241 to 1244 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: oid 35 version 1244 is already newer than 1242 2026-02-21T22:27:03.401 INFO:tasks.rados.rados.0.trial178.stdout:2598: oid 35 version 1244 is already newer than 1243 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1244 (ObjNum 907 snap 268 seq_num 907) dirty exists 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2598: left oid 35 (ObjNum 907 snap 268 seq_num 907) 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2599: finishing copy_from racing read to trial17818531-45 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2600: finishing write tid 1 to trial17818531-5 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2599: finishing copy_from to trial17818531-45 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 836 (ObjNum 890 snap 261 seq_num 890) dirty exists 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2600: finishing write tid 2 to trial17818531-5 2026-02-21T22:27:03.402 INFO:tasks.rados.rados.0.trial178.stdout:2601: expect (ObjNum 854 snap 243 seq_num 854) 2026-02-21T22:27:03.524 INFO:tasks.rados.rados.0.trial178.stdout:2600: finishing write tid 3 to trial17818531-5 2026-02-21T22:27:03.524 INFO:tasks.rados.rados.0.trial178.stdout:2600: finishing write tid 4 to trial17818531-5 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: finishing write tid 5 to trial17818531-5 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: oid 5 updating version 0 to 1421 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: oid 5 updating version 1421 to 1422 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: oid 5 version 1422 is already newer than 1420 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: oid 5 updating version 1422 to 1423 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1423 (ObjNum 908 snap 268 seq_num 908) dirty exists 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2600: left oid 5 (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:03.525 INFO:tasks.rados.rados.0.trial178.stdout:2602: expect (ObjNum 883 snap 259 seq_num 883) 2026-02-21T22:27:03.554 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2839406 2026-02-21T22:27:03.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2598: done (5 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2599: done (4 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2600: done (3 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2601: done (2 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2602: done (1 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2603: done (0 left) 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2604: copy_from oid 37 from oid 30 current snap is 268 2026-02-21T22:27:03.575 INFO:tasks.rados.rados.0.trial178.stdout:2605: snap_create 2026-02-21T22:27:03.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:03.582 INFO:tasks.rados.rados.0.trial178.stdout:2604: finishing copy_from racing read to trial17818531-37 2026-02-21T22:27:03.586 INFO:tasks.rados.rados.0.trial178.stdout:2604: finishing copy_from to trial17818531-37 2026-02-21T22:27:03.586 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:27:04.210 INFO:tasks.rados.rados.0.trial178.stdout:2604: done (1 left) 2026-02-21T22:27:04.210 INFO:tasks.rados.rados.0.trial178.stdout:2605: done (0 left) 2026-02-21T22:27:04.210 INFO:tasks.rados.rados.0.trial178.stdout:2606: read oid 14 snap -1 2026-02-21T22:27:04.210 INFO:tasks.rados.rados.0.trial178.stdout:2606: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:27:04.210 INFO:tasks.rados.rados.0.trial178.stdout:2607: snap_create 2026-02-21T22:27:04.216 INFO:tasks.rados.rados.0.trial178.stdout:2606: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:27:04.223 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2361430 2026-02-21T22:27:04.227 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 28575 2026-02-21T22:27:05.282 INFO:tasks.rados.rados.0.trial178.stdout:2606: done (1 left) 2026-02-21T22:27:05.282 INFO:tasks.rados.rados.0.trial178.stdout:2607: done (0 left) 2026-02-21T22:27:05.283 INFO:tasks.rados.rados.0.trial178.stdout:2608: delete oid 4 current snap is 270 2026-02-21T22:27:05.285 INFO:tasks.rados.rados.0.trial178.stdout:2608: done (0 left) 2026-02-21T22:27:05.285 INFO:tasks.rados.rados.0.trial178.stdout:2609: write (excl) oid 8 current snap is 270 2026-02-21T22:27:05.285 INFO:tasks.rados.rados.0.trial178.stdout:2609: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2026-02-21T22:27:05.293 INFO:tasks.rados.rados.0.trial178.stdout:2609: writing trial17818531-8 from 511177 to 1309023 tid 1 2026-02-21T22:27:05.302 INFO:tasks.rados.rados.0.trial178.stdout:2609: writing trial17818531-8 from 1976390 to 2568029 tid 2 2026-02-21T22:27:05.307 INFO:tasks.rados.rados.0.trial178.stdout:2609: writing trial17818531-8 from 3034464 to 3311177 tid 3 2026-02-21T22:27:05.307 INFO:tasks.rados.rados.0.trial178.stdout:2610: rollback oid 23 current snap is 270 2026-02-21T22:27:05.307 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 258 2026-02-21T22:27:05.307 INFO:tasks.rados.rados.0.trial178.stdout:2609: finishing write tid 1 to trial17818531-8 2026-02-21T22:27:05.308 INFO:tasks.rados.rados.0.trial178.stdout:2609: finishing write tid 2 to trial17818531-8 2026-02-21T22:27:05.308 INFO:tasks.rados.rados.0.trial178.stdout:2611: write (excl) oid 18 current snap is 270 2026-02-21T22:27:05.308 INFO:tasks.rados.rados.0.trial178.stdout:2611: seq_num 912 ranges {527984=473095,1634872=451650,2661887=634311} 2026-02-21T22:27:05.314 INFO:tasks.rados.rados.0.trial178.stdout:2611: writing trial17818531-18 from 527984 to 1001079 tid 1 2026-02-21T22:27:05.321 INFO:tasks.rados.rados.0.trial178.stdout:2611: writing trial17818531-18 from 1634872 to 2086522 tid 2 2026-02-21T22:27:05.330 INFO:tasks.rados.rados.0.trial178.stdout:2611: writing trial17818531-18 from 2661887 to 3296198 tid 3 2026-02-21T22:27:05.330 INFO:tasks.rados.rados.0.trial178.stdout:2612: copy_from oid 21 from oid 41 current snap is 270 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: finishing write tid 3 to trial17818531-8 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2610: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2610: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: finishing write tid 4 to trial17818531-8 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2610: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1377 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: finishing write tid 5 to trial17818531-8 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: oid 8 updating version 0 to 1448 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: oid 8 updating version 1448 to 1450 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: oid 8 version 1450 is already newer than 1449 2026-02-21T22:27:05.331 INFO:tasks.rados.rados.0.trial178.stdout:2609: oid 8 version 1450 is already newer than 1447 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1450 (ObjNum 911 snap 270 seq_num 911) dirty exists 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2609: left oid 8 (ObjNum 911 snap 270 seq_num 911) 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2611: finishing write tid 1 to trial17818531-18 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2609: done (3 left) 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2610: done (2 left) 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2613: delete oid 50 current snap is 270 2026-02-21T22:27:05.332 INFO:tasks.rados.rados.0.trial178.stdout:2611: finishing write tid 2 to trial17818531-18 2026-02-21T22:27:05.333 INFO:tasks.rados.rados.0.trial178.stdout:2611: finishing write tid 3 to trial17818531-18 2026-02-21T22:27:05.333 INFO:tasks.rados.rados.0.trial178.stdout:2611: finishing write tid 4 to trial17818531-18 2026-02-21T22:27:05.333 INFO:tasks.rados.rados.0.trial178.stdout:2611: finishing write tid 5 to trial17818531-18 2026-02-21T22:27:05.333 INFO:tasks.rados.rados.0.trial178.stdout:2611: oid 18 updating version 0 to 1426 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2611: oid 18 version 1426 is already newer than 1425 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2611: oid 18 version 1426 is already newer than 1424 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2611: oid 18 updating version 1426 to 1427 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1427 (ObjNum 912 snap 270 seq_num 912) dirty exists 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2611: left oid 18 (ObjNum 912 snap 270 seq_num 912) 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2611: done (2 left) 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2613: done (1 left) 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2614: write (excl) oid 2 current snap is 270 2026-02-21T22:27:05.334 INFO:tasks.rados.rados.0.trial178.stdout:2614: seq_num 914 ranges {561598=739946,1751836=455319,2716733=644865} 2026-02-21T22:27:05.341 INFO:tasks.rados.rados.0.trial178.stdout:2614: writing trial17818531-2 from 561598 to 1301544 tid 1 2026-02-21T22:27:05.347 INFO:tasks.rados.rados.0.trial178.stdout:2614: writing trial17818531-2 from 1751836 to 2207155 tid 2 2026-02-21T22:27:05.355 INFO:tasks.rados.rados.0.trial178.stdout:2614: writing trial17818531-2 from 2716733 to 3361598 tid 3 2026-02-21T22:27:05.355 INFO:tasks.rados.rados.0.trial178.stdout:2615: read oid 20 snap -1 2026-02-21T22:27:05.355 INFO:tasks.rados.rados.0.trial178.stdout:2615: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2616: copy_from oid 35 from oid 38 current snap is 270 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2612: finishing copy_from racing read to trial17818531-21 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2612: finishing copy_from to trial17818531-21 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2614: finishing write tid 1 to trial17818531-2 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2614: finishing write tid 2 to trial17818531-2 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2612: done (3 left) 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2617: write oid 13 current snap is 270 2026-02-21T22:27:05.356 INFO:tasks.rados.rados.0.trial178.stdout:2617: seq_num 915 ranges {578405=415195,1410318=598977,2744156=584480} 2026-02-21T22:27:05.358 INFO:tasks.rados.rados.0.trial178.stdout:2617: writing trial17818531-13 from 578405 to 993600 tid 1 2026-02-21T22:27:05.362 INFO:tasks.rados.rados.0.trial178.stdout:2617: writing trial17818531-13 from 1410318 to 2009295 tid 2 2026-02-21T22:27:05.367 INFO:tasks.rados.rados.0.trial178.stdout:2617: writing trial17818531-13 from 2744156 to 3328636 tid 3 2026-02-21T22:27:05.367 INFO:tasks.rados.rados.0.trial178.stdout:2618: delete oid 6 current snap is 270 2026-02-21T22:27:05.367 INFO:tasks.rados.rados.0.trial178.stdout:2615: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2614: finishing write tid 3 to trial17818531-2 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2614: finishing write tid 4 to trial17818531-2 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2617: finishing write tid 1 to trial17818531-13 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2614: finishing write tid 5 to trial17818531-2 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2614: oid 2 updating version 0 to 1171 2026-02-21T22:27:05.495 INFO:tasks.rados.rados.0.trial178.stdout:2614: oid 2 version 1171 is already newer than 1168 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2614: oid 2 version 1171 is already newer than 1170 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2614: oid 2 version 1171 is already newer than 1169 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1171 (ObjNum 914 snap 270 seq_num 914) dirty exists 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2614: left oid 2 (ObjNum 914 snap 270 seq_num 914) 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2617: finishing write tid 2 to trial17818531-13 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2616: finishing copy_from racing read to trial17818531-35 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2616: finishing copy_from to trial17818531-35 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 856 (ObjNum 899 snap 265 seq_num 899) dirty exists 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2617: finishing write tid 3 to trial17818531-13 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2617: finishing write tid 4 to trial17818531-13 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2617: finishing write tid 5 to trial17818531-13 2026-02-21T22:27:05.496 INFO:tasks.rados.rados.0.trial178.stdout:2617: oid 13 updating version 0 to 1245 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2617: oid 13 updating version 1245 to 1246 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2617: oid 13 updating version 1246 to 1250 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2617: oid 13 version 1250 is already newer than 1249 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1250 (ObjNum 915 snap 270 seq_num 915) dirty exists 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2617: left oid 13 (ObjNum 915 snap 270 seq_num 915) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2614: done (4 left) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2615: done (3 left) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2616: done (2 left) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2617: done (1 left) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2618: done (0 left) 2026-02-21T22:27:05.497 INFO:tasks.rados.rados.0.trial178.stdout:2619: snap_remove snap 263 2026-02-21T22:27:06.218 INFO:tasks.rados.rados.0.trial178.stdout:2619: done (0 left) 2026-02-21T22:27:06.219 INFO:tasks.rados.rados.0.trial178.stdout:2620: snap_remove snap 261 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:2620: done (0 left) 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:2621: copy_from oid 16 from oid 10 current snap is 270 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:2622: rollback oid 41 current snap is 270 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 264 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:2623: write (excl) oid 21 current snap is 270 2026-02-21T22:27:07.167 INFO:tasks.rados.rados.0.trial178.stdout:2623: seq_num 917 ranges {612019=565693,1810929=602646,3082649=329370} 2026-02-21T22:27:07.174 INFO:tasks.rados.rados.0.trial178.stdout:2623: writing trial17818531-21 from 612019 to 1177712 tid 1 2026-02-21T22:27:07.182 INFO:tasks.rados.rados.0.trial178.stdout:2623: writing trial17818531-21 from 1810929 to 2413575 tid 2 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2623: writing trial17818531-21 from 3082649 to 3412019 tid 3 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2624: copy_from oid 29 from oid 22 current snap is 270 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2621: finishing copy_from to trial17818531-16 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2621: got expected ENOENT (src dne) 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2621: finishing copy_from racing read to trial17818531-16 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2622: finishing rollback tid 0 to trial17818531-41 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2622: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:2622: finishing rollback tid 2 to trial17818531-41 2026-02-21T22:27:07.190 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1015 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:27:07.191 INFO:tasks.rados.rados.0.trial178.stdout:2623: finishing write tid 1 to trial17818531-21 2026-02-21T22:27:07.191 INFO:tasks.rados.rados.0.trial178.stdout:2623: finishing write tid 2 to trial17818531-21 2026-02-21T22:27:07.191 INFO:tasks.rados.rados.0.trial178.stdout:2621: done (3 left) 2026-02-21T22:27:07.191 INFO:tasks.rados.rados.0.trial178.stdout:2622: done (2 left) 2026-02-21T22:27:07.191 INFO:tasks.rados.rados.0.trial178.stdout:2625: snap_remove snap 264 2026-02-21T22:27:08.171 INFO:tasks.rados.rados.0.trial178.stdout:2625: done (2 left) 2026-02-21T22:27:08.171 INFO:tasks.rados.rados.0.trial178.stdout:2626: read oid 46 snap -1 2026-02-21T22:27:08.171 INFO:tasks.rados.rados.0.trial178.stdout:2626: expect deleted 2026-02-21T22:27:08.171 INFO:tasks.rados.rados.0.trial178.stdout:2627: copy_from oid 35 from oid 48 current snap is 270 2026-02-21T22:27:08.171 INFO:tasks.rados.rados.0.trial178.stdout:2623: finishing write tid 3 to trial17818531-21 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: finishing write tid 4 to trial17818531-21 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: finishing write tid 5 to trial17818531-21 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: oid 21 updating version 0 to 1459 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: oid 21 updating version 1459 to 1468 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: oid 21 version 1468 is already newer than 1466 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: oid 21 version 1468 is already newer than 1467 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1468 (ObjNum 917 snap 270 seq_num 917) dirty exists 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: left oid 21 (ObjNum 917 snap 270 seq_num 917) 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2624: finishing copy_from racing read to trial17818531-29 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2624: finishing copy_from to trial17818531-29 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1162 (ObjNum 870 snap 253 seq_num 870) dirty exists 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2623: done (3 left) 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2624: done (2 left) 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2628: write oid 31 current snap is 270 2026-02-21T22:27:08.172 INFO:tasks.rados.rados.0.trial178.stdout:2628: seq_num 918 ranges {628826=640942,1869411=462657,2826425=534649} 2026-02-21T22:27:08.179 INFO:tasks.rados.rados.0.trial178.stdout:2628: writing trial17818531-31 from 628826 to 1269768 tid 1 2026-02-21T22:27:08.186 INFO:tasks.rados.rados.0.trial178.stdout:2628: writing trial17818531-31 from 1869411 to 2332068 tid 2 2026-02-21T22:27:08.193 INFO:tasks.rados.rados.0.trial178.stdout:2628: writing trial17818531-31 from 2826425 to 3361074 tid 3 2026-02-21T22:27:08.193 INFO:tasks.rados.rados.0.trial178.stdout:2629: write oid 1 current snap is 270 2026-02-21T22:27:08.193 INFO:tasks.rados.rados.0.trial178.stdout:2629: seq_num 919 ranges {645633=716191,1927893=606315,3137495=308138} 2026-02-21T22:27:08.202 INFO:tasks.rados.rados.0.trial178.stdout:2629: writing trial17818531-1 from 645633 to 1361824 tid 1 2026-02-21T22:27:08.210 INFO:tasks.rados.rados.0.trial178.stdout:2629: writing trial17818531-1 from 1927893 to 2534208 tid 2 2026-02-21T22:27:08.216 INFO:tasks.rados.rados.0.trial178.stdout:2629: writing trial17818531-1 from 3137495 to 3445633 tid 3 2026-02-21T22:27:08.217 INFO:tasks.rados.rados.0.trial178.stdout:2630: write oid 15 current snap is 270 2026-02-21T22:27:08.217 INFO:tasks.rados.rados.0.trial178.stdout:2630: seq_num 920 ranges {662440=507793,1702728=749973,2881271=557193} 2026-02-21T22:27:08.221 INFO:tasks.rados.rados.0.trial178.stdout:2630: writing trial17818531-15 from 662440 to 1170233 tid 1 2026-02-21T22:27:08.225 INFO:tasks.rados.rados.0.trial178.stdout:2630: writing trial17818531-15 from 1702728 to 2452701 tid 2 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2630: writing trial17818531-15 from 2881271 to 3438464 tid 3 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2631: read oid 44 snap 258 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2631: expect deleted 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2632: read oid 20 snap -1 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2632: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2633: copy_from oid 37 from oid 30 current snap is 270 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2627: finishing copy_from racing read to trial17818531-35 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: finishing write tid 1 to trial17818531-31 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2627: finishing copy_from to trial17818531-35 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1296 (ObjNum 904 snap 268 seq_num 904) dirty exists 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: finishing write tid 2 to trial17818531-31 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2626: done (7 left) 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2627: done (6 left) 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2634: snap_create 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: finishing write tid 3 to trial17818531-31 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: finishing write tid 4 to trial17818531-31 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: finishing write tid 5 to trial17818531-31 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: oid 31 updating version 0 to 1170 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: oid 31 version 1170 is already newer than 1169 2026-02-21T22:27:08.230 INFO:tasks.rados.rados.0.trial178.stdout:2628: oid 31 version 1170 is already newer than 1168 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2628: oid 31 updating version 1170 to 1171 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1171 (ObjNum 918 snap 270 seq_num 918) dirty exists 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2628: left oid 31 (ObjNum 918 snap 270 seq_num 918) 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: finishing write tid 1 to trial17818531-1 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: finishing write tid 2 to trial17818531-1 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: finishing write tid 3 to trial17818531-1 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: finishing write tid 4 to trial17818531-1 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: finishing write tid 5 to trial17818531-1 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: oid 1 updating version 0 to 1297 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: oid 1 updating version 1297 to 1299 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: oid 1 updating version 1299 to 1300 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: oid 1 version 1300 is already newer than 1298 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1300 (ObjNum 919 snap 270 seq_num 919) dirty exists 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2629: left oid 1 (ObjNum 919 snap 270 seq_num 919) 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2630: finishing write tid 1 to trial17818531-15 2026-02-21T22:27:08.231 INFO:tasks.rados.rados.0.trial178.stdout:2630: finishing write tid 2 to trial17818531-15 2026-02-21T22:27:08.234 INFO:tasks.rados.rados.0.trial178.stdout:2630: finishing write tid 3 to trial17818531-15 2026-02-21T22:27:08.234 INFO:tasks.rados.rados.0.trial178.stdout:2630: finishing write tid 4 to trial17818531-15 2026-02-21T22:27:08.234 INFO:tasks.rados.rados.0.trial178.stdout:2630: finishing write tid 5 to trial17818531-15 2026-02-21T22:27:08.234 INFO:tasks.rados.rados.0.trial178.stdout:2630: oid 15 updating version 0 to 1023 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:2630: oid 15 updating version 1023 to 1024 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:2630: oid 15 updating version 1024 to 1026 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:2630: oid 15 version 1026 is already newer than 1025 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1026 (ObjNum 920 snap 270 seq_num 920) dirty exists 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:2630: left oid 15 (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:08.235 INFO:tasks.rados.rados.0.trial178.stdout:2632: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:08.370 INFO:tasks.rados.rados.0.trial178.stdout:2633: finishing copy_from racing read to trial17818531-37 2026-02-21T22:27:08.370 INFO:tasks.rados.rados.0.trial178.stdout:2633: finishing copy_from to trial17818531-37 2026-02-21T22:27:08.370 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:27:08.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:08.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:10.163 INFO:tasks.rados.rados.0.trial178.stdout:2628: done (6 left) 2026-02-21T22:27:10.163 INFO:tasks.rados.rados.0.trial178.stdout:2629: done (5 left) 2026-02-21T22:27:10.163 INFO:tasks.rados.rados.0.trial178.stdout:2630: done (4 left) 2026-02-21T22:27:10.163 INFO:tasks.rados.rados.0.trial178.stdout:2631: done (3 left) 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2632: done (2 left) 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2633: done (1 left) 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2634: done (0 left) 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2635: read oid 27 snap -1 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2635: expect (ObjNum 876 snap 254 seq_num 876) 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2636: write oid 12 current snap is 271 2026-02-21T22:27:10.164 INFO:tasks.rados.rados.0.trial178.stdout:2636: seq_num 921 ranges {679247=583042,2044857=609984,3192341=286906} 2026-02-21T22:27:10.167 INFO:tasks.rados.rados.0.trial178.stdout:2636: writing trial17818531-12 from 679247 to 1262289 tid 1 2026-02-21T22:27:10.173 INFO:tasks.rados.rados.0.trial178.stdout:2636: writing trial17818531-12 from 2044857 to 2654841 tid 2 2026-02-21T22:27:10.176 INFO:tasks.rados.rados.0.trial178.stdout:2636: writing trial17818531-12 from 3192341 to 3479247 tid 3 2026-02-21T22:27:10.176 INFO:tasks.rados.rados.0.trial178.stdout:2637: delete oid 42 current snap is 271 2026-02-21T22:27:10.176 INFO:tasks.rados.rados.0.trial178.stdout:2635: expect (ObjNum 876 snap 254 seq_num 876) 2026-02-21T22:27:10.240 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:10.273 INFO:tasks.rados.rados.0.trial178.stdout:2636: finishing write tid 1 to trial17818531-12 2026-02-21T22:27:10.273 INFO:tasks.rados.rados.0.trial178.stdout:2636: finishing write tid 2 to trial17818531-12 2026-02-21T22:27:10.273 INFO:tasks.rados.rados.0.trial178.stdout:2636: finishing write tid 3 to trial17818531-12 2026-02-21T22:27:10.273 INFO:tasks.rados.rados.0.trial178.stdout:2636: finishing write tid 4 to trial17818531-12 2026-02-21T22:27:10.273 INFO:tasks.rados.rados.0.trial178.stdout:2636: finishing write tid 5 to trial17818531-12 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: oid 12 updating version 0 to 842 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: oid 12 updating version 842 to 843 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: oid 12 updating version 843 to 844 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: oid 12 updating version 844 to 845 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 845 (ObjNum 921 snap 271 seq_num 921) dirty exists 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: left oid 12 (ObjNum 921 snap 271 seq_num 921) 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2635: done (2 left) 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2636: done (1 left) 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2637: done (0 left) 2026-02-21T22:27:10.274 INFO:tasks.rados.rados.0.trial178.stdout:2638: snap_remove snap 267 2026-02-21T22:27:11.166 INFO:tasks.rados.rados.0.trial178.stdout:2638: done (0 left) 2026-02-21T22:27:11.166 INFO:tasks.rados.rados.0.trial178.stdout:2639: delete oid 49 current snap is 271 2026-02-21T22:27:11.170 INFO:tasks.rados.rados.0.trial178.stdout:2639: done (0 left) 2026-02-21T22:27:11.170 INFO:tasks.rados.rados.0.trial178.stdout:2640: snap_remove snap 258 2026-02-21T22:27:12.173 INFO:tasks.rados.rados.0.trial178.stdout:2640: done (0 left) 2026-02-21T22:27:12.174 INFO:tasks.rados.rados.0.trial178.stdout:2641: rollback oid 34 current snap is 271 2026-02-21T22:27:12.174 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 34 to 266 2026-02-21T22:27:12.174 INFO:tasks.rados.rados.0.trial178.stdout:2642: delete oid 6 current snap is 271 2026-02-21T22:27:12.177 INFO:tasks.rados.rados.0.trial178.stdout:2642: done (1 left) 2026-02-21T22:27:12.177 INFO:tasks.rados.rados.0.trial178.stdout:2643: rollback oid 13 current snap is 271 2026-02-21T22:27:12.177 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 262 2026-02-21T22:27:12.177 INFO:tasks.rados.rados.0.trial178.stdout:2644: write (excl) oid 36 current snap is 271 2026-02-21T22:27:12.177 INFO:tasks.rados.rados.0.trial178.stdout:2644: seq_num 925 ranges {746475=484038,1762432=617322,2785680=529906} 2026-02-21T22:27:12.185 INFO:tasks.rados.rados.0.trial178.stdout:2644: writing trial17818531-36 from 746475 to 1230513 tid 1 2026-02-21T22:27:12.192 INFO:tasks.rados.rados.0.trial178.stdout:2644: writing trial17818531-36 from 1762432 to 2379754 tid 2 2026-02-21T22:27:12.199 INFO:tasks.rados.rados.0.trial178.stdout:2644: writing trial17818531-36 from 2785680 to 3315586 tid 3 2026-02-21T22:27:12.199 INFO:tasks.rados.rados.0.trial178.stdout:2645: delete oid 18 current snap is 271 2026-02-21T22:27:12.199 INFO:tasks.rados.rados.0.trial178.stdout:2641: finishing rollback tid 1 to trial17818531-34 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 0 (ObjNum 161 snap 0 seq_num 102033904) dirty dne 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:2643: finishing rollback tid 0 to trial17818531-13 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:2643: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:2643: finishing rollback tid 2 to trial17818531-13 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1299 (ObjNum 883 snap 259 seq_num 883) dirty exists 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:2644: finishing write tid 1 to trial17818531-36 2026-02-21T22:27:12.200 INFO:tasks.rados.rados.0.trial178.stdout:2644: finishing write tid 2 to trial17818531-36 2026-02-21T22:27:12.201 INFO:tasks.rados.rados.0.trial178.stdout:2644: finishing write tid 3 to trial17818531-36 2026-02-21T22:27:12.201 INFO:tasks.rados.rados.0.trial178.stdout:2644: finishing write tid 4 to trial17818531-36 2026-02-21T22:27:12.201 INFO:tasks.rados.rados.0.trial178.stdout:2644: finishing write tid 5 to trial17818531-36 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:2644: oid 36 updating version 0 to 1086 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:2644: oid 36 updating version 1086 to 1087 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:2644: oid 36 updating version 1087 to 1089 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:2644: oid 36 version 1089 is already newer than 1088 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1089 (ObjNum 925 snap 271 seq_num 925) dirty exists 2026-02-21T22:27:12.202 INFO:tasks.rados.rados.0.trial178.stdout:2644: left oid 36 (ObjNum 925 snap 271 seq_num 925) 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2641: done (3 left) 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2643: done (2 left) 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2644: done (1 left) 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2645: done (0 left) 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2646: rollback oid 32 current snap is 271 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 269 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2647: write (excl) oid 34 current snap is 271 2026-02-21T22:27:12.212 INFO:tasks.rados.rados.0.trial178.stdout:2647: seq_num 927 ranges {780089=634536,1879396=620991,3240526=339563} 2026-02-21T22:27:12.221 INFO:tasks.rados.rados.0.trial178.stdout:2647: writing trial17818531-34 from 780089 to 1414625 tid 1 2026-02-21T22:27:12.228 INFO:tasks.rados.rados.0.trial178.stdout:2647: writing trial17818531-34 from 1879396 to 2500387 tid 2 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:2647: writing trial17818531-34 from 3240526 to 3580089 tid 3 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:2648: write (excl) oid 3 current snap is 271 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:2646: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:2646: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:2646: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:27:12.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1302 (ObjNum 781 snap 219 seq_num 781) dirty exists 2026-02-21T22:27:12.237 INFO:tasks.rados.rados.0.trial178.stdout:2647: finishing write tid 1 to trial17818531-34 2026-02-21T22:27:12.237 INFO:tasks.rados.rados.0.trial178.stdout:2647: finishing write tid 2 to trial17818531-34 2026-02-21T22:27:12.237 INFO:tasks.rados.rados.0.trial178.stdout:2648: seq_num 928 ranges {796896=426138,1654231=764649,2984302=480075} 2026-02-21T22:27:12.244 INFO:tasks.rados.rados.0.trial178.stdout:2648: writing trial17818531-3 from 796896 to 1223034 tid 1 2026-02-21T22:27:12.251 INFO:tasks.rados.rados.0.trial178.stdout:2648: writing trial17818531-3 from 1654231 to 2418880 tid 2 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2648: writing trial17818531-3 from 2984302 to 3464377 tid 3 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2646: done (2 left) 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2649: read oid 9 snap 266 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2649: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2650: rollback oid 2 current snap is 271 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 265 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2647: finishing write tid 3 to trial17818531-34 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2647: finishing write tid 4 to trial17818531-34 2026-02-21T22:27:12.256 INFO:tasks.rados.rados.0.trial178.stdout:2647: finishing write tid 5 to trial17818531-34 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: oid 34 updating version 0 to 1166 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: oid 34 version 1166 is already newer than 1164 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: oid 34 version 1166 is already newer than 1163 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: oid 34 version 1166 is already newer than 1165 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1166 (ObjNum 927 snap 271 seq_num 927) dirty exists 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: left oid 34 (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2648: finishing write tid 1 to trial17818531-3 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2648: finishing write tid 2 to trial17818531-3 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2647: done (3 left) 2026-02-21T22:27:12.257 INFO:tasks.rados.rados.0.trial178.stdout:2651: snap_create 2026-02-21T22:27:12.259 INFO:tasks.rados.rados.0.trial178.stdout:2648: finishing write tid 3 to trial17818531-3 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: finishing write tid 4 to trial17818531-3 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2650: finishing rollback tid 0 to trial17818531-2 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: finishing write tid 5 to trial17818531-3 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: oid 3 updating version 0 to 1303 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: oid 3 updating version 1303 to 1304 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: oid 3 updating version 1304 to 1305 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:2648: oid 3 updating version 1305 to 1306 2026-02-21T22:27:12.260 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1306 (ObjNum 928 snap 271 seq_num 928) dirty exists 2026-02-21T22:27:12.261 INFO:tasks.rados.rados.0.trial178.stdout:2648: left oid 3 (ObjNum 928 snap 271 seq_num 928) 2026-02-21T22:27:12.261 INFO:tasks.rados.rados.0.trial178.stdout:2650: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:27:12.261 INFO:tasks.rados.rados.0.trial178.stdout:2650: finishing rollback tid 2 to trial17818531-2 2026-02-21T22:27:12.261 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1175 (ObjNum 900 snap 265 seq_num 900) dirty exists 2026-02-21T22:27:12.263 INFO:tasks.rados.rados.0.trial178.stdout:2649: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:12.263 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:12.336 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:13.176 INFO:tasks.rados.rados.0.trial178.stdout:2648: done (3 left) 2026-02-21T22:27:13.177 INFO:tasks.rados.rados.0.trial178.stdout:2649: done (2 left) 2026-02-21T22:27:13.177 INFO:tasks.rados.rados.0.trial178.stdout:2650: done (1 left) 2026-02-21T22:27:13.177 INFO:tasks.rados.rados.0.trial178.stdout:2651: done (0 left) 2026-02-21T22:27:13.177 INFO:tasks.rados.rados.0.trial178.stdout:2652: snap_create 2026-02-21T22:27:13.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:13.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:15.057 INFO:tasks.rados.rados.0.trial178.stdout:2652: done (0 left) 2026-02-21T22:27:15.057 INFO:tasks.rados.rados.0.trial178.stdout:2653: snap_remove snap 268 2026-02-21T22:27:15.184 INFO:tasks.rados.rados.0.trial178.stdout:2653: done (0 left) 2026-02-21T22:27:15.185 INFO:tasks.rados.rados.0.trial178.stdout:2654: snap_create 2026-02-21T22:27:16.190 INFO:tasks.rados.rados.0.trial178.stdout:2654: done (0 left) 2026-02-21T22:27:16.190 INFO:tasks.rados.rados.0.trial178.stdout:2655: snap_remove snap 273 2026-02-21T22:27:17.172 INFO:tasks.rados.rados.0.trial178.stdout:2655: done (0 left) 2026-02-21T22:27:17.172 INFO:tasks.rados.rados.0.trial178.stdout:2656: write oid 29 current snap is 274 2026-02-21T22:27:17.172 INFO:tasks.rados.rados.0.trial178.stdout:2656: seq_num 929 ranges {413703=501387,1596360=624660,2895372=691347} 2026-02-21T22:27:17.178 INFO:tasks.rados.rados.0.trial178.stdout:2656: writing trial17818531-29 from 413703 to 915090 tid 1 2026-02-21T22:27:17.186 INFO:tasks.rados.rados.0.trial178.stdout:2656: writing trial17818531-29 from 1596360 to 2221020 tid 2 2026-02-21T22:27:17.196 INFO:tasks.rados.rados.0.trial178.stdout:2656: writing trial17818531-29 from 2895372 to 3586719 tid 3 2026-02-21T22:27:17.196 INFO:tasks.rados.rados.0.trial178.stdout:2657: copy_from oid 4 from oid 24 current snap is 274 2026-02-21T22:27:17.196 INFO:tasks.rados.rados.0.trial178.stdout:2658: write oid 13 current snap is 274 2026-02-21T22:27:17.196 INFO:tasks.rados.rados.0.trial178.stdout:2658: seq_num 930 ranges {430510=576636,1654842=768318,2922795=502619} 2026-02-21T22:27:17.202 INFO:tasks.rados.rados.0.trial178.stdout:2658: writing trial17818531-13 from 430510 to 1007146 tid 1 2026-02-21T22:27:17.212 INFO:tasks.rados.rados.0.trial178.stdout:2658: writing trial17818531-13 from 1654842 to 2423160 tid 2 2026-02-21T22:27:17.218 INFO:tasks.rados.rados.0.trial178.stdout:2658: writing trial17818531-13 from 2922795 to 3425414 tid 3 2026-02-21T22:27:17.218 INFO:tasks.rados.rados.0.trial178.stdout:2659: snap_remove snap 269 2026-02-21T22:27:18.195 INFO:tasks.rados.rados.0.trial178.stdout:2659: done (3 left) 2026-02-21T22:27:18.195 INFO:tasks.rados.rados.0.trial178.stdout:2660: read oid 36 snap -1 2026-02-21T22:27:18.196 INFO:tasks.rados.rados.0.trial178.stdout:2660: expect (ObjNum 925 snap 271 seq_num 925) 2026-02-21T22:27:18.196 INFO:tasks.rados.rados.0.trial178.stdout:2661: write oid 27 current snap is 274 2026-02-21T22:27:18.196 INFO:tasks.rados.rados.0.trial178.stdout:2661: seq_num 931 ranges {447317=651885,1713324=628329,3066571=580746} 2026-02-21T22:27:18.199 INFO:tasks.rados.rados.0.trial178.stdout:2661: writing trial17818531-27 from 447317 to 1099202 tid 1 2026-02-21T22:27:18.204 INFO:tasks.rados.rados.0.trial178.stdout:2661: writing trial17818531-27 from 1713324 to 2341653 tid 2 2026-02-21T22:27:18.209 INFO:tasks.rados.rados.0.trial178.stdout:2661: writing trial17818531-27 from 3066571 to 3647317 tid 3 2026-02-21T22:27:18.209 INFO:tasks.rados.rados.0.trial178.stdout:2662: write oid 37 current snap is 274 2026-02-21T22:27:18.210 INFO:tasks.rados.rados.0.trial178.stdout:2657: finishing copy_from racing read to trial17818531-4 2026-02-21T22:27:18.210 INFO:tasks.rados.rados.0.trial178.stdout:2662: seq_num 932 ranges {464124=727134,1771806=771987,2977641=641516} 2026-02-21T22:27:18.216 INFO:tasks.rados.rados.0.trial178.stdout:2662: writing trial17818531-37 from 464124 to 1191258 tid 1 2026-02-21T22:27:18.225 INFO:tasks.rados.rados.0.trial178.stdout:2662: writing trial17818531-37 from 1771806 to 2543793 tid 2 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2662: writing trial17818531-37 from 2977641 to 3619157 tid 3 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2663: snap_create 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2657: finishing copy_from to trial17818531-4 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1445 (ObjNum 903 snap 268 seq_num 903) dirty exists 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: finishing write tid 1 to trial17818531-13 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: finishing write tid 2 to trial17818531-13 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: finishing write tid 3 to trial17818531-13 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: finishing write tid 4 to trial17818531-13 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: finishing write tid 5 to trial17818531-13 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: oid 13 updating version 0 to 1310 2026-02-21T22:27:18.234 INFO:tasks.rados.rados.0.trial178.stdout:2658: oid 13 version 1310 is already newer than 1309 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2658: oid 13 version 1310 is already newer than 1307 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2658: oid 13 version 1310 is already newer than 1308 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1310 (ObjNum 930 snap 274 seq_num 930) dirty exists 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2658: left oid 13 (ObjNum 930 snap 274 seq_num 930) 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: finishing write tid 1 to trial17818531-29 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: finishing write tid 2 to trial17818531-29 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: finishing write tid 3 to trial17818531-29 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: finishing write tid 4 to trial17818531-29 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: finishing write tid 5 to trial17818531-29 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: oid 29 updating version 0 to 1170 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: oid 29 version 1170 is already newer than 1168 2026-02-21T22:27:18.235 INFO:tasks.rados.rados.0.trial178.stdout:2656: oid 29 version 1170 is already newer than 1169 2026-02-21T22:27:18.236 INFO:tasks.rados.rados.0.trial178.stdout:2656: oid 29 version 1170 is already newer than 1167 2026-02-21T22:27:18.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1170 (ObjNum 929 snap 274 seq_num 929) dirty exists 2026-02-21T22:27:18.236 INFO:tasks.rados.rados.0.trial178.stdout:2656: left oid 29 (ObjNum 929 snap 274 seq_num 929) 2026-02-21T22:27:18.236 INFO:tasks.rados.rados.0.trial178.stdout:2661: finishing write tid 1 to trial17818531-27 2026-02-21T22:27:18.236 INFO:tasks.rados.rados.0.trial178.stdout:2660: expect (ObjNum 925 snap 271 seq_num 925) 2026-02-21T22:27:18.250 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1280776 2026-02-21T22:27:18.300 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:18.353 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 334522 2026-02-21T22:27:18.403 INFO:tasks.rados.rados.0.trial178.stdout:2661: finishing write tid 2 to trial17818531-27 2026-02-21T22:27:18.403 INFO:tasks.rados.rados.0.trial178.stdout:2661: finishing write tid 3 to trial17818531-27 2026-02-21T22:27:18.403 INFO:tasks.rados.rados.0.trial178.stdout:2661: finishing write tid 4 to trial17818531-27 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: finishing write tid 5 to trial17818531-27 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: oid 27 updating version 0 to 1194 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: oid 27 updating version 1194 to 1197 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: oid 27 version 1197 is already newer than 1196 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: oid 27 version 1197 is already newer than 1195 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1197 (ObjNum 931 snap 274 seq_num 931) dirty exists 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2661: left oid 27 (ObjNum 931 snap 274 seq_num 931) 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2662: finishing write tid 1 to trial17818531-37 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2662: finishing write tid 2 to trial17818531-37 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2662: finishing write tid 3 to trial17818531-37 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2662: finishing write tid 4 to trial17818531-37 2026-02-21T22:27:18.404 INFO:tasks.rados.rados.0.trial178.stdout:2662: finishing write tid 5 to trial17818531-37 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:2662: oid 37 updating version 0 to 1302 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:2662: oid 37 version 1302 is already newer than 1301 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:2662: oid 37 updating version 1302 to 1304 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:2662: oid 37 version 1304 is already newer than 1303 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1304 (ObjNum 932 snap 274 seq_num 932) dirty exists 2026-02-21T22:27:18.405 INFO:tasks.rados.rados.0.trial178.stdout:2662: left oid 37 (ObjNum 932 snap 274 seq_num 932) 2026-02-21T22:27:18.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:18.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2656: done (6 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2657: done (5 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2658: done (4 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2660: done (3 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2661: done (2 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2662: done (1 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2663: done (0 left) 2026-02-21T22:27:19.199 INFO:tasks.rados.rados.0.trial178.stdout:2664: write (excl) oid 35 current snap is 275 2026-02-21T22:27:19.200 INFO:tasks.rados.rados.0.trial178.stdout:2664: seq_num 933 ranges {480931=402383,1313935=515645,2488711=452788,3443230=237701} 2026-02-21T22:27:19.202 INFO:tasks.rados.rados.0.trial178.stdout:2664: writing trial17818531-35 from 480931 to 883314 tid 1 2026-02-21T22:27:19.206 INFO:tasks.rados.rados.0.trial178.stdout:2664: writing trial17818531-35 from 1313935 to 1829580 tid 2 2026-02-21T22:27:19.210 INFO:tasks.rados.rados.0.trial178.stdout:2664: writing trial17818531-35 from 2488711 to 2941499 tid 3 2026-02-21T22:27:19.213 INFO:tasks.rados.rados.0.trial178.stdout:2664: writing trial17818531-35 from 3443230 to 3680931 tid 4 2026-02-21T22:27:19.213 INFO:tasks.rados.rados.0.trial178.stdout:2665: write (excl) oid 50 current snap is 275 2026-02-21T22:27:19.213 INFO:tasks.rados.rados.0.trial178.stdout:2665: seq_num 934 ranges {497738=477632,1772417=775656,3316134=381604} 2026-02-21T22:27:19.217 INFO:tasks.rados.rados.0.trial178.stdout:2665: writing trial17818531-50 from 497738 to 975370 tid 1 2026-02-21T22:27:19.223 INFO:tasks.rados.rados.0.trial178.stdout:2665: writing trial17818531-50 from 1772417 to 2548073 tid 2 2026-02-21T22:27:19.227 INFO:tasks.rados.rados.0.trial178.stdout:2665: writing trial17818531-50 from 3316134 to 3697738 tid 3 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2666: read oid 34 snap -1 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2666: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2667: read oid 31 snap 271 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2667: expect (ObjNum 918 snap 270 seq_num 918) 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2668: read oid 9 snap -1 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2668: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:19.228 INFO:tasks.rados.rados.0.trial178.stdout:2669: snap_remove snap 266 2026-02-21T22:27:20.204 INFO:tasks.rados.rados.0.trial178.stdout:2669: done (5 left) 2026-02-21T22:27:20.205 INFO:tasks.rados.rados.0.trial178.stdout:2670: snap_remove snap 262 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2670: done (5 left) 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2671: read oid 5 snap -1 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2671: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2672: rollback oid 18 current snap is 275 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 18 to 271 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 1 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 2 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 3 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 4 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 5 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: finishing write tid 6 to trial17818531-35 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: oid 35 updating version 0 to 1312 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: oid 35 updating version 1312 to 1315 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: oid 35 version 1315 is already newer than 1311 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: oid 35 version 1315 is already newer than 1314 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:2664: oid 35 version 1315 is already newer than 1313 2026-02-21T22:27:21.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1315 (ObjNum 933 snap 275 seq_num 933) dirty exists 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2664: left oid 35 (ObjNum 933 snap 275 seq_num 933) 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2665: finishing write tid 1 to trial17818531-50 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2665: finishing write tid 2 to trial17818531-50 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2664: done (6 left) 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2673: read oid 42 snap -1 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2673: expect deleted 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2674: write oid 37 current snap is 275 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2674: seq_num 935 ranges {514545=552881,1830899=519314,2943557=475332} 2026-02-21T22:27:21.218 INFO:tasks.rados.rados.0.trial178.stdout:2674: writing trial17818531-37 from 514545 to 1067426 tid 1 2026-02-21T22:27:21.223 INFO:tasks.rados.rados.0.trial178.stdout:2674: writing trial17818531-37 from 1830899 to 2350213 tid 2 2026-02-21T22:27:21.230 INFO:tasks.rados.rados.0.trial178.stdout:2674: writing trial17818531-37 from 2943557 to 3418889 tid 3 2026-02-21T22:27:21.230 INFO:tasks.rados.rados.0.trial178.stdout:2675: snap_remove snap 270 2026-02-21T22:27:21.230 INFO:tasks.rados.rados.0.trial178.stdout:2666: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:21.318 INFO:tasks.rados.rados.0.trial178.stdout:2665: finishing write tid 3 to trial17818531-50 2026-02-21T22:27:21.318 INFO:tasks.rados.rados.0.trial178.stdout:2665: finishing write tid 4 to trial17818531-50 2026-02-21T22:27:21.318 INFO:tasks.rados.rados.0.trial178.stdout:2665: finishing write tid 5 to trial17818531-50 2026-02-21T22:27:21.318 INFO:tasks.rados.rados.0.trial178.stdout:2665: oid 50 updating version 0 to 1168 2026-02-21T22:27:21.318 INFO:tasks.rados.rados.0.trial178.stdout:2665: oid 50 updating version 1168 to 1169 2026-02-21T22:27:21.319 INFO:tasks.rados.rados.0.trial178.stdout:2665: oid 50 updating version 1169 to 1170 2026-02-21T22:27:21.319 INFO:tasks.rados.rados.0.trial178.stdout:2665: oid 50 version 1170 is already newer than 1167 2026-02-21T22:27:21.319 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1170 (ObjNum 934 snap 275 seq_num 934) dirty exists 2026-02-21T22:27:21.319 INFO:tasks.rados.rados.0.trial178.stdout:2665: left oid 50 (ObjNum 934 snap 275 seq_num 934) 2026-02-21T22:27:21.319 INFO:tasks.rados.rados.0.trial178.stdout:2668: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:21.325 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1028084 2026-02-21T22:27:21.413 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:21.466 INFO:tasks.rados.rados.0.trial178.stdout:2667: expect (ObjNum 918 snap 270 seq_num 918) 2026-02-21T22:27:21.487 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:21.536 INFO:tasks.rados.rados.0.trial178.stdout:2671: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:21.679 INFO:tasks.rados.rados.0.trial178.stdout:2672: finishing rollback tid 1 to trial17818531-18 2026-02-21T22:27:21.679 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1428 (ObjNum 529 snap 0 seq_num 100491528) dirty dne 2026-02-21T22:27:21.679 INFO:tasks.rados.rados.0.trial178.stdout:2674: finishing write tid 1 to trial17818531-37 2026-02-21T22:27:21.679 INFO:tasks.rados.rados.0.trial178.stdout:2674: finishing write tid 2 to trial17818531-37 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2665: done (8 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2666: done (7 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2667: done (6 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2668: done (5 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2671: done (4 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2672: done (3 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2673: done (2 left) 2026-02-21T22:27:22.173 INFO:tasks.rados.rados.0.trial178.stdout:2675: done (1 left) 2026-02-21T22:27:22.174 INFO:tasks.rados.rados.0.trial178.stdout:2676: snap_remove snap 271 2026-02-21T22:27:23.174 INFO:tasks.rados.rados.0.trial178.stdout:2676: done (1 left) 2026-02-21T22:27:23.174 INFO:tasks.rados.rados.0.trial178.stdout:2677: snap_create 2026-02-21T22:27:23.174 INFO:tasks.rados.rados.0.trial178.stdout:2674: finishing write tid 3 to trial17818531-37 2026-02-21T22:27:23.174 INFO:tasks.rados.rados.0.trial178.stdout:2674: finishing write tid 4 to trial17818531-37 2026-02-21T22:27:23.174 INFO:tasks.rados.rados.0.trial178.stdout:2674: finishing write tid 5 to trial17818531-37 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:2674: oid 37 updating version 0 to 1308 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:2674: oid 37 version 1308 is already newer than 1306 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:2674: oid 37 version 1308 is already newer than 1307 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:2674: oid 37 version 1308 is already newer than 1305 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1308 (ObjNum 935 snap 275 seq_num 935) dirty exists 2026-02-21T22:27:23.175 INFO:tasks.rados.rados.0.trial178.stdout:2674: left oid 37 (ObjNum 935 snap 275 seq_num 935) 2026-02-21T22:27:23.532 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:27:23] 2026-02-21T22:27:23.533 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:27:23, mode upmap, max misplaced 0.050000 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'unique_pool_0', 'rbd'] 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:27:23.534 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:27:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:23.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:23.559 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:27:23.560 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:27:23.561 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:27:23.562 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:27:23.563 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:27:23.564 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:27:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:27:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:27:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:27:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:27:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:27:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:27:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:27:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:27:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:27:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:27:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "571", 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:27:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:27:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:27:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:27:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 571, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 281, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:27:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:27:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:27:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00045715614601417825 of space, bias 1.0, pg target 0.13714684380425346 quantized to 32 (current 16) 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:24.180 INFO:tasks.rados.rados.0.trial178.stdout:2674: done (1 left) 2026-02-21T22:27:24.180 INFO:tasks.rados.rados.0.trial178.stdout:2677: done (0 left) 2026-02-21T22:27:24.180 INFO:tasks.rados.rados.0.trial178.stdout:2678: delete oid 3 current snap is 276 2026-02-21T22:27:24.185 INFO:tasks.rados.rados.0.trial178.stdout:2678: done (0 left) 2026-02-21T22:27:24.185 INFO:tasks.rados.rados.0.trial178.stdout:2679: read oid 42 snap -1 2026-02-21T22:27:24.185 INFO:tasks.rados.rados.0.trial178.stdout:2679: expect deleted 2026-02-21T22:27:24.185 INFO:tasks.rados.rados.0.trial178.stdout:2680: write (excl) oid 9 current snap is 276 2026-02-21T22:27:24.185 INFO:tasks.rados.rados.0.trial178.stdout:2680: seq_num 937 ranges {548159=419732,1664216=522983,2714756=614229} 2026-02-21T22:27:24.191 INFO:tasks.rados.rados.0.trial178.stdout:2680: writing trial17818531-9 from 548159 to 967891 tid 1 2026-02-21T22:27:24.199 INFO:tasks.rados.rados.0.trial178.stdout:2680: writing trial17818531-9 from 1664216 to 2187199 tid 2 2026-02-21T22:27:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2680: writing trial17818531-9 from 2714756 to 3328985 tid 3 2026-02-21T22:27:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2681: write (excl) oid 41 current snap is 276 2026-02-21T22:27:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2681: seq_num 938 ranges {564966=494981,1606345=782994,3025826=425501} 2026-02-21T22:27:24.214 INFO:tasks.rados.rados.0.trial178.stdout:2681: writing trial17818531-41 from 564966 to 1059947 tid 1 2026-02-21T22:27:24.223 INFO:tasks.rados.rados.0.trial178.stdout:2681: writing trial17818531-41 from 1606345 to 2389339 tid 2 2026-02-21T22:27:24.230 INFO:tasks.rados.rados.0.trial178.stdout:2681: writing trial17818531-41 from 3025826 to 3451327 tid 3 2026-02-21T22:27:24.230 INFO:tasks.rados.rados.0.trial178.stdout:2682: write oid 8 current snap is 276 2026-02-21T22:27:24.230 INFO:tasks.rados.rados.0.trial178.stdout:2682: seq_num 939 ranges {581773=570230,1664827=526652,2653249=636773} 2026-02-21T22:27:24.234 INFO:tasks.rados.rados.0.trial178.stdout:2682: writing trial17818531-8 from 581773 to 1152003 tid 1 2026-02-21T22:27:24.238 INFO:tasks.rados.rados.0.trial178.stdout:2682: writing trial17818531-8 from 1664827 to 2191479 tid 2 2026-02-21T22:27:24.242 INFO:tasks.rados.rados.0.trial178.stdout:2682: writing trial17818531-8 from 2653249 to 3290022 tid 3 2026-02-21T22:27:24.242 INFO:tasks.rados.rados.0.trial178.stdout:2683: write oid 43 current snap is 276 2026-02-21T22:27:24.243 INFO:tasks.rados.rados.0.trial178.stdout:2683: seq_num 940 ranges {598580=645479,1723309=786663,3080672=448045} 2026-02-21T22:27:24.247 INFO:tasks.rados.rados.0.trial178.stdout:2683: writing trial17818531-43 from 598580 to 1244059 tid 1 2026-02-21T22:27:24.251 INFO:tasks.rados.rados.0.trial178.stdout:2683: writing trial17818531-43 from 1723309 to 2509972 tid 2 2026-02-21T22:27:24.256 INFO:tasks.rados.rados.0.trial178.stdout:2683: writing trial17818531-43 from 3080672 to 3528717 tid 3 2026-02-21T22:27:24.256 INFO:tasks.rados.rados.0.trial178.stdout:2684: delete oid 49 current snap is 276 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: finishing write tid 1 to trial17818531-9 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: finishing write tid 2 to trial17818531-9 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: finishing write tid 3 to trial17818531-9 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: finishing write tid 4 to trial17818531-9 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: finishing write tid 5 to trial17818531-9 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: oid 9 updating version 0 to 1199 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: oid 9 updating version 1199 to 1200 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: oid 9 updating version 1200 to 1201 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: oid 9 version 1201 is already newer than 1198 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1201 (ObjNum 937 snap 276 seq_num 937) dirty exists 2026-02-21T22:27:24.257 INFO:tasks.rados.rados.0.trial178.stdout:2680: left oid 9 (ObjNum 937 snap 276 seq_num 937) 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: finishing write tid 1 to trial17818531-41 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: finishing write tid 2 to trial17818531-41 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: finishing write tid 3 to trial17818531-41 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: finishing write tid 4 to trial17818531-41 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: finishing write tid 5 to trial17818531-41 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: oid 41 updating version 0 to 1055 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: oid 41 version 1055 is already newer than 1052 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: oid 41 version 1055 is already newer than 1053 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: oid 41 version 1055 is already newer than 1054 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1055 (ObjNum 938 snap 276 seq_num 938) dirty exists 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2681: left oid 41 (ObjNum 938 snap 276 seq_num 938) 2026-02-21T22:27:24.258 INFO:tasks.rados.rados.0.trial178.stdout:2682: finishing write tid 1 to trial17818531-8 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: finishing write tid 2 to trial17818531-8 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: finishing write tid 3 to trial17818531-8 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: finishing write tid 4 to trial17818531-8 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: finishing write tid 5 to trial17818531-8 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: oid 8 updating version 0 to 1528 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: oid 8 updating version 1528 to 1531 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: oid 8 version 1531 is already newer than 1530 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: oid 8 version 1531 is already newer than 1529 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1531 (ObjNum 939 snap 276 seq_num 939) dirty exists 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2682: left oid 8 (ObjNum 939 snap 276 seq_num 939) 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2683: finishing write tid 1 to trial17818531-43 2026-02-21T22:27:24.259 INFO:tasks.rados.rados.0.trial178.stdout:2683: finishing write tid 2 to trial17818531-43 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2679: done (5 left) 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2680: done (4 left) 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2681: done (3 left) 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2682: done (2 left) 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2684: done (1 left) 2026-02-21T22:27:24.260 INFO:tasks.rados.rados.0.trial178.stdout:2685: snap_remove snap 265 2026-02-21T22:27:25.218 INFO:tasks.rados.rados.0.trial178.stdout:2685: done (1 left) 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2686: delete oid 12 current snap is 276 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: finishing write tid 3 to trial17818531-43 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: finishing write tid 4 to trial17818531-43 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: finishing write tid 5 to trial17818531-43 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: oid 43 updating version 0 to 1145 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: oid 43 version 1145 is already newer than 1144 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: oid 43 updating version 1145 to 1147 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:2683: oid 43 version 1147 is already newer than 1146 2026-02-21T22:27:25.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1147 (ObjNum 940 snap 276 seq_num 940) dirty exists 2026-02-21T22:27:25.220 INFO:tasks.rados.rados.0.trial178.stdout:2683: left oid 43 (ObjNum 940 snap 276 seq_num 940) 2026-02-21T22:27:25.222 INFO:tasks.rados.rados.0.trial178.stdout:2683: done (1 left) 2026-02-21T22:27:25.222 INFO:tasks.rados.rados.0.trial178.stdout:2686: done (0 left) 2026-02-21T22:27:25.223 INFO:tasks.rados.rados.0.trial178.stdout:2687: snap_remove snap 274 2026-02-21T22:27:26.343 INFO:tasks.rados.rados.0.trial178.stdout:2687: done (0 left) 2026-02-21T22:27:26.343 INFO:tasks.rados.rados.0.trial178.stdout:2688: snap_remove snap 275 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2688: done (0 left) 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2689: read oid 38 snap -1 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2689: expect (ObjNum 899 snap 265 seq_num 899) 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2690: read oid 36 snap -1 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2690: expect (ObjNum 925 snap 271 seq_num 925) 2026-02-21T22:27:27.174 INFO:tasks.rados.rados.0.trial178.stdout:2691: snap_create 2026-02-21T22:27:27.181 INFO:tasks.rados.rados.0.trial178.stdout:2689: expect (ObjNum 899 snap 265 seq_num 899) 2026-02-21T22:27:27.181 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:27.285 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:27.325 INFO:tasks.rados.rados.0.trial178.stdout:2690: expect (ObjNum 925 snap 271 seq_num 925) 2026-02-21T22:27:27.378 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:27.453 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2044405 2026-02-21T22:27:28.178 INFO:tasks.rados.rados.0.trial178.stdout:2689: done (2 left) 2026-02-21T22:27:28.178 INFO:tasks.rados.rados.0.trial178.stdout:2690: done (1 left) 2026-02-21T22:27:28.178 INFO:tasks.rados.rados.0.trial178.stdout:2691: done (0 left) 2026-02-21T22:27:28.178 INFO:tasks.rados.rados.0.trial178.stdout:2692: write (excl) oid 36 current snap is 277 2026-02-21T22:27:28.179 INFO:tasks.rados.rados.0.trial178.stdout:2692: seq_num 943 ranges {649001=587579,1898755=533990,3162941=686060} 2026-02-21T22:27:28.186 INFO:tasks.rados.rados.0.trial178.stdout:2692: writing trial17818531-36 from 649001 to 1236580 tid 1 2026-02-21T22:27:28.194 INFO:tasks.rados.rados.0.trial178.stdout:2692: writing trial17818531-36 from 1898755 to 2432745 tid 2 2026-02-21T22:27:28.204 INFO:tasks.rados.rados.0.trial178.stdout:2692: writing trial17818531-36 from 3162941 to 3849001 tid 3 2026-02-21T22:27:28.204 INFO:tasks.rados.rados.0.trial178.stdout:2693: read oid 17 snap -1 2026-02-21T22:27:28.204 INFO:tasks.rados.rados.0.trial178.stdout:2693: expect (ObjNum 797 snap 224 seq_num 797) 2026-02-21T22:27:28.204 INFO:tasks.rados.rados.0.trial178.stdout:2694: read oid 9 snap 272 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:2694: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:2695: rollback oid 40 current snap is 277 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 276 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:2692: finishing write tid 1 to trial17818531-36 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:2692: finishing write tid 2 to trial17818531-36 2026-02-21T22:27:28.205 INFO:tasks.rados.rados.0.trial178.stdout:2696: delete oid 46 current snap is 277 2026-02-21T22:27:28.206 INFO:tasks.rados.rados.0.trial178.stdout:2696: done (4 left) 2026-02-21T22:27:28.206 INFO:tasks.rados.rados.0.trial178.stdout:2697: read oid 35 snap 276 2026-02-21T22:27:28.207 INFO:tasks.rados.rados.0.trial178.stdout:2697: expect (ObjNum 933 snap 275 seq_num 933) 2026-02-21T22:27:28.207 INFO:tasks.rados.rados.0.trial178.stdout:2698: write oid 49 current snap is 277 2026-02-21T22:27:28.207 INFO:tasks.rados.rados.0.trial178.stdout:2698: seq_num 945 ranges {682615=738077,2015719=537659,3217787=420758} 2026-02-21T22:27:28.216 INFO:tasks.rados.rados.0.trial178.stdout:2698: writing trial17818531-49 from 682615 to 1420692 tid 1 2026-02-21T22:27:28.223 INFO:tasks.rados.rados.0.trial178.stdout:2698: writing trial17818531-49 from 2015719 to 2553378 tid 2 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2698: writing trial17818531-49 from 3217787 to 3638545 tid 3 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2699: delete oid 13 current snap is 277 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2692: finishing write tid 3 to trial17818531-36 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2692: finishing write tid 4 to trial17818531-36 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2692: finishing write tid 5 to trial17818531-36 2026-02-21T22:27:28.231 INFO:tasks.rados.rados.0.trial178.stdout:2692: oid 36 updating version 0 to 1090 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:2692: oid 36 updating version 1090 to 1092 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:2692: oid 36 version 1092 is already newer than 1091 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:2692: oid 36 updating version 1092 to 1093 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1093 (ObjNum 943 snap 277 seq_num 943) dirty exists 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:2692: left oid 36 (ObjNum 943 snap 277 seq_num 943) 2026-02-21T22:27:28.232 INFO:tasks.rados.rados.0.trial178.stdout:2694: expect (ObjNum 889 snap 261 seq_num 889) 2026-02-21T22:27:28.352 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:28.407 INFO:tasks.rados.rados.0.trial178.stdout:2693: expect (ObjNum 797 snap 224 seq_num 797) 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:2695: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:2695: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:2695: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1173 (ObjNum 854 snap 243 seq_num 854) dirty exists 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:2698: finishing write tid 1 to trial17818531-49 2026-02-21T22:27:28.522 INFO:tasks.rados.rados.0.trial178.stdout:2697: expect (ObjNum 933 snap 275 seq_num 933) 2026-02-21T22:27:28.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:28.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:28.579 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:28.693 INFO:tasks.rados.rados.0.trial178.stdout:2698: finishing write tid 2 to trial17818531-49 2026-02-21T22:27:28.693 INFO:tasks.rados.rados.0.trial178.stdout:2698: finishing write tid 3 to trial17818531-49 2026-02-21T22:27:28.693 INFO:tasks.rados.rados.0.trial178.stdout:2698: finishing write tid 4 to trial17818531-49 2026-02-21T22:27:28.693 INFO:tasks.rados.rados.0.trial178.stdout:2692: done (6 left) 2026-02-21T22:27:28.693 INFO:tasks.rados.rados.0.trial178.stdout:2693: done (5 left) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2694: done (4 left) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2695: done (3 left) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2697: done (2 left) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2699: done (1 left) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2700: read oid 5 snap -1 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2700: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2701: delete oid 22 current snap is 277 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2698: finishing write tid 5 to trial17818531-49 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2698: oid 49 updating version 0 to 1173 2026-02-21T22:27:28.694 INFO:tasks.rados.rados.0.trial178.stdout:2698: oid 49 updating version 1173 to 1175 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:2698: oid 49 version 1175 is already newer than 1174 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:2698: oid 49 updating version 1175 to 1176 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1176 (ObjNum 945 snap 277 seq_num 945) dirty exists 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:2698: left oid 49 (ObjNum 945 snap 277 seq_num 945) 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:2698: done (2 left) 2026-02-21T22:27:28.695 INFO:tasks.rados.rados.0.trial178.stdout:2701: done (1 left) 2026-02-21T22:27:28.696 INFO:tasks.rados.rados.0.trial178.stdout:2702: snap_create 2026-02-21T22:27:28.696 INFO:tasks.rados.rados.0.trial178.stdout:2700: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:28.757 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 248476 2026-02-21T22:27:28.803 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 185822 2026-02-21T22:27:29.223 INFO:tasks.rados.rados.0.trial178.stdout:2700: done (1 left) 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2702: done (0 left) 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2703: read oid 36 snap 277 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2703: expect (ObjNum 943 snap 277 seq_num 943) 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2704: rollback oid 27 current snap is 278 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 277 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2705: write (excl) oid 25 current snap is 278 2026-02-21T22:27:29.224 INFO:tasks.rados.rados.0.trial178.stdout:2705: seq_num 948 ranges {733036=563824,2074812=684986,3183703=749333} 2026-02-21T22:27:29.232 INFO:tasks.rados.rados.0.trial178.stdout:2705: writing trial17818531-25 from 733036 to 1296860 tid 1 2026-02-21T22:27:29.241 INFO:tasks.rados.rados.0.trial178.stdout:2705: writing trial17818531-25 from 2074812 to 2759798 tid 2 2026-02-21T22:27:29.249 INFO:tasks.rados.rados.0.trial178.stdout:2705: writing trial17818531-25 from 3183703 to 3933036 tid 3 2026-02-21T22:27:29.249 INFO:tasks.rados.rados.0.trial178.stdout:2706: snap_remove snap 272 2026-02-21T22:27:30.226 INFO:tasks.rados.rados.0.trial178.stdout:2706: done (3 left) 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2707: read oid 22 snap -1 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2707: expect deleted 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2708: read oid 43 snap 276 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2708: expect (ObjNum 940 snap 276 seq_num 940) 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2709: write oid 3 current snap is 278 2026-02-21T22:27:30.227 INFO:tasks.rados.rados.0.trial178.stdout:2709: seq_num 949 ranges {749843=639073,2133294=544997,3211126=582199} 2026-02-21T22:27:30.235 INFO:tasks.rados.rados.0.trial178.stdout:2709: writing trial17818531-3 from 749843 to 1388916 tid 1 2026-02-21T22:27:30.244 INFO:tasks.rados.rados.0.trial178.stdout:2709: writing trial17818531-3 from 2133294 to 2678291 tid 2 2026-02-21T22:27:30.252 INFO:tasks.rados.rados.0.trial178.stdout:2709: writing trial17818531-3 from 3211126 to 3793325 tid 3 2026-02-21T22:27:30.252 INFO:tasks.rados.rados.0.trial178.stdout:2710: delete oid 29 current snap is 278 2026-02-21T22:27:30.252 INFO:tasks.rados.rados.0.trial178.stdout:2704: finishing rollback tid 0 to trial17818531-27 2026-02-21T22:27:30.252 INFO:tasks.rados.rados.0.trial178.stdout:2704: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:27:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2704: finishing rollback tid 2 to trial17818531-27 2026-02-21T22:27:30.253 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1204 (ObjNum 931 snap 274 seq_num 931) dirty exists 2026-02-21T22:27:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2703: expect (ObjNum 943 snap 277 seq_num 943) 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: finishing write tid 1 to trial17818531-25 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: finishing write tid 2 to trial17818531-25 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: finishing write tid 3 to trial17818531-25 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: finishing write tid 4 to trial17818531-25 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: finishing write tid 5 to trial17818531-25 2026-02-21T22:27:30.458 INFO:tasks.rados.rados.0.trial178.stdout:2705: oid 25 updating version 0 to 1179 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:2705: oid 25 version 1179 is already newer than 1178 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:2705: oid 25 version 1179 is already newer than 1177 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:2705: oid 25 updating version 1179 to 1180 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1180 (ObjNum 948 snap 278 seq_num 948) dirty exists 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:2705: left oid 25 (ObjNum 948 snap 278 seq_num 948) 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:2708: expect (ObjNum 940 snap 276 seq_num 940) 2026-02-21T22:27:30.459 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:30.501 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: finishing write tid 1 to trial17818531-3 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: finishing write tid 2 to trial17818531-3 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: finishing write tid 3 to trial17818531-3 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: finishing write tid 4 to trial17818531-3 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: finishing write tid 5 to trial17818531-3 2026-02-21T22:27:30.529 INFO:tasks.rados.rados.0.trial178.stdout:2709: oid 3 updating version 0 to 1342 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2709: oid 3 updating version 1342 to 1345 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2709: oid 3 version 1345 is already newer than 1344 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2709: oid 3 version 1345 is already newer than 1343 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1345 (ObjNum 949 snap 278 seq_num 949) dirty exists 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2709: left oid 3 (ObjNum 949 snap 278 seq_num 949) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2703: done (6 left) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2704: done (5 left) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2705: done (4 left) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2707: done (3 left) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2708: done (2 left) 2026-02-21T22:27:30.530 INFO:tasks.rados.rados.0.trial178.stdout:2709: done (1 left) 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2710: done (0 left) 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2711: read oid 15 snap 276 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2711: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2712: rollback oid 10 current snap is 278 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 276 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2713: rollback oid 47 current snap is 278 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 276 2026-02-21T22:27:30.531 INFO:tasks.rados.rados.0.trial178.stdout:2714: delete oid 17 current snap is 278 2026-02-21T22:27:30.532 INFO:tasks.rados.rados.0.trial178.stdout:2713: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:27:30.532 INFO:tasks.rados.rados.0.trial178.stdout:2713: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:27:30.532 INFO:tasks.rados.rados.0.trial178.stdout:2713: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:27:30.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1549 (ObjNum 881 snap 257 seq_num 881) dirty exists 2026-02-21T22:27:30.533 INFO:tasks.rados.rados.0.trial178.stdout:2712: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:27:30.533 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:27:30.535 INFO:tasks.rados.rados.0.trial178.stdout:2711: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:30.599 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3373431 2026-02-21T22:27:30.641 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2480556 2026-02-21T22:27:30.738 INFO:tasks.rados.rados.0.trial178.stdout:2711: done (3 left) 2026-02-21T22:27:30.738 INFO:tasks.rados.rados.0.trial178.stdout:2712: done (2 left) 2026-02-21T22:27:30.738 INFO:tasks.rados.rados.0.trial178.stdout:2713: done (1 left) 2026-02-21T22:27:30.738 INFO:tasks.rados.rados.0.trial178.stdout:2714: done (0 left) 2026-02-21T22:27:30.739 INFO:tasks.rados.rados.0.trial178.stdout:2715: delete oid 17 current snap is 278 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2715: done (0 left) 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2716: copy_from oid 5 from oid 13 current snap is 278 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2717: copy_from oid 46 from oid 48 current snap is 278 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2718: delete oid 33 current snap is 278 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2716: finishing copy_from to trial17818531-5 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2716: got expected ENOENT (src dne) 2026-02-21T22:27:30.740 INFO:tasks.rados.rados.0.trial178.stdout:2716: finishing copy_from racing read to trial17818531-5 2026-02-21T22:27:30.741 INFO:tasks.rados.rados.0.trial178.stdout:2716: done (2 left) 2026-02-21T22:27:30.742 INFO:tasks.rados.rados.0.trial178.stdout:2718: done (1 left) 2026-02-21T22:27:30.742 INFO:tasks.rados.rados.0.trial178.stdout:2719: write oid 5 current snap is 278 2026-02-21T22:27:30.742 INFO:tasks.rados.rados.0.trial178.stdout:2719: seq_num 954 ranges {433878=731671,1909351=124527} 2026-02-21T22:27:30.749 INFO:tasks.rados.rados.0.trial178.stdout:2719: writing trial17818531-5 from 433878 to 1165549 tid 1 2026-02-21T22:27:30.755 INFO:tasks.rados.rados.0.trial178.stdout:2719: writing trial17818531-5 from 1909351 to 2033878 tid 2 2026-02-21T22:27:30.755 INFO:tasks.rados.rados.0.trial178.stdout:2720: write (excl) oid 33 current snap is 278 2026-02-21T22:27:30.756 INFO:tasks.rados.rados.0.trial178.stdout:2720: seq_num 955 ranges {450685=406920,1567833=482852} 2026-02-21T22:27:30.761 INFO:tasks.rados.rados.0.trial178.stdout:2720: writing trial17818531-33 from 450685 to 857605 tid 1 2026-02-21T22:27:30.769 INFO:tasks.rados.rados.0.trial178.stdout:2720: writing trial17818531-33 from 1567833 to 2050685 tid 2 2026-02-21T22:27:30.769 INFO:tasks.rados.rados.0.trial178.stdout:2721: read oid 22 snap -1 2026-02-21T22:27:30.769 INFO:tasks.rados.rados.0.trial178.stdout:2721: expect deleted 2026-02-21T22:27:30.770 INFO:tasks.rados.rados.0.trial178.stdout:2722: read oid 6 snap -1 2026-02-21T22:27:30.770 INFO:tasks.rados.rados.0.trial178.stdout:2722: expect deleted 2026-02-21T22:27:30.770 INFO:tasks.rados.rados.0.trial178.stdout:2723: write oid 13 current snap is 278 2026-02-21T22:27:30.770 INFO:tasks.rados.rados.0.trial178.stdout:2723: seq_num 956 ranges {467492=482169,1626315=441177} 2026-02-21T22:27:30.775 INFO:tasks.rados.rados.0.trial178.stdout:2723: writing trial17818531-13 from 467492 to 949661 tid 1 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2723: writing trial17818531-13 from 1626315 to 2067492 tid 2 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2724: delete oid 25 current snap is 278 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2717: finishing copy_from racing read to trial17818531-46 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2717: finishing copy_from to trial17818531-46 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1296 (ObjNum 904 snap 268 seq_num 904) dirty exists 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2719: finishing write tid 1 to trial17818531-5 2026-02-21T22:27:30.784 INFO:tasks.rados.rados.0.trial178.stdout:2719: finishing write tid 2 to trial17818531-5 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: finishing write tid 3 to trial17818531-5 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: finishing write tid 4 to trial17818531-5 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: oid 5 updating version 0 to 1431 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: oid 5 version 1431 is already newer than 1429 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: oid 5 version 1431 is already newer than 1430 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1431 (ObjNum 954 snap 278 seq_num 954) dirty exists 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2719: left oid 5 (ObjNum 954 snap 278 seq_num 954) 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2720: finishing write tid 1 to trial17818531-33 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2720: finishing write tid 2 to trial17818531-33 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2720: finishing write tid 3 to trial17818531-33 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2720: finishing write tid 4 to trial17818531-33 2026-02-21T22:27:30.785 INFO:tasks.rados.rados.0.trial178.stdout:2720: oid 33 updating version 0 to 858 2026-02-21T22:27:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2720: oid 33 updating version 858 to 860 2026-02-21T22:27:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2720: oid 33 version 860 is already newer than 859 2026-02-21T22:27:30.786 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 860 (ObjNum 955 snap 278 seq_num 955) dirty exists 2026-02-21T22:27:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2720: left oid 33 (ObjNum 955 snap 278 seq_num 955) 2026-02-21T22:27:30.786 INFO:tasks.rados.rados.0.trial178.stdout:2723: finishing write tid 1 to trial17818531-13 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2723: finishing write tid 2 to trial17818531-13 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2717: done (6 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2719: done (5 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2720: done (4 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2721: done (3 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2722: done (2 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2724: done (1 left) 2026-02-21T22:27:30.787 INFO:tasks.rados.rados.0.trial178.stdout:2725: write oid 18 current snap is 278 2026-02-21T22:27:30.788 INFO:tasks.rados.rados.0.trial178.stdout:2725: seq_num 958 ranges {501106=749020,1743279=357827} 2026-02-21T22:27:30.793 INFO:tasks.rados.rados.0.trial178.stdout:2725: writing trial17818531-18 from 501106 to 1250126 tid 1 2026-02-21T22:27:30.799 INFO:tasks.rados.rados.0.trial178.stdout:2725: writing trial17818531-18 from 1743279 to 2101106 tid 2 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2726: delete oid 32 current snap is 278 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: finishing write tid 3 to trial17818531-13 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: finishing write tid 4 to trial17818531-13 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: oid 13 updating version 0 to 1347 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: oid 13 version 1347 is already newer than 1346 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: oid 13 updating version 1347 to 1348 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1348 (ObjNum 956 snap 278 seq_num 956) dirty exists 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2723: left oid 13 (ObjNum 956 snap 278 seq_num 956) 2026-02-21T22:27:30.800 INFO:tasks.rados.rados.0.trial178.stdout:2725: finishing write tid 1 to trial17818531-18 2026-02-21T22:27:30.801 INFO:tasks.rados.rados.0.trial178.stdout:2725: finishing write tid 2 to trial17818531-18 2026-02-21T22:27:30.801 INFO:tasks.rados.rados.0.trial178.stdout:2725: finishing write tid 3 to trial17818531-18 2026-02-21T22:27:30.801 INFO:tasks.rados.rados.0.trial178.stdout:2725: finishing write tid 4 to trial17818531-18 2026-02-21T22:27:30.801 INFO:tasks.rados.rados.0.trial178.stdout:2725: oid 18 updating version 0 to 1434 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2725: oid 18 version 1434 is already newer than 1433 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2725: oid 18 version 1434 is already newer than 1432 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1434 (ObjNum 958 snap 278 seq_num 958) dirty exists 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2725: left oid 18 (ObjNum 958 snap 278 seq_num 958) 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2723: done (2 left) 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2725: done (1 left) 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2726: done (0 left) 2026-02-21T22:27:30.802 INFO:tasks.rados.rados.0.trial178.stdout:2727: snap_create 2026-02-21T22:27:31.230 INFO:tasks.rados.rados.0.trial178.stdout:2727: done (0 left) 2026-02-21T22:27:31.231 INFO:tasks.rados.rados.0.trial178.stdout:2728: rollback oid 12 current snap is 279 2026-02-21T22:27:31.231 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 277 2026-02-21T22:27:31.231 INFO:tasks.rados.rados.0.trial178.stdout:2729: write oid 46 current snap is 279 2026-02-21T22:27:31.231 INFO:tasks.rados.rados.0.trial178.stdout:2729: seq_num 960 ranges {534720=499518,1460243=674477} 2026-02-21T22:27:31.237 INFO:tasks.rados.rados.0.trial178.stdout:2729: writing trial17818531-46 from 534720 to 1034238 tid 1 2026-02-21T22:27:31.245 INFO:tasks.rados.rados.0.trial178.stdout:2729: writing trial17818531-46 from 1460243 to 2134720 tid 2 2026-02-21T22:27:31.245 INFO:tasks.rados.rados.0.trial178.stdout:2730: snap_create 2026-02-21T22:27:31.245 INFO:tasks.rados.rados.0.trial178.stdout:2728: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:27:31.245 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:27:31.245 INFO:tasks.rados.rados.0.trial178.stdout:2729: finishing write tid 1 to trial17818531-46 2026-02-21T22:27:31.248 INFO:tasks.rados.rados.0.trial178.stdout:2729: finishing write tid 2 to trial17818531-46 2026-02-21T22:27:31.248 INFO:tasks.rados.rados.0.trial178.stdout:2729: finishing write tid 3 to trial17818531-46 2026-02-21T22:27:31.248 INFO:tasks.rados.rados.0.trial178.stdout:2729: finishing write tid 4 to trial17818531-46 2026-02-21T22:27:31.248 INFO:tasks.rados.rados.0.trial178.stdout:2729: oid 46 updating version 0 to 1378 2026-02-21T22:27:31.249 INFO:tasks.rados.rados.0.trial178.stdout:2729: oid 46 updating version 1378 to 1379 2026-02-21T22:27:31.249 INFO:tasks.rados.rados.0.trial178.stdout:2729: oid 46 updating version 1379 to 1380 2026-02-21T22:27:31.249 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1380 (ObjNum 960 snap 279 seq_num 960) dirty exists 2026-02-21T22:27:31.249 INFO:tasks.rados.rados.0.trial178.stdout:2729: left oid 46 (ObjNum 960 snap 279 seq_num 960) 2026-02-21T22:27:32.175 INFO:tasks.rados.rados.0.trial178.stdout:2728: done (2 left) 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2729: done (1 left) 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2730: done (0 left) 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2731: copy_from oid 37 from oid 36 current snap is 280 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2732: read oid 46 snap 277 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2732: expect deleted 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2733: write (excl) oid 25 current snap is 280 2026-02-21T22:27:32.176 INFO:tasks.rados.rados.0.trial178.stdout:2733: seq_num 961 ranges {551527=574767,1918725=232802} 2026-02-21T22:27:32.183 INFO:tasks.rados.rados.0.trial178.stdout:2733: writing trial17818531-25 from 551527 to 1126294 tid 1 2026-02-21T22:27:32.190 INFO:tasks.rados.rados.0.trial178.stdout:2733: writing trial17818531-25 from 1918725 to 2151527 tid 2 2026-02-21T22:27:32.190 INFO:tasks.rados.rados.0.trial178.stdout:2734: read oid 16 snap -1 2026-02-21T22:27:32.190 INFO:tasks.rados.rados.0.trial178.stdout:2734: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:32.190 INFO:tasks.rados.rados.0.trial178.stdout:2735: snap_remove snap 279 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2735: done (4 left) 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2736: read oid 30 snap 278 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2736: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2737: read oid 43 snap -1 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2737: expect (ObjNum 940 snap 276 seq_num 940) 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2738: write oid 45 current snap is 280 2026-02-21T22:27:33.179 INFO:tasks.rados.rados.0.trial178.stdout:2738: seq_num 962 ranges {568334=650016,1860854=307480} 2026-02-21T22:27:33.187 INFO:tasks.rados.rados.0.trial178.stdout:2738: writing trial17818531-45 from 568334 to 1218350 tid 1 2026-02-21T22:27:33.193 INFO:tasks.rados.rados.0.trial178.stdout:2738: writing trial17818531-45 from 1860854 to 2168334 tid 2 2026-02-21T22:27:33.194 INFO:tasks.rados.rados.0.trial178.stdout:2739: read oid 14 snap -1 2026-02-21T22:27:33.194 INFO:tasks.rados.rados.0.trial178.stdout:2739: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:27:33.194 INFO:tasks.rados.rados.0.trial178.stdout:2740: delete oid 8 current snap is 280 2026-02-21T22:27:33.194 INFO:tasks.rados.rados.0.trial178.stdout:2731: finishing copy_from racing read to trial17818531-37 2026-02-21T22:27:33.194 INFO:tasks.rados.rados.0.trial178.stdout:2734: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:33.277 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1632844 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: finishing write tid 1 to trial17818531-25 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: finishing write tid 2 to trial17818531-25 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: finishing write tid 3 to trial17818531-25 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: finishing write tid 4 to trial17818531-25 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: oid 25 updating version 0 to 1183 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: oid 25 updating version 1183 to 1184 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: oid 25 version 1184 is already newer than 1182 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1184 (ObjNum 961 snap 280 seq_num 961) dirty exists 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2733: left oid 25 (ObjNum 961 snap 280 seq_num 961) 2026-02-21T22:27:33.300 INFO:tasks.rados.rados.0.trial178.stdout:2731: finishing copy_from to trial17818531-37 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1093 (ObjNum 943 snap 277 seq_num 943) dirty exists 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2731: done (8 left) 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2732: done (7 left) 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2733: done (6 left) 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2734: done (5 left) 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2740: done (4 left) 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2741: read oid 8 snap -1 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2741: expect deleted 2026-02-21T22:27:33.301 INFO:tasks.rados.rados.0.trial178.stdout:2742: snap_remove snap 277 2026-02-21T22:27:33.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:33.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:33.671 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:27:33.672 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2026-02-21T22:27:33.672 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-21T22:27:33.672 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2026-02-21T22:27:33.684 INFO:tasks.ceph.mon.c:Started 2026-02-21T22:27:33.684 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:27:33.684 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 10 monclient: get_monmap_and_config 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 10 monclient: build_initial_monmap 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 10 monclient: monmap: 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:33.744239+0000 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:33.744239+0000 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:33.743 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.744 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:33.744 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:33.744 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:33.744 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 1 Processor -- start 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 1 -- start start 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.740+0000 7fdb595c8640 10 monclient: init 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-a con 0x7fdb5408cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-c con 0x7fdb5408d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-b con 0x7fdb5408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb540890e0 con 0x7fdb5408cd00 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb54088ea0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5408aea0 con 0x7fdb5408b8d0 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:33.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient(hunting): authenticate will time out at 1862.980143s 2026-02-21T22:27:33.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb52ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdb5408cd00 0x7fdb5408d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44790/0 (socket says 10.20.193.102:44790) 2026-02-21T22:27:33.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb52ffd640 1 -- 10.20.193.102:0/1941446046 learned_addr learned my addr 10.20.193.102:0/1941446046 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:33.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb52ffd640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.102:6789/0 conn(0x7fdb5408cd00 legacy=0x7fdb5408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb537fe640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.102:6790/0 conn(0x7fdb5408b8d0 legacy=0x7fdb541a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb527fc640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.178:6789/0 conn(0x7fdb5408d7e0 legacy=0x7fdb541840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3056617520 0 0) 0x7fdb54088ea0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient(hunting): my global_id is 54367 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb40003620 con 0x7fdb5408d7e0 2026-02-21T22:27:33.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2866027926 0 0) 0x7fdb40003620 con 0x7fdb5408d7e0 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.102:6790/0 conn(0x7fdb5408b8d0 legacy=0x7fdb541a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.102:6789/0 conn(0x7fdb5408cd00 legacy=0x7fdb5408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 monclient: found mon.noname-c 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:33.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:10.20.193.178:6789/0 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5408a600 con 0x7fdb5408d7e0 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: _finish_auth 0 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdb3c0031d0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: dump: 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.751 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 5 monclient: authenticate success, global_id 54367 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 <== mon.1 v1:10.20.193.178:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdb3c003460 con 0x7fdb5408d7e0 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:33.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 1 -- 10.20.193.102:0/1941446046 <== mon.1 v1:10.20.193.178:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdb3c0038c0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb51ffb640 10 monclient: dump: 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.753 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 4 set_mon_vals no callback set 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 10 monclient: shutdown 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.744+0000 7fdb595c8640 1 -- 10.20.193.102:0/1941446046 >> v1:10.20.193.178:6789/0 conn(0x7fdb5408d7e0 legacy=0x7fdb541840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/1941446046 shutdown_connections 2026-02-21T22:27:33.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/1941446046 >> 10.20.193.102:0/1941446046 conn(0x7fdb54091a50 msgr2=0x7fdb54091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:33.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/1941446046 shutdown_connections 2026-02-21T22:27:33.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/1941446046 wait complete. 2026-02-21T22:27:33.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: build_initial_monmap 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: monmap: 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:33.760 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 Processor -- start 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- start start 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: init 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-b con 0x7fdb5408b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-c con 0x7fdb5408cd00 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient(hunting): picked mon.noname-a con 0x7fdb5408d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5407fe00 con 0x7fdb5408d7e0 2026-02-21T22:27:33.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb54082160 con 0x7fdb5408b8d0 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb540818c0 con 0x7fdb5408cd00 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient(hunting): authenticate will time out at 1862.984959s 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb527fc640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fdb5408cd00 0x7fdb54083690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50966/0 (socket says 10.20.193.102:50966) 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb527fc640 1 -- 10.20.193.102:0/3437890809 learned_addr learned my addr 10.20.193.102:0/3437890809 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb537fe640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.102:6789/0 conn(0x7fdb5408d7e0 legacy=0x7fdb54195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb527fc640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.102:6790/0 conn(0x7fdb5408cd00 legacy=0x7fdb54083690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb52ffd640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.178:6789/0 conn(0x7fdb5408b8d0 legacy=0x7fdb541531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033704761 0 0) 0x7fdb5407fe00 con 0x7fdb5408d7e0 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): my global_id is 34572 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb20003620 con 0x7fdb5408d7e0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450029389 0 0) 0x7fdb54082160 con 0x7fdb5408b8d0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): my global_id is 54370 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb5407fe00 con 0x7fdb5408b8d0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1458660345 0 0) 0x7fdb20003620 con 0x7fdb5408d7e0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdb54082160 con 0x7fdb5408d7e0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdb44004890 con 0x7fdb5408d7e0 2026-02-21T22:27:33.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): dump: 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.764 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2166202946 0 0) 0x7fdb54082160 con 0x7fdb5408d7e0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.102:6790/0 conn(0x7fdb5408cd00 legacy=0x7fdb54083690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.178:6789/0 conn(0x7fdb5408b8d0 legacy=0x7fdb541531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient: _finish_hunting 0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 monclient: found mon.a 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb54080960 con 0x7fdb5408d7e0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient: _finish_auth 0 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 10 monclient: _check_auth_tickets 2026-02-21T22:27:33.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 5 monclient: authenticate success, global_id 34572 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _renew_subs 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdb54083da0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _renew_subs 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.748+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdb54196e70 con 0x7fdb5408d7e0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdb44004ed0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdb44005260 con 0x7fdb5408d7e0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: dump: 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb517fa640 4 set_mon_vals no callback set 2026-02-21T22:27:33.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb595c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb595c8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb595c8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb540889f0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdb44005530 con 0x7fdb5408d7e0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(581..581 src has 395..581) ==== 3959+0+0 (unknown 2389675722 0 0) 0x7fdb4404da80 con 0x7fdb5408d7e0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb52ffd640 10 monclient: get_auth_request con 0x7fdb2003d560 auth_method 0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fdb44018a00 con 0x7fdb5408d7e0 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:33.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.752+0000 7fdb337fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:33.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:27:33.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:33.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdb54062ce0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb337fe640 1 -- 10.20.193.102:0/3437890809 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 14477134) 0x7fdb4401dcc0 con 0x7fdb5408d7e0 2026-02-21T22:27:33.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb337fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:33.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb337fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:33.869 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:27:33.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.178:6800/2549949439 conn(0x7fdb2003d560 legacy=0x7fdb2005da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 10 monclient: shutdown 2026-02-21T22:27:33.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:33.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 >> v1:10.20.193.102:6789/0 conn(0x7fdb5408d7e0 legacy=0x7fdb54195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 shutdown_connections 2026-02-21T22:27:33.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 >> 10.20.193.102:0/3437890809 conn(0x7fdb54091a50 msgr2=0x7fdb54085740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:33.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 shutdown_connections 2026-02-21T22:27:33.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.868+0000 7fdb595c8640 1 -- 10.20.193.102:0/3437890809 wait complete. 2026-02-21T22:27:33.879 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:27:33.879 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:27:33.880 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:27:33.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: get_monmap_and_config 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: build_initial_monmap 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: monmap: 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:33.951163+0000 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:33.951163+0000 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.950 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:33.951 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:33.951 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:33.951 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 1 Processor -- start 2026-02-21T22:27:33.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 1 -- start start 2026-02-21T22:27:33.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: init 2026-02-21T22:27:33.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-a con 0x7ff5a008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-b con 0x7ff5a008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a0086a10 con 0x7ff5a008ce00 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a008b130 con 0x7ff5a008d8e0 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a008bd60 con 0x7ff5a008b7f0 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a8539640 10 monclient(hunting): authenticate will time out at 1863.187143s 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a62ae640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff5a008ce00 0x7ff5a008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44796/0 (socket says 10.20.193.102:44796) 2026-02-21T22:27:33.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.948+0000 7ff5a62ae640 1 -- 10.20.193.102:0/725431758 learned_addr learned my addr 10.20.193.102:0/725431758 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:33.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a62ae640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.102:6789/0 conn(0x7ff5a008ce00 legacy=0x7ff5a008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a5aad640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.102:6790/0 conn(0x7ff5a008b7f0 legacy=0x7ff5a0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a6aaf640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.178:6789/0 conn(0x7ff5a008d8e0 legacy=0x7ff5a01a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670086754 0 0) 0x7ff5a0086a10 con 0x7ff5a008ce00 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): my global_id is 34575 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff588003620 con 0x7ff5a008ce00 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2003322761 0 0) 0x7ff5a008b130 con 0x7ff5a008d8e0 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient(hunting): my global_id is 54373 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a0086a10 con 0x7ff5a008d8e0 2026-02-21T22:27:33.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3051016483 0 0) 0x7ff588003620 con 0x7ff5a008ce00 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.102:6790/0 conn(0x7ff5a008b7f0 legacy=0x7ff5a0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.178:6789/0 conn(0x7ff5a008d8e0 legacy=0x7ff5a01a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: _finish_hunting 0 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 monclient: found mon.noname-a 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a008b370 con 0x7ff5a008ce00 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: _finish_auth 0 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: _check_auth_tickets 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff590003110 con 0x7ff5a008ce00 2026-02-21T22:27:33.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: dump: 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 5 monclient: authenticate success, global_id 34575 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff5900033e0 con 0x7ff5a008ce00 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 1 -- 10.20.193.102:0/725431758 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff590004b50 con 0x7ff5a008ce00 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a52ac640 10 monclient: dump: 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.961 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 4 set_mon_vals no callback set 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 10 monclient: shutdown 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- 10.20.193.102:0/725431758 >> v1:10.20.193.102:6789/0 conn(0x7ff5a008ce00 legacy=0x7ff5a008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- 10.20.193.102:0/725431758 shutdown_connections 2026-02-21T22:27:33.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- 10.20.193.102:0/725431758 >> 10.20.193.102:0/725431758 conn(0x7ff5a0091a50 msgr2=0x7ff5a0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- 10.20.193.102:0/725431758 shutdown_connections 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- 10.20.193.102:0/725431758 wait complete. 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 10 monclient: build_initial_monmap 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 10 monclient: monmap: 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:33.963 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 Processor -- start 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 1 -- start start 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.952+0000 7ff5a8539640 10 monclient: init 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-b con 0x7ff5a008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-c con 0x7ff5a008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient(hunting): picked mon.noname-a con 0x7ff5a008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a008a870 con 0x7ff5a008d8e0 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a008a250 con 0x7ff5a008b7f0 2026-02-21T22:27:33.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a005e230 con 0x7ff5a008ce00 2026-02-21T22:27:33.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:33.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient(hunting): authenticate will time out at 1863.192272s 2026-02-21T22:27:33.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a6aaf640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff5a008d8e0 0x7ff5a0195f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44804/0 (socket says 10.20.193.102:44804) 2026-02-21T22:27:33.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a6aaf640 1 -- 10.20.193.102:0/1027558155 learned_addr learned my addr 10.20.193.102:0/1027558155 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:33.965 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a5aad640 1 --1- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6790/0 conn(0x7ff5a008ce00 0x7ff5a0194870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50986/0 (socket says 10.20.193.102:50986) 2026-02-21T22:27:33.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a5aad640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6790/0 conn(0x7ff5a008ce00 legacy=0x7ff5a0194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a6aaf640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a008d8e0 legacy=0x7ff5a0195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323532455 0 0) 0x7ff5a005e230 con 0x7ff5a008ce00 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): my global_id is 94100 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a62ae640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.178:6789/0 conn(0x7ff5a008b7f0 legacy=0x7ff5a0084b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff570003620 con 0x7ff5a008ce00 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128608727 0 0) 0x7ff5a008a870 con 0x7ff5a008d8e0 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): my global_id is 34578 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a005e230 con 0x7ff5a008d8e0 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624699070 0 0) 0x7ff5a008a250 con 0x7ff5a008b7f0 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): my global_id is 54376 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a008a870 con 0x7ff5a008b7f0 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2259970356 0 0) 0x7ff570003620 con 0x7ff5a008ce00 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff5a008a250 con 0x7ff5a008ce00 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff5940031e0 con 0x7ff5a008ce00 2026-02-21T22:27:33.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): dump: 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.973 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3143173624 0 0) 0x7ff5a005e230 con 0x7ff5a008d8e0 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff570003620 con 0x7ff5a008d8e0 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff59c0049a0 con 0x7ff5a008d8e0 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient(hunting): dump: 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.974 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3246772368 0 0) 0x7ff5a008a250 con 0x7ff5a008ce00 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.178:6789/0 conn(0x7ff5a008b7f0 legacy=0x7ff5a0084b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a008d8e0 legacy=0x7ff5a0195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 monclient: found mon.c 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:33.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a00835e0 con 0x7ff5a008ce00 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: _finish_auth 0 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 5 monclient: authenticate success, global_id 94100 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _renew_subs 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a0082d40 con 0x7ff5a008ce00 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _renew_subs 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff5a0081de0 con 0x7ff5a008ce00 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:33.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff5a01538e0 con 0x7ff5a0153510 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff5940039c0 con 0x7ff5a008ce00 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff594003d50 con 0x7ff5a008ce00 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 10 monclient: dump: 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.977 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff5a4aab640 4 set_mon_vals no callback set 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff594003fa0 con 0x7ff5a008ce00 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.956+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(581..581 src has 395..581) ==== 3959+0+0 (unknown 2389675722 0 0) 0x7ff59404c5e0 con 0x7ff5a008ce00 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff5a6aaf640 10 monclient: get_auth_request con 0x7ff57003d850 auth_method 0 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff5a62ae640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a0153510 legacy=0x7ff5a005ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918021572 0 0) 0x7ff5a01538e0 con 0x7ff5a0153510 2026-02-21T22:27:33.978 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient(hunting): my global_id is 34581 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a008a250 con 0x7ff5a0153510 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 831168063 0 0) 0x7ff5a008a250 con 0x7ff5a0153510 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff5a01538e0 con 0x7ff5a0153510 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff590004a90 con 0x7ff5a0153510 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient: dump: 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.979 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1533027161 0 0) 0x7ff5a01538e0 con 0x7ff5a0153510 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5a00820d0 con 0x7ff5a0153510 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7ff590002db0 con 0x7ff5a0153510 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:33.980 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a0153510 legacy=0x7ff5a005ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.960+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a0153510 legacy=0x7ff5a005ba30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.964+0000 7ff5a8539640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.964+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff5a0062ce0 con 0x7ff5a005c140 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a62ae640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a005c140 legacy=0x7ff5a01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 73376681 0 0) 0x7ff5a0062ce0 con 0x7ff5a005c140 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient(hunting): my global_id is 34584 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a01538e0 con 0x7ff5a005c140 2026-02-21T22:27:33.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3835603237 0 0) 0x7ff5a01538e0 con 0x7ff5a005c140 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff5a0062ce0 con 0x7ff5a005c140 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff5900051b0 con 0x7ff5a005c140 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient: dump: 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:33.982 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3564104621 0 0) 0x7ff5a0062ce0 con 0x7ff5a005c140 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5a0090810 con 0x7ff5a005c140 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 2364427679) 0x7ff5900073b0 con 0x7ff5a005c140 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a005c140 legacy=0x7ff5a01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff586ffd640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6789/0 conn(0x7ff5a005c140 legacy=0x7ff5a01986c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 602837, 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:33.984 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:33.985 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:33.986 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:33.987 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:33.988 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.989 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:33.990 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.991 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.992 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:33.993 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:33.994 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:33.995 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff57003d850 legacy=0x7ff57005dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 10 monclient: shutdown 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 >> v1:10.20.193.102:6790/0 conn(0x7ff5a008ce00 legacy=0x7ff5a0194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a6aaf640 1 -- 10.20.193.102:0/1027558155 reap_dead start 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a6aaf640 1 -- 10.20.193.102:0/1027558155 reap_dead start 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 shutdown_connections 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 >> 10.20.193.102:0/1027558155 conn(0x7ff5a0091a50 msgr2=0x7ff5a00575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:33.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 shutdown_connections 2026-02-21T22:27:33.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:33.968+0000 7ff5a8539640 1 -- 10.20.193.102:0/1027558155 wait complete. 2026-02-21T22:27:33.997 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:33.997 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:27:33.997 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:27:34.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 10 monclient: get_monmap_and_config 2026-02-21T22:27:34.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 10 monclient: monmap: 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:34.067877+0000 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:34.067877+0000 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:34.067 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 1 Processor -- start 2026-02-21T22:27:34.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 1 -- start start 2026-02-21T22:27:34.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.064+0000 7f3d8808e640 10 monclient: init 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-c con 0x7f3d800584c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-b con 0x7f3d800588a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-a con 0x7f3d801435f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d800902a0 con 0x7f3d801435f0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d800921d0 con 0x7f3d800584c0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d800574a0 con 0x7f3d800588a0 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient(hunting): authenticate will time out at 1863.304202s 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d85602640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f3d800588a0 0x7f3d80183d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:50990/0 (socket says 10.20.193.102:50990) 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d85602640 1 -- 10.20.193.102:0/3547377935 learned_addr learned my addr 10.20.193.102:0/3547377935 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d86604640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 0x7f3d801a54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44844/0 (socket says 10.20.193.102:44844) 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d85602640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.102:6790/0 conn(0x7f3d800588a0 legacy=0x7f3d80183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d86604640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 legacy=0x7f3d801a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d85e03640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.178:6789/0 conn(0x7f3d800584c0 legacy=0x7f3d80142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398289184 0 0) 0x7f3d800574a0 con 0x7f3d800588a0 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): my global_id is 94103 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d68003620 con 0x7f3d800588a0 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423215040 0 0) 0x7f3d7c004170 con 0x7f3d801435f0 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): my global_id is 34587 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d800574a0 con 0x7f3d801435f0 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591302784 0 0) 0x7f3d800921d0 con 0x7f3d800584c0 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient(hunting): my global_id is 54379 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d7c004170 con 0x7f3d800584c0 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1162833370 0 0) 0x7f3d68003620 con 0x7f3d800588a0 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.178:6789/0 conn(0x7f3d800584c0 legacy=0x7f3d80142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 legacy=0x7f3d801a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 monclient: found mon.noname-b 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.074 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d80056640 con 0x7f3d800588a0 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d74003110 con 0x7f3d800588a0 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: dump: 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.075 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 5 monclient: authenticate success, global_id 94103 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d740033e0 con 0x7f3d800588a0 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 1 -- 10.20.193.102:0/3547377935 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d74004b50 con 0x7f3d800588a0 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d84e01640 10 monclient: dump: 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.076 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 4 set_mon_vals no callback set 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 10 monclient: shutdown 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- 10.20.193.102:0/3547377935 >> v1:10.20.193.102:6790/0 conn(0x7f3d800588a0 legacy=0x7f3d80183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- 10.20.193.102:0/3547377935 shutdown_connections 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- 10.20.193.102:0/3547377935 >> 10.20.193.102:0/3547377935 conn(0x7f3d8005e700 msgr2=0x7f3d80095ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- 10.20.193.102:0/3547377935 shutdown_connections 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.068+0000 7f3d8808e640 1 -- 10.20.193.102:0/3547377935 wait complete. 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: monmap: 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 Processor -- start 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- start start 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: init 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-b con 0x7f3d800588a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-a con 0x7f3d801435f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient(hunting): picked mon.noname-c con 0x7f3d80140310 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d80131320 con 0x7f3d801435f0 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d80131600 con 0x7f3d800588a0 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8012f2a0 con 0x7f3d80140310 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient(hunting): authenticate will time out at 1863.308840s 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d85602640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 0x7f3d8013fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44852/0 (socket says 10.20.193.102:44852) 2026-02-21T22:27:34.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d85602640 1 -- 10.20.193.102:0/2601822678 learned_addr learned my addr 10.20.193.102:0/2601822678 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d85602640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 legacy=0x7f3d8013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d86604640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d80140310 legacy=0x7f3d80195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d85e03640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.178:6789/0 conn(0x7f3d800588a0 legacy=0x7f3d80152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205526587 0 0) 0x7f3d80131320 con 0x7f3d801435f0 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): my global_id is 34590 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d50003620 con 0x7f3d801435f0 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120640878 0 0) 0x7f3d8012f2a0 con 0x7f3d80140310 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): my global_id is 94106 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d80131320 con 0x7f3d80140310 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577381519 0 0) 0x7f3d80131600 con 0x7f3d800588a0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): my global_id is 54382 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d8012f2a0 con 0x7f3d800588a0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1371091532 0 0) 0x7f3d50003620 con 0x7f3d801435f0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d80131600 con 0x7f3d801435f0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d740048e0 con 0x7f3d801435f0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): dump: 2026-02-21T22:27:34.084 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2343037539 0 0) 0x7f3d80131320 con 0x7f3d80140310 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d50003620 con 0x7f3d80140310 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d7c005fa0 con 0x7f3d80140310 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:34.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient(hunting): dump: 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 765617169 0 0) 0x7f3d80131600 con 0x7f3d801435f0 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d80140310 legacy=0x7f3d80195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.178:6789/0 conn(0x7f3d800588a0 legacy=0x7f3d80152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 monclient: found mon.a 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8012fe00 con 0x7f3d801435f0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 5 monclient: authenticate success, global_id 34590 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _renew_subs 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3d80196a50 con 0x7f3d801435f0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _renew_subs 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3d80196f60 con 0x7f3d801435f0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d74004ac0 con 0x7f3d801435f0 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d74005170 con 0x7f3d801435f0 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6ffff640 4 set_mon_vals no callback set 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.072+0000 7f3d6e7fc640 10 monclient: dump: 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d8808e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f3d8008fbe0 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f3d74011800 con 0x7f3d801435f0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(581..581 src has 395..581) ==== 3959+0+0 (unknown 2389675722 0 0) 0x7f3d7404c690 con 0x7f3d801435f0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d86604640 10 monclient: get_auth_request con 0x7f3d5003d800 auth_method 0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d85e03640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d800584c0 legacy=0x7f3d8008bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367856703 0 0) 0x7f3d8008fbe0 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient(hunting): my global_id is 94109 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d80131600 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2951424952 0 0) 0x7f3d80131600 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d8008fbe0 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d70003390 con 0x7f3d800584c0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient: dump: 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2386970336 0 0) 0x7f3d8008fbe0 con 0x7f3d800584c0 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d80059020 con 0x7f3d800584c0 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f3d700058c0 con 0x7f3d800584c0 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:34.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d800584c0 legacy=0x7f3d8008bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.076+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d800584c0 legacy=0x7f3d8008bd00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:34.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.080+0000 7f3d8808e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:34.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.080+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f3d8005eb70 con 0x7f3d80094c10 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d85e03640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d80094c10 legacy=0x7f3d8004d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569210716 0 0) 0x7f3d8005eb70 con 0x7f3d80094c10 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient(hunting): my global_id is 94112 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d8008fbe0 con 0x7f3d80094c10 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1513587704 0 0) 0x7f3d8008fbe0 con 0x7f3d80094c10 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d8005eb70 con 0x7f3d80094c10 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3d70003b70 con 0x7f3d80094c10 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient: dump: 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.093 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 207331548 0 0) 0x7f3d8005eb70 con 0x7f3d80094c10 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3d8004d990 con 0x7f3d80094c10 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4344 (unknown 0 0 654128442) 0x7f3d70005af0 con 0x7f3d80094c10 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d80094c10 legacy=0x7f3d8004d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d6e7fc640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6790/0 conn(0x7f3d80094c10 legacy=0x7f3d8004d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:34.094 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 359, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:34.095 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:34.096 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.097 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:34.098 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.099 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:34.100 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.101 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.102 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.103 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:34.104 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.178:6800/2549949439 conn(0x7f3d5003d800 legacy=0x7f3d5005dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 10 monclient: shutdown 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 >> v1:10.20.193.102:6789/0 conn(0x7f3d801435f0 legacy=0x7f3d8013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d86604640 1 -- 10.20.193.102:0/2601822678 reap_dead start 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d86604640 1 -- 10.20.193.102:0/2601822678 reap_dead start 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 shutdown_connections 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 >> 10.20.193.102:0/2601822678 conn(0x7f3d8005e700 msgr2=0x7f3d80092f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 shutdown_connections 2026-02-21T22:27:34.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.084+0000 7f3d8808e640 1 -- 10.20.193.102:0/2601822678 wait complete. 2026-02-21T22:27:34.106 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:34.106 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:27:34.106 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:27:34.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: get_monmap_and_config 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: monmap: 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:34.179243+0000 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:34.179243+0000 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:34.178 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 1 Processor -- start 2026-02-21T22:27:34.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 1 -- start start 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: init 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient(hunting): picked mon.noname-c con 0x7f497c091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient(hunting): picked mon.noname-b con 0x7f497c0909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient(hunting): picked mon.noname-a con 0x7f497c1854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c057a20 con 0x7f497c1854b0 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c091d20 con 0x7f497c091240 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c057740 con 0x7f497c0909a0 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f4981370640 10 monclient(hunting): authenticate will time out at 1863.415816s 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f497b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f497c1854b0 0x7f497c1a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44862/0 (socket says 10.20.193.102:44862) 2026-02-21T22:27:34.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.176+0000 7f497b7fe640 1 -- 10.20.193.102:0/2970229989 learned_addr learned my addr 10.20.193.102:0/2970229989 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.181 INFO:tasks.rados.rados.0.trial178.stdout:2742: done (5 left) 2026-02-21T22:27:34.181 INFO:tasks.rados.rados.0.trial178.stdout:2743: rollback oid 5 current snap is 280 2026-02-21T22:27:34.181 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 276 2026-02-21T22:27:34.181 INFO:tasks.rados.rados.0.trial178.stdout:2737: expect (ObjNum 940 snap 276 seq_num 940) 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f497b7fe640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.102:6789/0 conn(0x7f497c1854b0 legacy=0x7f497c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f497a7fc640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.102:6790/0 conn(0x7f497c0909a0 legacy=0x7f497c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078748644 0 0) 0x7f497c057a20 con 0x7f497c1854b0 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.183 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient(hunting): my global_id is 34593 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4968003620 con 0x7f497c1854b0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f497affd640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.178:6789/0 conn(0x7f497c091240 legacy=0x7f497c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1290451951 0 0) 0x7f4968003620 con 0x7f497c1854b0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.102:6790/0 conn(0x7f497c0909a0 legacy=0x7f497c183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.178:6789/0 conn(0x7f497c091240 legacy=0x7f497c091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 monclient: found mon.noname-a 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c090dc0 con 0x7f497c1854b0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 5 monclient: authenticate success, global_id 34593 2026-02-21T22:27:34.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f496c004620 con 0x7f497c1854b0 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: dump: 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f496c0048f0 con 0x7f497c1854b0 2026-02-21T22:27:34.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 1 -- 10.20.193.102:0/2970229989 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f496c006060 con 0x7f497c1854b0 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 4 set_mon_vals no callback set 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4979ffb640 10 monclient: dump: 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 10 monclient: shutdown 2026-02-21T22:27:34.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 1 -- 10.20.193.102:0/2970229989 >> v1:10.20.193.102:6789/0 conn(0x7f497c1854b0 legacy=0x7f497c1a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 1 -- 10.20.193.102:0/2970229989 shutdown_connections 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 1 -- 10.20.193.102:0/2970229989 >> 10.20.193.102:0/2970229989 conn(0x7f497c05e660 msgr2=0x7f497c095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 1 -- 10.20.193.102:0/2970229989 shutdown_connections 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 1 -- 10.20.193.102:0/2970229989 wait complete. 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.180+0000 7f4981370640 10 monclient: monmap: 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:34.187 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 Processor -- start 2026-02-21T22:27:34.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- start start 2026-02-21T22:27:34.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: init 2026-02-21T22:27:34.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:34.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient(hunting): picked mon.noname-c con 0x7f497c0909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient(hunting): picked mon.noname-b con 0x7f497c091240 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient(hunting): picked mon.noname-a con 0x7f497c1854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0895d0 con 0x7f497c1854b0 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c112440 con 0x7f497c091240 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c12c060 con 0x7f497c0909a0 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient(hunting): authenticate will time out at 1863.420529s 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f497affd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f497c0909a0 0x7f497c12b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51034/0 (socket says 10.20.193.102:51034) 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f497affd640 1 -- 10.20.193.102:0/543786865 learned_addr learned my addr 10.20.193.102:0/543786865 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f497affd640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.102:6790/0 conn(0x7f497c0909a0 legacy=0x7f497c12b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f497b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.102:6789/0 conn(0x7f497c1854b0 legacy=0x7f497c174430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1723586547 0 0) 0x7f497c0895d0 con 0x7f497c1854b0 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): my global_id is 34596 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4948003620 con 0x7f497c1854b0 2026-02-21T22:27:34.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f497a7fc640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c091240 legacy=0x7f497c172d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1140734794 0 0) 0x7f4948003620 con 0x7f497c1854b0 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f497c0895d0 con 0x7f497c1854b0 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f496c005df0 con 0x7f497c1854b0 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): dump: 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656756828 0 0) 0x7f497c112440 con 0x7f497c091240 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient(hunting): my global_id is 54385 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4948003620 con 0x7f497c091240 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2507496046 0 0) 0x7f497c0895d0 con 0x7f497c1854b0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.102:6790/0 conn(0x7f497c0909a0 legacy=0x7f497c12b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c091240 legacy=0x7f497c172d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 monclient: found mon.a 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c12f360 con 0x7f497c1854b0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.194 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 5 monclient: authenticate success, global_id 34596 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _renew_subs 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f497c112a10 con 0x7f497c1854b0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _renew_subs 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f497c111ab0 con 0x7f497c1854b0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f496c006010 con 0x7f497c1854b0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f496c006680 con 0x7f497c1854b0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f495b7fe640 10 monclient: dump: 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.195 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f49797fa640 4 set_mon_vals no callback set 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.184+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f497c12ecc0 con 0x7f497c12e8f0 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f496c012d10 con 0x7f497c1854b0 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(582..582 src has 395..582) ==== 3975+0+0 (unknown 2513902031 0 0) 0x7f496c04daa0 con 0x7f497c1854b0 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f497a7fc640 10 monclient: get_auth_request con 0x7f494803d590 auth_method 0 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f497affd640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c12e8f0 legacy=0x7f497c08cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254447061 0 0) 0x7f497c12ecc0 con 0x7f497c12e8f0 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient(hunting): my global_id is 54388 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f497c0895d0 con 0x7f497c12e8f0 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1581468921 0 0) 0x7f497c0895d0 con 0x7f497c12e8f0 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f497c12ecc0 con 0x7f497c12e8f0 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4970003270 con 0x7f497c12e8f0 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient: dump: 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.197 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3517092249 0 0) 0x7f497c12ecc0 con 0x7f497c12e8f0 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f497c110f50 con 0x7f497c12e8f0 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f49700058c0 con 0x7f497c12e8f0 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c12e8f0 legacy=0x7f497c08cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.188+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c12e8f0 legacy=0x7f497c08cbd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.192+0000 7f4981370640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.192+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f497c111df0 con 0x7f497c08d2e0 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f497affd640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c08d2e0 legacy=0x7f497c176b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493525305 0 0) 0x7f497c111df0 con 0x7f497c08d2e0 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.199 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient(hunting): my global_id is 54391 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f497c12ecc0 con 0x7f497c08d2e0 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 14369307 0 0) 0x7f497c12ecc0 con 0x7f497c08d2e0 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f497c111df0 con 0x7f497c08d2e0 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4970003af0 con 0x7f497c08d2e0 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient: dump: 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.200 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 520429731 0 0) 0x7f497c111df0 con 0x7f497c08d2e0 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f497c05eb70 con 0x7f497c08d2e0 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 4289910592) 0x7f4970005b40 con 0x7f497c08d2e0 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c08d2e0 legacy=0x7f497c176b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f495b7fe640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6789/0 conn(0x7f497c08d2e0 legacy=0x7f497c176b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:34.201 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 357842, 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.202 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.203 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:34.204 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:34.205 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:34.206 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.207 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:34.208 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:34.209 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:34.210 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:27:34.211 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:34.211 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:34.211 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:34.211 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:34.211 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.178:6800/2549949439 conn(0x7f494803d590 legacy=0x7f494805da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 10 monclient: shutdown 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 >> v1:10.20.193.102:6789/0 conn(0x7f497c1854b0 legacy=0x7f497c174430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f497b7fe640 1 -- 10.20.193.102:0/543786865 reap_dead start 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f497b7fe640 1 -- 10.20.193.102:0/543786865 reap_dead start 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 shutdown_connections 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 >> 10.20.193.102:0/543786865 conn(0x7f497c05e660 msgr2=0x7f497c08bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 shutdown_connections 2026-02-21T22:27:34.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.196+0000 7f4981370640 1 -- 10.20.193.102:0/543786865 wait complete. 2026-02-21T22:27:34.213 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:34.213 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:27:34.213 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:27:34.252 INFO:tasks.rados.rados.0.trial178.stdout:2736: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:34.252 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: get_monmap_and_config 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: monmap: 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:34.275355+0000 2026-02-21T22:27:34.274 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:34.275355+0000 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:34.275 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 1 Processor -- start 2026-02-21T22:27:34.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 1 -- start start 2026-02-21T22:27:34.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: init 2026-02-21T22:27:34.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f7c9c13f9c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f7c9c13fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-b con 0x7f7c9c1401f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c092490 con 0x7f7c9c13fd90 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0571f0 con 0x7f7c9c13f9c0 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0574d0 con 0x7f7c9c1401f0 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9bfff640 10 monclient(hunting): authenticate will time out at 1863.511530s 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9a7fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 0x7f7c9c183d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44894/0 (socket says 10.20.193.102:44894) 2026-02-21T22:27:34.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.272+0000 7f7c9a7fc640 1 -- 10.20.193.102:0/2263081144 learned_addr learned my addr 10.20.193.102:0/2263081144 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9a7fc640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 legacy=0x7f7c9c183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9b7fe640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.102:6790/0 conn(0x7f7c9c1401f0 legacy=0x7f7c9c1a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9affd640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.178:6789/0 conn(0x7f7c9c13f9c0 legacy=0x7f7c9c0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023993649 0 0) 0x7f7c9c092490 con 0x7f7c9c13fd90 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): my global_id is 34599 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c88003620 con 0x7f7c9c13fd90 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638523201 0 0) 0x7f7c9c0571f0 con 0x7f7c9c13f9c0 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient(hunting): my global_id is 54394 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c9c092490 con 0x7f7c9c13f9c0 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1520011171 0 0) 0x7f7c88003620 con 0x7f7c9c13fd90 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.102:6790/0 conn(0x7f7c9c1401f0 legacy=0x7f7c9c1a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.178:6789/0 conn(0x7f7c9c13f9c0 legacy=0x7f7c9c0906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 monclient: found mon.noname-a 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0577b0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7c84003110 con 0x7f7c9c13fd90 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: dump: 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.283 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 5 monclient: authenticate success, global_id 34599 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7c840033e0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 1 -- 10.20.193.102:0/2263081144 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7c84004b50 con 0x7f7c9c13fd90 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c99ffb640 10 monclient: dump: 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.284 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 4 set_mon_vals no callback set 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 10 monclient: shutdown 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- 10.20.193.102:0/2263081144 >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 legacy=0x7f7c9c183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- 10.20.193.102:0/2263081144 shutdown_connections 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- 10.20.193.102:0/2263081144 >> 10.20.193.102:0/2263081144 conn(0x7f7c9c05e3f0 msgr2=0x7f7c9c095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- 10.20.193.102:0/2263081144 shutdown_connections 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- 10.20.193.102:0/2263081144 wait complete. 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 10 monclient: build_initial_monmap 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 10 monclient: monmap: 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:34.286 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 Processor -- start 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 1 -- start start 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.276+0000 7f7c9bfff640 10 monclient: init 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-c con 0x7f7c9c13f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f7c9c13fd90 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient(hunting): picked mon.noname-b con 0x7f7c9c1401f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c133260 con 0x7f7c9c13fd90 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c133540 con 0x7f7c9c1401f0 2026-02-21T22:27:34.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c1324b0 con 0x7f7c9c13f9c0 2026-02-21T22:27:34.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:34.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient(hunting): authenticate will time out at 1863.516173s 2026-02-21T22:27:34.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9a7fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 0x7f7c9c193fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44904/0 (socket says 10.20.193.102:44904) 2026-02-21T22:27:34.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9affd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7c9c13f9c0 0x7f7c9c13eb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51066/0 (socket says 10.20.193.102:51066) 2026-02-21T22:27:34.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9a7fc640 1 -- 10.20.193.102:0/294082765 learned_addr learned my addr 10.20.193.102:0/294082765 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:34.289 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:34.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9a7fc640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 legacy=0x7f7c9c193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9affd640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.102:6790/0 conn(0x7f7c9c13f9c0 legacy=0x7f7c9c13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 631370417 0 0) 0x7f7c9c133260 con 0x7f7c9c13fd90 2026-02-21T22:27:34.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): my global_id is 34602 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c68003620 con 0x7f7c9c13fd90 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1688447994 0 0) 0x7f7c68003620 con 0x7f7c9c13fd90 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7c9c133260 con 0x7f7c9c13fd90 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7c84002b70 con 0x7f7c9c13fd90 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): dump: 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.292 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9b7fe640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.178:6789/0 conn(0x7f7c9c1401f0 legacy=0x7f7c9c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2490847886 0 0) 0x7f7c9c133260 con 0x7f7c9c13fd90 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.102:6790/0 conn(0x7f7c9c13f9c0 legacy=0x7f7c9c13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.178:6789/0 conn(0x7f7c9c1401f0 legacy=0x7f7c9c1956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:27:34.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 monclient: found mon.a 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c1311e0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 5 monclient: authenticate success, global_id 34602 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _renew_subs 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c131d40 con 0x7f7c9c13fd90 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _renew_subs 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7c9c13f3f0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7c840033c0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7c84003570 con 0x7f7c9c13fd90 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 10 monclient: dump: 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:34.295 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c997fa640 4 set_mon_vals no callback set 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0008e0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f7c84011810 con 0x7f7c9c13fd90 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.280+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(582..582 src has 395..582) ==== 3975+0+0 (unknown 2513902031 0 0) 0x7f7c8404c720 con 0x7f7c9c13fd90 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.284+0000 7f7c9affd640 10 monclient: get_auth_request con 0x7f7c6803d450 auth_method 0 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.284+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f7c840176f0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.284+0000 7f7c7b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:34.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.284+0000 7f7c7b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2738: finishing write tid 1 to trial17818531-45 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2738: finishing write tid 2 to trial17818531-45 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2738: finishing write tid 3 to trial17818531-45 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2736: done (5 left) 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2737: done (4 left) 2026-02-21T22:27:34.353 INFO:tasks.rados.rados.0.trial178.stdout:2744: write (excl) oid 36 current snap is 280 2026-02-21T22:27:34.354 INFO:tasks.rados.rados.0.trial178.stdout:2744: seq_num 964 ranges {601948=400514,1577818=624130} 2026-02-21T22:27:34.360 INFO:tasks.rados.rados.0.trial178.stdout:2744: writing trial17818531-36 from 601948 to 1002462 tid 1 2026-02-21T22:27:34.368 INFO:tasks.rados.rados.0.trial178.stdout:2744: writing trial17818531-36 from 1577818 to 2201948 tid 2 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2745: read oid 47 snap -1 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2745: expect (ObjNum 881 snap 257 seq_num 881) 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2746: copy_from oid 33 from oid 34 current snap is 280 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2738: finishing write tid 4 to trial17818531-45 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2738: oid 45 updating version 0 to 1311 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2738: oid 45 version 1311 is already newer than 1310 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2738: oid 45 version 1311 is already newer than 1309 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1311 (ObjNum 962 snap 280 seq_num 962) dirty exists 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2738: left oid 45 (ObjNum 962 snap 280 seq_num 962) 2026-02-21T22:27:34.369 INFO:tasks.rados.rados.0.trial178.stdout:2739: expect (ObjNum 746 snap 209 seq_num 746) 2026-02-21T22:27:34.394 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:34.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.404+0000 7f7c9bfff640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:27:34.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.404+0000 7f7c9bfff640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:27:34.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.404+0000 7f7c9bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:34.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.404+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7c9c1a6da0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c7b7fe640 1 -- 10.20.193.102:0/294082765 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f7c84016fd0 con 0x7f7c9c13fd90 2026-02-21T22:27:34.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c7b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:27:34.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c7b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:34.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=2316650458,config=1618278320,health=2038788593,kv=514132712,logm=884971829}) 2026-02-21T22:27:34.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=17,config=2,health=13,kv=10,logm=58} crc {auth=3538047944,config=1618278320,health=547011000,kv=2092583389,logm=211665702}) 2026-02-21T22:27:34.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=15,config=2,health=12,kv=12,logm=59} crc {auth=2316650458,config=1618278320,health=2038788593,kv=514132712,logm=884971829}) 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=17,config=2,health=12,kv=12,logm=57} crc {auth=1781975772,config=1618278320,health=118182103,kv=514132713,logm=2060604946}) 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628390659}) 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1535948919}) 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628390659}) 2026-02-21T22:27:34.410 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1723398671}) 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.178:6800/2549949439 conn(0x7f7c6803d450 legacy=0x7f7c6805d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 10 monclient: shutdown 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 >> v1:10.20.193.102:6789/0 conn(0x7f7c9c13fd90 legacy=0x7f7c9c193fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 shutdown_connections 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 >> 10.20.193.102:0/294082765 conn(0x7f7c9c05e3f0 msgr2=0x7f7c9c08bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:34.411 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 shutdown_connections 2026-02-21T22:27:34.412 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:34.408+0000 7f7c9bfff640 1 -- 10.20.193.102:0/294082765 wait complete. 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2743: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2743: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2743: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1437 (ObjNum 908 snap 268 seq_num 908) dirty exists 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: finishing write tid 1 to trial17818531-36 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: finishing write tid 2 to trial17818531-36 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: finishing write tid 3 to trial17818531-36 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: finishing write tid 4 to trial17818531-36 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: oid 36 updating version 0 to 1096 2026-02-21T22:27:34.414 INFO:tasks.rados.rados.0.trial178.stdout:2744: oid 36 version 1096 is already newer than 1095 2026-02-21T22:27:34.415 INFO:tasks.rados.rados.0.trial178.stdout:2744: oid 36 version 1096 is already newer than 1094 2026-02-21T22:27:34.415 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1096 (ObjNum 964 snap 280 seq_num 964) dirty exists 2026-02-21T22:27:34.415 INFO:tasks.rados.rados.0.trial178.stdout:2744: left oid 36 (ObjNum 964 snap 280 seq_num 964) 2026-02-21T22:27:34.415 INFO:tasks.rados.rados.0.trial178.stdout:2745: expect (ObjNum 881 snap 257 seq_num 881) 2026-02-21T22:27:34.415 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:34.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469977585}) 2026-02-21T22:27:34.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1143015560}) 2026-02-21T22:27:34.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=469977585}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=965578673}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2629723382}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=953948667}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2629723382}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.408+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3280406888}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3560994363}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1312401097}) 2026-02-21T22:27:34.419 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3560994363}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=827784087}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1990904207}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2838495059}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1990904207}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2414430697}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3937884956}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2755415731}) 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.420 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3937884956}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=900929044}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=842222789}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=332672454}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=842222789}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2531920687}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3960001242}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=404569340}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3960001242}) 2026-02-21T22:27:34.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.412+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2863493643}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=306504101}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4288718028}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=306504101}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4192502358}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=795557017}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2019011100}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=795557017}) 2026-02-21T22:27:34.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2647912328}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3699631262}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2338352713}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3699631262}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2184353425}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295354622}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4088938424}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2295354622}) 2026-02-21T22:27:34.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2327706739}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4155656368}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=312133121}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4155656368}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.416+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2346650338}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1110918085}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1402698658}) 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1110918085}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3180104848}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3953578703}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2137802816}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3953578703}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3327271926}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2688756895}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2004561259}) 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2688756895}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4254385412}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1880743868}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3874395379}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1880743868}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1664068654}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2874011448}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1548315202}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2874011448}) 2026-02-21T22:27:34.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.420+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=545358060}) 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3048582093}) 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=270853859}) 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3048582093}) 2026-02-21T22:27:34.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3392687404}) 2026-02-21T22:27:34.431 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3179636940}) 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=921880064}) 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3179636940}) 2026-02-21T22:27:34.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.424+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600210772}) 2026-02-21T22:27:34.443 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:34.507 INFO:tasks.rados.rados.0.trial178.stdout:2746: finishing copy_from racing read to trial17818531-33 2026-02-21T22:27:34.507 INFO:tasks.rados.rados.0.trial178.stdout:2746: finishing copy_from to trial17818531-33 2026-02-21T22:27:34.507 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1166 (ObjNum 927 snap 271 seq_num 927) dirty exists 2026-02-21T22:27:34.507 INFO:tasks.rados.rados.0.trial178.stdout:2738: done (6 left) 2026-02-21T22:27:34.507 INFO:tasks.rados.rados.0.trial178.stdout:2739: done (5 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2741: done (4 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2743: done (3 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2744: done (2 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2745: done (1 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2746: done (0 left) 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2747: write oid 11 current snap is 280 2026-02-21T22:27:34.508 INFO:tasks.rados.rados.0.trial178.stdout:2747: seq_num 965 ranges {618755=475763,1636300=457996} 2026-02-21T22:27:34.514 INFO:tasks.rados.rados.0.trial178.stdout:2747: writing trial17818531-11 from 618755 to 1094518 tid 1 2026-02-21T22:27:34.522 INFO:tasks.rados.rados.0.trial178.stdout:2747: writing trial17818531-11 from 1636300 to 2094296 tid 2 2026-02-21T22:27:34.522 INFO:tasks.rados.rados.0.trial178.stdout:2748: read oid 41 snap -1 2026-02-21T22:27:34.522 INFO:tasks.rados.rados.0.trial178.stdout:2748: expect (ObjNum 938 snap 276 seq_num 938) 2026-02-21T22:27:34.522 INFO:tasks.rados.rados.0.trial178.stdout:2749: snap_remove snap 276 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325175605}) 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2404136557}) 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3325175605}) 2026-02-21T22:27:34.523 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.520+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4060527365}) 2026-02-21T22:27:34.533 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.533 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100547992}) 2026-02-21T22:27:34.533 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3048170688}) 2026-02-21T22:27:34.533 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.533 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=100547992}) 2026-02-21T22:27:34.534 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=449372158}) 2026-02-21T22:27:34.535 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.535 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2633759314}) 2026-02-21T22:27:34.535 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=320110767}) 2026-02-21T22:27:34.535 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2633759314}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4012756594}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3467681623}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1169464338}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3467681623}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1945214310}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568774091}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3013990969}) 2026-02-21T22:27:34.536 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.537 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3568774091}) 2026-02-21T22:27:34.537 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.532+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=302180905}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=1} crc {logm=3565557514,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=520051809,mon_config_key=2063301084,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3711774378}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=58,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=4} crc {logm=4160724243,mdsmap=3074481156,mgr=2886354254,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2002184583,mon_config_key=259807171,monmap=918322475,nvmeofgw=3360707892,osd_metadata=1213673387}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=60,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=1} crc {logm=3565557514,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=520051809,mon_config_key=2063301084,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3711774378}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=3} crc {logm=2801619228,mdsmap=3074481156,mgr=2974333102,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=3094160041,mon_config_key=2034202511,monmap=918322475,nvmeofgw=3360707892,osd_metadata=156584977}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1493522314,osd_pg_creating=650239432,osd_snap=1928682829}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=5,osd_snap=95} crc {osd_metadata=1493522314,osd_snap=2173112829}) 2026-02-21T22:27:34.539 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1493522314,osd_pg_creating=650239432,osd_snap=1928682829}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=5,osd_pg_creating=1,osd_snap=94} crc {osd_metadata=1493522314,osd_pg_creating=650239432,osd_snap=4031219457}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=17,osdmap=83} crc {osd_snap=4070149196,osdmap=1536002858}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=15,osdmap=85} crc {osd_snap=3718699044,osdmap=3141857542}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=17,osdmap=83} crc {osd_snap=4070149196,osdmap=1536002858}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=17,osdmap=83} crc {osd_snap=4070149196,osdmap=1067565857}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1803614130}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2243308455}) 2026-02-21T22:27:34.540 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.541 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1803614130}) 2026-02-21T22:27:34.541 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.536+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4236787042}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4293771933}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=99} crc {osdmap=3349576848}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4293771933}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3734711429}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=2} crc {osdmap=1525206359}) 2026-02-21T22:27:34.542 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:27:34.540+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=2} crc {osdmap=1525206360}) 2026-02-21T22:27:35.194 INFO:tasks.rados.rados.0.trial178.stdout:2749: done (2 left) 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2750: delete oid 8 current snap is 280 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: finishing write tid 1 to trial17818531-11 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: finishing write tid 2 to trial17818531-11 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: finishing write tid 3 to trial17818531-11 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: finishing write tid 4 to trial17818531-11 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: oid 11 updating version 0 to 1353 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: oid 11 updating version 1353 to 1354 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:2747: oid 11 updating version 1354 to 1355 2026-02-21T22:27:35.195 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1355 (ObjNum 965 snap 280 seq_num 965) dirty exists 2026-02-21T22:27:35.196 INFO:tasks.rados.rados.0.trial178.stdout:2747: left oid 11 (ObjNum 965 snap 280 seq_num 965) 2026-02-21T22:27:35.196 INFO:tasks.rados.rados.0.trial178.stdout:2748: expect (ObjNum 938 snap 276 seq_num 938) 2026-02-21T22:27:35.245 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2854655 2026-02-21T22:27:35.369 INFO:tasks.rados.rados.0.trial178.stdout:2747: done (2 left) 2026-02-21T22:27:35.369 INFO:tasks.rados.rados.0.trial178.stdout:2748: done (1 left) 2026-02-21T22:27:35.369 INFO:tasks.rados.rados.0.trial178.stdout:2750: done (0 left) 2026-02-21T22:27:35.369 INFO:tasks.rados.rados.0.trial178.stdout:2751: delete oid 29 current snap is 280 2026-02-21T22:27:35.370 INFO:tasks.rados.rados.0.trial178.stdout:2751: done (0 left) 2026-02-21T22:27:35.370 INFO:tasks.rados.rados.0.trial178.stdout:2752: write (excl) oid 19 current snap is 280 2026-02-21T22:27:35.370 INFO:tasks.rados.rados.0.trial178.stdout:2752: seq_num 968 ranges {669176=417863,1811746=457430} 2026-02-21T22:27:35.377 INFO:tasks.rados.rados.0.trial178.stdout:2752: writing trial17818531-19 from 669176 to 1087039 tid 1 2026-02-21T22:27:35.386 INFO:tasks.rados.rados.0.trial178.stdout:2752: writing trial17818531-19 from 1811746 to 2269176 tid 2 2026-02-21T22:27:35.386 INFO:tasks.rados.rados.0.trial178.stdout:2753: delete oid 40 current snap is 280 2026-02-21T22:27:35.386 INFO:tasks.rados.rados.0.trial178.stdout:2752: finishing write tid 1 to trial17818531-19 2026-02-21T22:27:35.387 INFO:tasks.rados.rados.0.trial178.stdout:2753: done (1 left) 2026-02-21T22:27:35.387 INFO:tasks.rados.rados.0.trial178.stdout:2754: copy_from oid 31 from oid 42 current snap is 280 2026-02-21T22:27:35.387 INFO:tasks.rados.rados.0.trial178.stdout:2755: write oid 40 current snap is 280 2026-02-21T22:27:35.387 INFO:tasks.rados.rados.0.trial178.stdout:2755: seq_num 970 ranges {702790=568361,1812357=490433} 2026-02-21T22:27:35.396 INFO:tasks.rados.rados.0.trial178.stdout:2755: writing trial17818531-40 from 702790 to 1271151 tid 1 2026-02-21T22:27:35.403 INFO:tasks.rados.rados.0.trial178.stdout:2755: writing trial17818531-40 from 1812357 to 2302790 tid 2 2026-02-21T22:27:35.404 INFO:tasks.rados.rados.0.trial178.stdout:2756: write (excl) oid 11 current snap is 280 2026-02-21T22:27:35.404 INFO:tasks.rados.rados.0.trial178.stdout:2756: seq_num 971 ranges {719597=643610,1870839=448758} 2026-02-21T22:27:35.413 INFO:tasks.rados.rados.0.trial178.stdout:2756: writing trial17818531-11 from 719597 to 1363207 tid 1 2026-02-21T22:27:35.420 INFO:tasks.rados.rados.0.trial178.stdout:2756: writing trial17818531-11 from 1870839 to 2319597 tid 2 2026-02-21T22:27:35.420 INFO:tasks.rados.rados.0.trial178.stdout:2757: rollback oid 12 current snap is 280 2026-02-21T22:27:35.420 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 278 2026-02-21T22:27:35.420 INFO:tasks.rados.rados.0.trial178.stdout:2752: finishing write tid 2 to trial17818531-19 2026-02-21T22:27:35.420 INFO:tasks.rados.rados.0.trial178.stdout:2752: finishing write tid 3 to trial17818531-19 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2754: finishing copy_from to trial17818531-31 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2754: got expected ENOENT (src dne) 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2754: finishing copy_from racing read to trial17818531-31 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: finishing write tid 4 to trial17818531-19 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: oid 19 updating version 0 to 1564 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: oid 19 version 1564 is already newer than 1563 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: oid 19 version 1564 is already newer than 1562 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1564 (ObjNum 968 snap 280 seq_num 968) dirty exists 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: left oid 19 (ObjNum 968 snap 280 seq_num 968) 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2755: finishing write tid 1 to trial17818531-40 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2755: finishing write tid 2 to trial17818531-40 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2755: finishing write tid 3 to trial17818531-40 2026-02-21T22:27:35.421 INFO:tasks.rados.rados.0.trial178.stdout:2752: done (4 left) 2026-02-21T22:27:35.422 INFO:tasks.rados.rados.0.trial178.stdout:2754: done (3 left) 2026-02-21T22:27:35.422 INFO:tasks.rados.rados.0.trial178.stdout:2758: write oid 38 current snap is 280 2026-02-21T22:27:35.422 INFO:tasks.rados.rados.0.trial178.stdout:2758: seq_num 972 ranges {736404=718859,1929321=407083} 2026-02-21T22:27:35.426 INFO:tasks.rados.rados.0.trial178.stdout:2758: writing trial17818531-38 from 736404 to 1455263 tid 1 2026-02-21T22:27:35.431 INFO:tasks.rados.rados.0.trial178.stdout:2758: writing trial17818531-38 from 1929321 to 2336404 tid 2 2026-02-21T22:27:35.431 INFO:tasks.rados.rados.0.trial178.stdout:2759: write (excl) oid 24 current snap is 280 2026-02-21T22:27:35.431 INFO:tasks.rados.rados.0.trial178.stdout:2759: seq_num 973 ranges {753211=794108,1987803=365408} 2026-02-21T22:27:35.432 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:27:35.432 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:27:35.437 INFO:tasks.rados.rados.0.trial178.stdout:2759: writing trial17818531-24 from 753211 to 1547319 tid 1 2026-02-21T22:27:35.440 INFO:tasks.rados.rados.0.trial178.stdout:2759: writing trial17818531-24 from 1987803 to 2353211 tid 2 2026-02-21T22:27:35.440 INFO:tasks.rados.rados.0.trial178.stdout:2760: copy_from oid 27 from oid 15 current snap is 280 2026-02-21T22:27:35.440 INFO:tasks.rados.rados.0.trial178.stdout:2755: finishing write tid 4 to trial17818531-40 2026-02-21T22:27:35.440 INFO:tasks.rados.rados.0.trial178.stdout:2755: oid 40 updating version 0 to 1176 2026-02-21T22:27:35.440 INFO:tasks.rados.rados.0.trial178.stdout:2755: oid 40 updating version 1176 to 1178 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2755: oid 40 version 1178 is already newer than 1177 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1178 (ObjNum 970 snap 280 seq_num 970) dirty exists 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2755: left oid 40 (ObjNum 970 snap 280 seq_num 970) 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: finishing write tid 1 to trial17818531-11 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2757: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: finishing write tid 2 to trial17818531-11 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: finishing write tid 3 to trial17818531-11 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: finishing write tid 4 to trial17818531-11 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: oid 11 updating version 0 to 1361 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: oid 11 version 1361 is already newer than 1360 2026-02-21T22:27:35.441 INFO:tasks.rados.rados.0.trial178.stdout:2756: oid 11 updating version 1361 to 1362 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1362 (ObjNum 971 snap 280 seq_num 971) dirty exists 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2756: left oid 11 (ObjNum 971 snap 280 seq_num 971) 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2758: finishing write tid 1 to trial17818531-38 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2758: finishing write tid 2 to trial17818531-38 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2755: done (5 left) 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2756: done (4 left) 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2757: done (3 left) 2026-02-21T22:27:35.442 INFO:tasks.rados.rados.0.trial178.stdout:2761: snap_remove snap 278 2026-02-21T22:27:35.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: get_monmap_and_config 2026-02-21T22:27:35.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: monmap: 2026-02-21T22:27:35.492 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.492 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:35.492044+0000 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:35.492044+0000 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 1 Processor -- start 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 1 -- start start 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: init 2026-02-21T22:27:35.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-b con 0x7f5e2408c4e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.488+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2408d410 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-a con 0x7f5e2408d890 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2408bd50 con 0x7f5e2408d890 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2408c8b0 con 0x7f5e2408d410 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e24083560 con 0x7f5e2408c4e0 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 10 monclient(hunting): authenticate will time out at 1864.728049s 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a8b7640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d890 0x7f5e241aa1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44910/0 (socket says 10.20.193.102:44910) 2026-02-21T22:27:35.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a8b7640 1 -- 10.20.193.102:0/2359489622 learned_addr learned my addr 10.20.193.102:0/2359489622 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a0b6640 1 --1- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6790/0 conn(0x7f5e2408c4e0 0x7f5e2408cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51072/0 (socket says 10.20.193.102:51072) 2026-02-21T22:27:35.501 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a8b7640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d890 legacy=0x7f5e241aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a0b6640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6790/0 conn(0x7f5e2408c4e0 legacy=0x7f5e2408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e298b5640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.178:6789/0 conn(0x7f5e2408d410 legacy=0x7f5e24188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819693267 0 0) 0x7f5e2408bd50 con 0x7f5e2408d890 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): my global_id is 34605 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e0c003620 con 0x7f5e2408d890 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607012088 0 0) 0x7f5e14002c60 con 0x7f5e2408c4e0 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): my global_id is 94115 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e2408bd50 con 0x7f5e2408c4e0 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512220612 0 0) 0x7f5e2408c8b0 con 0x7f5e2408d410 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient(hunting): my global_id is 54400 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e14002c60 con 0x7f5e2408d410 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 667802731 0 0) 0x7f5e0c003620 con 0x7f5e2408d890 2026-02-21T22:27:35.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6790/0 conn(0x7f5e2408c4e0 legacy=0x7f5e2408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.178:6789/0 conn(0x7f5e2408d410 legacy=0x7f5e24188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 monclient: found mon.noname-a 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e24084bc0 con 0x7f5e2408d890 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 5 monclient: authenticate success, global_id 34605 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5e20004620 con 0x7f5e2408d890 2026-02-21T22:27:35.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: dump: 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.505 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2a8b7640 1 --1- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d890 0x7f5e241aa1c0 :-1 s=OPENED pgs=474 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f5e20006060 mon_map magic: 0 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5e200048f0 con 0x7f5e2408d890 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 4 set_mon_vals no callback set 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 10 monclient: shutdown 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- 10.20.193.102:0/2359489622 >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d890 legacy=0x7f5e241aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 1 -- 10.20.193.102:0/2359489622 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5e20006060 con 0x7f5e2408d890 2026-02-21T22:27:35.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e290b4640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- 10.20.193.102:0/2359489622 shutdown_connections 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- 10.20.193.102:0/2359489622 >> 10.20.193.102:0/2359489622 conn(0x7f5e24091a50 msgr2=0x7f5e24091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- 10.20.193.102:0/2359489622 shutdown_connections 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.492+0000 7f5e2c341640 1 -- 10.20.193.102:0/2359489622 wait complete. 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: monmap: 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.507 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 Processor -- start 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- start start 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: init 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-b con 0x7f5e2408c4e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-a con 0x7f5e2408d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient(hunting): picked mon.noname-c con 0x7f5e2408d890 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e24087bd0 con 0x7f5e2408d410 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2405d780 con 0x7f5e2408c4e0 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e2405da60 con 0x7f5e2408d890 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient(hunting): authenticate will time out at 1864.732809s 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2a8b7640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5e2408d890 0x7f5e24241000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51088/0 (socket says 10.20.193.102:51088) 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2a8b7640 1 -- 10.20.193.102:0/3511057832 learned_addr learned my addr 10.20.193.102:0/3511057832 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2a8b7640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.102:6790/0 conn(0x7f5e2408d890 legacy=0x7f5e24241000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e298b5640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d410 legacy=0x7f5e2421f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2a0b6640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.178:6789/0 conn(0x7f5e2408c4e0 legacy=0x7f5e241b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122218771 0 0) 0x7f5e2405da60 con 0x7f5e2408d890 2026-02-21T22:27:35.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): my global_id is 94118 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5df4003620 con 0x7f5e2408d890 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450063171 0 0) 0x7f5e14002c60 con 0x7f5e2408d410 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): my global_id is 34608 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e2405da60 con 0x7f5e2408d410 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794075384 0 0) 0x7f5e2405d780 con 0x7f5e2408c4e0 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): my global_id is 54403 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e14002c60 con 0x7f5e2408c4e0 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3335002318 0 0) 0x7f5df4003620 con 0x7f5e2408d890 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e2405d780 con 0x7f5e2408d890 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1515023489 0 0) 0x7f5e2405da60 con 0x7f5e2408d410 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5df4003620 con 0x7f5e2408d410 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5e180033d0 con 0x7f5e2408d410 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): dump: 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.511 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5e20005fa0 con 0x7f5e2408d890 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient(hunting): dump: 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1066087831 0 0) 0x7f5e2405d780 con 0x7f5e2408d890 2026-02-21T22:27:35.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.178:6789/0 conn(0x7f5e2408c4e0 legacy=0x7f5e241b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.102:6789/0 conn(0x7f5e2408d410 legacy=0x7f5e2421f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 monclient: found mon.c 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e24085f90 con 0x7f5e2408d890 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 5 monclient: authenticate success, global_id 94118 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _renew_subs 2026-02-21T22:27:35.514 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2405c260 con 0x7f5e2408d890 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _renew_subs 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5e2405e170 con 0x7f5e2408d890 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5e20004040 con 0x7f5e2408d890 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5e200066f0 con 0x7f5e2408d890 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e288b3640 4 set_mon_vals no callback set 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.496+0000 7f5e0affd640 10 monclient: dump: 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.515 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e2c341640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e2c341640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e2c341640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.516 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e240008e0 con 0x7f5e2408d890 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5e20012d80 con 0x7f5e2408d890 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f5e2004dac0 con 0x7f5e2408d890 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e2a0b6640 10 monclient: get_auth_request con 0x7f5df403d540 auth_method 0 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5e20018aa0 con 0x7f5e2408d890 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e0affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.500+0000 7f5e0affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:35.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:27:35.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:35.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5e24090810 con 0x7f5e2408d890 2026-02-21T22:27:35.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e0affd640 1 -- 10.20.193.102:0/3511057832 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 394174254) 0x7f5e2001dd00 con 0x7f5e2408d890 2026-02-21T22:27:35.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e0affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:35.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e0affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:35.617 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5df403d540 legacy=0x7f5df405d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 10 monclient: shutdown 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 >> v1:10.20.193.102:6790/0 conn(0x7f5e2408d890 legacy=0x7f5e24241000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 shutdown_connections 2026-02-21T22:27:35.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 >> 10.20.193.102:0/3511057832 conn(0x7f5e24091a50 msgr2=0x7f5e241b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 shutdown_connections 2026-02-21T22:27:35.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.616+0000 7f5e2c341640 1 -- 10.20.193.102:0/3511057832 wait complete. 2026-02-21T22:27:35.626 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:27:35.627 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:27:35.627 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:27:35.627 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:27:35.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.684+0000 7f7d9e24e640 10 monclient: get_monmap_and_config 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.684+0000 7f7d9e24e640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.684+0000 7f7d9e24e640 10 monclient: monmap: 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:35.688604+0000 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:35.688604+0000 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.688 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:35.689 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:35.689 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 1 Processor -- start 2026-02-21T22:27:35.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 1 -- start start 2026-02-21T22:27:35.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient: init 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-a con 0x7f7d9805d7c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-b con 0x7f7d981409c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-c con 0x7f7d981831e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d98059940 con 0x7f7d9805d7c0 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d98090440 con 0x7f7d981831e0 2026-02-21T22:27:35.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9805a9e0 con 0x7f7d981409c0 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient(hunting): authenticate will time out at 1864.924772s 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d977fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7d9805d7c0 0x7f7d9805db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44942/0 (socket says 10.20.193.102:44942) 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d977fe640 1 -- 10.20.193.102:0/4021232055 learned_addr learned my addr 10.20.193.102:0/4021232055 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d96ffd640 1 --1- 10.20.193.102:0/4021232055 >> v1:10.20.193.102:6790/0 conn(0x7f7d981409c0 0x7f7d98181ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51096/0 (socket says 10.20.193.102:51096) 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d977fe640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.102:6789/0 conn(0x7f7d9805d7c0 legacy=0x7f7d9805db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d96ffd640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.102:6790/0 conn(0x7f7d981409c0 legacy=0x7f7d98181ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d97fff640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.178:6789/0 conn(0x7f7d981831e0 legacy=0x7f7d981a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370834554 0 0) 0x7f7d9805a9e0 con 0x7f7d981409c0 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): my global_id is 94121 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d7c003620 con 0x7f7d981409c0 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400016919 0 0) 0x7f7d98090440 con 0x7f7d981831e0 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient(hunting): my global_id is 54406 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9805a9e0 con 0x7f7d981831e0 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 952779306 0 0) 0x7f7d7c003620 con 0x7f7d981409c0 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.178:6789/0 conn(0x7f7d981831e0 legacy=0x7f7d981a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.102:6789/0 conn(0x7f7d9805d7c0 legacy=0x7f7d9805db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 monclient: found mon.noname-b 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d98140e40 con 0x7f7d981409c0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d88003110 con 0x7f7d981409c0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 5 monclient: authenticate success, global_id 94121 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:27:35.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: dump: 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d88003450 con 0x7f7d981409c0 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 1 -- 10.20.193.102:0/4021232055 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d88004a90 con 0x7f7d981409c0 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d967fc640 10 monclient: dump: 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.699 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.688+0000 7f7d9e24e640 4 set_mon_vals no callback set 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: shutdown 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- 10.20.193.102:0/4021232055 >> v1:10.20.193.102:6790/0 conn(0x7f7d981409c0 legacy=0x7f7d98181ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- 10.20.193.102:0/4021232055 shutdown_connections 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- 10.20.193.102:0/4021232055 >> 10.20.193.102:0/4021232055 conn(0x7f7d98056d40 msgr2=0x7f7d98093bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- 10.20.193.102:0/4021232055 shutdown_connections 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- 10.20.193.102:0/4021232055 wait complete. 2026-02-21T22:27:35.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: monmap: 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 Processor -- start 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- start start 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: init 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-b con 0x7f7d9805d7c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-a con 0x7f7d981409c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient(hunting): picked mon.noname-c con 0x7f7d981831e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d98129cd0 con 0x7f7d981409c0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9805e900 con 0x7f7d9805d7c0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9810cdb0 con 0x7f7d981831e0 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d9e24e640 10 monclient(hunting): authenticate will time out at 1864.929523s 2026-02-21T22:27:35.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d97fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7d981831e0 0x7f7d98182880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51106/0 (socket says 10.20.193.102:51106) 2026-02-21T22:27:35.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d96ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7d981409c0 0x7f7d9813de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44944/0 (socket says 10.20.193.102:44944) 2026-02-21T22:27:35.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d97fff640 1 -- 10.20.193.102:0/440643772 learned_addr learned my addr 10.20.193.102:0/440643772 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d97fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6790/0 conn(0x7f7d981831e0 legacy=0x7f7d98182880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d96ffd640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d981409c0 legacy=0x7f7d9813de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 26996957 0 0) 0x7f7d9810cdb0 con 0x7f7d981831e0 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): my global_id is 94124 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d68003620 con 0x7f7d981831e0 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d977fe640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.178:6789/0 conn(0x7f7d9805d7c0 legacy=0x7f7d9813d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347458753 0 0) 0x7f7d98140e40 con 0x7f7d981409c0 2026-02-21T22:27:35.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 10 monclient(hunting): my global_id is 34611 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.692+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9810cdb0 con 0x7f7d981409c0 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406820051 0 0) 0x7f7d9805e900 con 0x7f7d9805d7c0 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): my global_id is 54409 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d98140e40 con 0x7f7d9805d7c0 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1879767794 0 0) 0x7f7d68003620 con 0x7f7d981831e0 2026-02-21T22:27:35.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d9805e900 con 0x7f7d981831e0 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d8c0049a0 con 0x7f7d981831e0 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): dump: 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.711 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2134658759 0 0) 0x7f7d9810cdb0 con 0x7f7d981409c0 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d68003620 con 0x7f7d981409c0 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d880048e0 con 0x7f7d981409c0 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): dump: 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.712 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2967943788 0 0) 0x7f7d9805e900 con 0x7f7d981831e0 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.178:6789/0 conn(0x7f7d9805d7c0 legacy=0x7f7d9813d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d981409c0 legacy=0x7f7d9813de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 monclient: found mon.c 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9810b8c0 con 0x7f7d981831e0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 5 monclient: authenticate success, global_id 94124 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 10 monclient: _renew_subs 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9810dc20 con 0x7f7d981831e0 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 10 monclient: _renew_subs 2026-02-21T22:27:35.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7d9810c420 con 0x7f7d981831e0 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d8c005080 con 0x7f7d981831e0 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d8c005410 con 0x7f7d981831e0 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: dump: 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.715 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d95ffb640 4 set_mon_vals no callback set 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f7d8c005660 con 0x7f7d981831e0 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7d98129630 con 0x7f7d98129260 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f7d8c04dbd0 con 0x7f7d981831e0 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d96ffd640 10 monclient: get_auth_request con 0x7f7d6803d780 auth_method 0 2026-02-21T22:27:35.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d977fe640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d98129260 legacy=0x7f7d98088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084441368 0 0) 0x7f7d98129630 con 0x7f7d98129260 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient(hunting): my global_id is 34614 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d9805e900 con 0x7f7d98129260 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1975139557 0 0) 0x7f7d9805e900 con 0x7f7d98129260 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d98129630 con 0x7f7d98129260 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d84003280 con 0x7f7d98129260 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:35.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 10 monclient: dump: 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3961354216 0 0) 0x7f7d98129630 con 0x7f7d98129260 2026-02-21T22:27:35.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.696+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d9810d430 con 0x7f7d98129260 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f7d840048e0 con 0x7f7d98129260 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d73fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d73fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d98129260 legacy=0x7f7d98088de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d98129260 legacy=0x7f7d98088de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d9e24e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.700+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7d98062ce0 con 0x7f7d980894f0 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d977fe640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d980894f0 legacy=0x7f7d981181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236654012 0 0) 0x7f7d98062ce0 con 0x7f7d980894f0 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient(hunting): my global_id is 34617 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d98129630 con 0x7f7d980894f0 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1061555461 0 0) 0x7f7d98129630 con 0x7f7d980894f0 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d98062ce0 con 0x7f7d980894f0 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7d84004ee0 con 0x7f7d980894f0 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient: dump: 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.720 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 330073213 0 0) 0x7f7d98062ce0 con 0x7f7d980894f0 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7d9810c760 con 0x7f7d980894f0 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3895774331) 0x7f7d84007220 con 0x7f7d980894f0 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d980894f0 legacy=0x7f7d981181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d73fff640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6789/0 conn(0x7f7d980894f0 legacy=0x7f7d981181e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 604573, 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:35.722 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.723 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.724 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:35.725 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:35.726 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.727 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.728 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.730 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.731 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.732 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.733 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:35.734 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:35.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.178:6800/2549949439 conn(0x7f7d6803d780 legacy=0x7f7d6805dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 10 monclient: shutdown 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 >> v1:10.20.193.102:6790/0 conn(0x7f7d981831e0 legacy=0x7f7d98182880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d97fff640 1 -- 10.20.193.102:0/440643772 reap_dead start 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d97fff640 1 -- 10.20.193.102:0/440643772 reap_dead start 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 shutdown_connections 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 >> 10.20.193.102:0/440643772 conn(0x7f7d98056d40 msgr2=0x7f7d98090680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 shutdown_connections 2026-02-21T22:27:35.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.704+0000 7f7d9e24e640 1 -- 10.20.193.102:0/440643772 wait complete. 2026-02-21T22:27:35.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:35.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:27:35.736 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: get_monmap_and_config 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: monmap: 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:35.806590+0000 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:35.806590+0000 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:35.807 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 1 Processor -- start 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 1 -- start start 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: init 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-b con 0x7f23c413f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-c con 0x7f23c413fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-a con 0x7f23c41401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c4092490 con 0x7f23c41401f0 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c40571f0 con 0x7f23c413fd90 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c40574d0 con 0x7f23c413f9c0 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c94c5640 10 monclient(hunting): authenticate will time out at 1865.042594s 2026-02-21T22:27:35.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 0x7f23c40906e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51116/0 (socket says 10.20.193.102:51116) 2026-02-21T22:27:35.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c3fff640 1 -- 10.20.193.102:0/969757227 learned_addr learned my addr 10.20.193.102:0/969757227 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.804+0000 7f23c8cc4640 1 --1- 10.20.193.102:0/969757227 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 0x7f23c41a5470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44954/0 (socket says 10.20.193.102:44954) 2026-02-21T22:27:35.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c3fff640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c40906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c8cc4640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 legacy=0x7f23c41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c37fe640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.178:6789/0 conn(0x7f23c413fd90 legacy=0x7f23c4183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355194113 0 0) 0x7f23c4092490 con 0x7f23c41401f0 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): my global_id is 34620 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23a8003620 con 0x7f23c41401f0 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312515434 0 0) 0x7f23c40574d0 con 0x7f23c413f9c0 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient(hunting): my global_id is 94127 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c4092490 con 0x7f23c413f9c0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 501416523 0 0) 0x7f23a8003620 con 0x7f23c41401f0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c40906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.178:6789/0 conn(0x7f23c413fd90 legacy=0x7f23c4183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 monclient: found mon.noname-a 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c40577b0 con 0x7f23c41401f0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 5 monclient: authenticate success, global_id 34620 2026-02-21T22:27:35.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b8004620 con 0x7f23c41401f0 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: dump: 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.814 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f23b80048f0 con 0x7f23c41401f0 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 1 -- 10.20.193.102:0/969757227 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b8005fa0 con 0x7f23c41401f0 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c2ffd640 10 monclient: dump: 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.815 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 4 set_mon_vals no callback set 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: shutdown 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- 10.20.193.102:0/969757227 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 legacy=0x7f23c41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- 10.20.193.102:0/969757227 shutdown_connections 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- 10.20.193.102:0/969757227 >> 10.20.193.102:0/969757227 conn(0x7f23c405e3f0 msgr2=0x7f23c4095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- 10.20.193.102:0/969757227 shutdown_connections 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- 10.20.193.102:0/969757227 wait complete. 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: monmap: 2026-02-21T22:27:35.816 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 Processor -- start 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- start start 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: init 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:35.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-c con 0x7f23c413f9c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-b con 0x7f23c413fd90 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient(hunting): picked mon.noname-a con 0x7f23c41401f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c413f440 con 0x7f23c41401f0 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c412f4a0 con 0x7f23c413fd90 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c412f780 con 0x7f23c413f9c0 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c94c5640 10 monclient(hunting): authenticate will time out at 1865.047066s 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 0x7f23c412bd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51130/0 (socket says 10.20.193.102:51130) 2026-02-21T22:27:35.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.808+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 learned_addr learned my addr 10.20.193.102:0/2534611309 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c412bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c8cc4640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 legacy=0x7f23c41a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.822 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c37fe640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.178:6789/0 conn(0x7f23c413fd90 legacy=0x7f23c412c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293081944 0 0) 0x7f23c412f780 con 0x7f23c413f9c0 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): my global_id is 94130 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c8cc4640 1 --1- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 0x7f23c41a1630 :-1 s=OPENED pgs=481 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f23c40577b0 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2394003620 con 0x7f23c413f9c0 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953506339 0 0) 0x7f23c412f4a0 con 0x7f23c413fd90 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): my global_id is 54415 2026-02-21T22:27:35.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c412f780 con 0x7f23c413fd90 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3064142932 0 0) 0x7f2394003620 con 0x7f23c413f9c0 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23c412f4a0 con 0x7f23c413f9c0 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b00033a0 con 0x7f23c413f9c0 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): dump: 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.824 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4121864586 0 0) 0x7f23c412f780 con 0x7f23c413fd90 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c412bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c412bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c3fff640 1 --1- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 0x7f23c412bd00 :-1 s=OPENED pgs=382 gs=1 cs=1 l=1).handle_message read tag failed 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2394003620 con 0x7f23c413fd90 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c3fff640 1 --1- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 0x7f23c412bd00 :-1 s=OPENED pgs=382 gs=1 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b4003380 con 0x7f23c413fd90 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): dump: 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.825 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.102:6790/0 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1902099070 0 0) 0x7f2394003620 con 0x7f23c413fd90 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c413f9c0 legacy=0x7f23c412bd00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6789/0 conn(0x7f23c41401f0 legacy=0x7f23c41a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 monclient: found mon.b 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c412cb20 con 0x7f23c413fd90 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395843837 0 0) 0x7f23c40577b0 con 0x7f23c41401f0 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 5 monclient: authenticate success, global_id 54415 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 10 monclient: _renew_subs 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.178:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f23c408a3f0 con 0x7f23c413fd90 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 10 monclient: _renew_subs 2026-02-21T22:27:35.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 10 monclient: _send_mon_message to mon.b at v1:10.20.193.178:6789/0 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.178:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f23c412fa60 con 0x7f23c413fd90 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f23b40039c0 con 0x7f23c413fd90 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b4003d20 con 0x7f23c413fd90 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 10 monclient: dump: 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.828 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c27fc640 4 set_mon_vals no callback set 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f23c412e630 con 0x7f23c412e260 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f23b4003ec0 con 0x7f23c413fd90 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.1 v1:10.20.193.178:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f23b404c610 con 0x7f23c413fd90 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.812+0000 7f23c8cc4640 10 monclient: get_auth_request con 0x7f239403d590 auth_method 0 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c412e260 legacy=0x7f23c408de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2324158790 0 0) 0x7f23c412e630 con 0x7f23c412e260 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient(hunting): my global_id is 94133 2026-02-21T22:27:35.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c40577b0 con 0x7f23c412e260 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3905380436 0 0) 0x7f23c40577b0 con 0x7f23c412e260 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23c412e630 con 0x7f23c412e260 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b0004e40 con 0x7f23c412e260 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient: dump: 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.830 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3736703710 0 0) 0x7f23c412e630 con 0x7f23c412e260 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23c40008e0 con 0x7f23c412e260 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f23b0007270 con 0x7f23c412e260 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c412e260 legacy=0x7f23c408de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.816+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c412e260 legacy=0x7f23c408de30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c94c5640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:35.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f23c41a6da0 con 0x7f23c404d280 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c3fff640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c404d280 legacy=0x7f23c411c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254853356 0 0) 0x7f23c41a6da0 con 0x7f23c404d280 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient(hunting): my global_id is 94136 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c412e630 con 0x7f23c404d280 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 535757542 0 0) 0x7f23c412e630 con 0x7f23c404d280 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f23c41a6da0 con 0x7f23c404d280 2026-02-21T22:27:35.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f23b0005600 con 0x7f23c404d280 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.820+0000 7f23c0ff9640 10 monclient: dump: 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 45969818 0 0) 0x7f23c41a6da0 con 0x7f23c404d280 2026-02-21T22:27:35.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f23c411ee90 con 0x7f23c404d280 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 552264956) 0x7f23b0006a00 con 0x7f23c404d280 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c404d280 legacy=0x7f23c411c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c0ff9640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.102:6790/0 conn(0x7f23c404d280 legacy=0x7f23c411c760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:27:35.835 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 2099, 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.836 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.837 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:35.838 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.839 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.840 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.841 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.842 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.843 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:35.844 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.178:6800/2549949439 conn(0x7f239403d590 legacy=0x7f239405da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 10 monclient: shutdown 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 >> v1:10.20.193.178:6789/0 conn(0x7f23c413fd90 legacy=0x7f23c412c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c8cc4640 1 -- 10.20.193.102:0/2534611309 reap_dead start 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c8cc4640 1 -- 10.20.193.102:0/2534611309 reap_dead start 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 shutdown_connections 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 >> 10.20.193.102:0/2534611309 conn(0x7f23c405e3f0 msgr2=0x7f23c40916c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 shutdown_connections 2026-02-21T22:27:35.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.824+0000 7f23c94c5640 1 -- 10.20.193.102:0/2534611309 wait complete. 2026-02-21T22:27:35.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:35.845 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:27:35.846 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: get_monmap_and_config 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: monmap: 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:35.918301+0000 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:35.918301+0000 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.917 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:35.918 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:35.918 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 1 Processor -- start 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 1 -- start start 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: init 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient(hunting): picked mon.noname-b con 0x7f662408c1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient(hunting): picked mon.noname-a con 0x7f662408cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient(hunting): picked mon.noname-c con 0x7f6624183480 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f662408d780 con 0x7f662408cc80 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6624085da0 con 0x7f6624183480 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6624085420 con 0x7f662408c1a0 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662b126640 10 monclient(hunting): authenticate will time out at 1865.154506s 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662a124640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f662408c1a0 0x7f662408c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51162/0 (socket says 10.20.193.102:51162) 2026-02-21T22:27:35.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.916+0000 7f662a124640 1 -- 10.20.193.102:0/88872503 learned_addr learned my addr 10.20.193.102:0/88872503 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662a124640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.102:6790/0 conn(0x7f662408c1a0 legacy=0x7f662408c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629923640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.102:6789/0 conn(0x7f662408cc80 legacy=0x7f6624181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131485721 0 0) 0x7f6624085420 con 0x7f662408c1a0 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): my global_id is 94139 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f660c003620 con 0x7f662408c1a0 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147937809 0 0) 0x7f6618002cb0 con 0x7f662408cc80 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient(hunting): my global_id is 34626 2026-02-21T22:27:35.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6624085420 con 0x7f662408cc80 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662a925640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.178:6789/0 conn(0x7f6624183480 legacy=0x7f66241a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486123867 0 0) 0x7f660c003620 con 0x7f662408c1a0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.178:6789/0 conn(0x7f6624183480 legacy=0x7f66241a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.102:6789/0 conn(0x7f662408cc80 legacy=0x7f6624181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 monclient: found mon.noname-b 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624086640 con 0x7f662408c1a0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6614003110 con 0x7f662408c1a0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: dump: 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 5 monclient: authenticate success, global_id 94139 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f66140033e0 con 0x7f662408c1a0 2026-02-21T22:27:35.925 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 1 -- 10.20.193.102:0/88872503 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6614004b50 con 0x7f662408c1a0 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f6629122640 10 monclient: dump: 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.926 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 4 set_mon_vals no callback set 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: shutdown 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- 10.20.193.102:0/88872503 >> v1:10.20.193.102:6790/0 conn(0x7f662408c1a0 legacy=0x7f662408c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- 10.20.193.102:0/88872503 shutdown_connections 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- 10.20.193.102:0/88872503 >> 10.20.193.102:0/88872503 conn(0x7f662408f6a0 msgr2=0x7f662408faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- 10.20.193.102:0/88872503 shutdown_connections 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- 10.20.193.102:0/88872503 wait complete. 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: build_initial_monmap 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: monmap: 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:35.927 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 Processor -- start 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- start start 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: init 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient(hunting): picked mon.noname-b con 0x7f662408c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient(hunting): picked mon.noname-c con 0x7f662408cc80 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient(hunting): picked mon.noname-a con 0x7f6624183480 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6624088990 con 0x7f6624183480 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6624153830 con 0x7f662408c1a0 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f662408a5e0 con 0x7f662408cc80 2026-02-21T22:27:35.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:35.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662b126640 10 monclient(hunting): authenticate will time out at 1865.158624s 2026-02-21T22:27:35.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662a925640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f6624183480 0x7f6624175bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44980/0 (socket says 10.20.193.102:44980) 2026-02-21T22:27:35.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.920+0000 7f662a925640 1 -- 10.20.193.102:0/958794331 learned_addr learned my addr 10.20.193.102:0/958794331 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:35.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662a925640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.102:6789/0 conn(0x7f6624183480 legacy=0x7f6624175bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f6629923640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.102:6790/0 conn(0x7f662408cc80 legacy=0x7f66241744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343682688 0 0) 0x7f6624088990 con 0x7f6624183480 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): my global_id is 34629 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662a124640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f662408c1a0 legacy=0x7f6624084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65f4003620 con 0x7f6624183480 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 514387988 0 0) 0x7f662408a5e0 con 0x7f662408cc80 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): my global_id is 94142 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6624088990 con 0x7f662408cc80 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1013361903 0 0) 0x7f65f4003620 con 0x7f6624183480 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f662408a5e0 con 0x7f6624183480 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6620004910 con 0x7f6624183480 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): dump: 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.934 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150705998 0 0) 0x7f6624153830 con 0x7f662408c1a0 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): my global_id is 54421 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65f4003620 con 0x7f662408c1a0 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4034315069 0 0) 0x7f6624088990 con 0x7f662408cc80 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6624153830 con 0x7f662408cc80 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6618003710 con 0x7f662408cc80 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:35.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient(hunting): dump: 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 443908725 0 0) 0x7f662408a5e0 con 0x7f6624183480 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.102:6790/0 conn(0x7f662408cc80 legacy=0x7f66241744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f662408c1a0 legacy=0x7f6624084a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: _finish_hunting 0 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 monclient: found mon.a 2026-02-21T22:27:35.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624083530 con 0x7f6624183480 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: _finish_auth 0 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: _check_auth_tickets 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 5 monclient: authenticate success, global_id 34629 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 10 monclient: _renew_subs 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6624082c90 con 0x7f6624183480 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 10 monclient: _renew_subs 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6624085170 con 0x7f6624183480 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6620004ed0 con 0x7f6624183480 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6620005260 con 0x7f6624183480 2026-02-21T22:27:35.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 10 monclient: dump: 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f6628921640 4 set_mon_vals no callback set 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f66240830e0 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f66200054f0 con 0x7f6624183480 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f662004dae0 con 0x7f6624183480 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.924+0000 7f6629923640 10 monclient: get_auth_request con 0x7f65f403d880 auth_method 0 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f662a124640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f6624153180 legacy=0x7f662405b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745761657 0 0) 0x7f65f4003620 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient(hunting): my global_id is 54424 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f662408a5e0 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3762950061 0 0) 0x7f662408a5e0 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f65f4003620 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6614002b30 con 0x7f6624153180 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient: dump: 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 269053518 0 0) 0x7f65f4003620 con 0x7f6624153180 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6624153550 con 0x7f6624153180 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f6614003020 con 0x7f6624153180 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:35.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:35.941 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f6624153180 legacy=0x7f662405b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.941 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.928+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f6624153180 legacy=0x7f662405b7b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.941 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f662b126640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:35.941 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6624062ce0 con 0x7f662405bec0 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f662a124640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f662405bec0 legacy=0x7f662417f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3165205350 0 0) 0x7f6624062ce0 con 0x7f662405bec0 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient(hunting): my global_id is 54427 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65f4003620 con 0x7f662405bec0 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 752178143 0 0) 0x7f65f4003620 con 0x7f662405bec0 2026-02-21T22:27:35.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6624062ce0 con 0x7f662405bec0 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6614005000 con 0x7f662405bec0 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.932+0000 7f660affd640 10 monclient: dump: 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:35.943 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2140678041 0 0) 0x7f6624062ce0 con 0x7f662405bec0 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f66241831c0 con 0x7f662405bec0 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 2399648091) 0x7f66140072c0 con 0x7f662405bec0 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f662405bec0 legacy=0x7f662417f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f660affd640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6789/0 conn(0x7f662405bec0 legacy=0x7f662417f910 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:35.944 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 46, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 359578, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.945 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.946 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:35.947 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:35.948 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.949 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.950 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.951 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:35.952 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.953 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:35.954 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:35.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.178:6800/2549949439 conn(0x7f65f403d880 legacy=0x7f65f405dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 10 monclient: shutdown 2026-02-21T22:27:35.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:35.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 >> v1:10.20.193.102:6789/0 conn(0x7f6624183480 legacy=0x7f6624175bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:35.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662a925640 1 -- 10.20.193.102:0/958794331 reap_dead start 2026-02-21T22:27:35.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662a925640 1 -- 10.20.193.102:0/958794331 reap_dead start 2026-02-21T22:27:35.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 shutdown_connections 2026-02-21T22:27:35.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 >> 10.20.193.102:0/958794331 conn(0x7f662408f6a0 msgr2=0x7f66240574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:35.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 shutdown_connections 2026-02-21T22:27:35.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:35.936+0000 7f662b126640 1 -- 10.20.193.102:0/958794331 wait complete. 2026-02-21T22:27:35.956 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:27:35.956 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-21T22:27:35.956 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:27:35.956 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-21T22:27:35.956 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-21T22:27:35.957 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:27:35.957 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:27:35.970 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:27:35.970 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:27:35.971 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: get_monmap_and_config 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: build_initial_monmap 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: monmap: 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:36.031966+0000 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:36.031966+0000 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:36.030 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 1 Processor -- start 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 1 -- start start 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: init 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-b con 0x7f5d98085970 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-c con 0x7f5d98085d40 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-a con 0x7f5d981854c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9808df30 con 0x7f5d981854c0 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9808d900 con 0x7f5d98085d40 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d98056dd0 con 0x7f5d98085970 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d9c8c9640 10 monclient(hunting): authenticate will time out at 1865.267833s 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d95d74640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f5d98085d40 legacy=0x7f5d98183da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d96d76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5d981854c0 0x7f5d981a58a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44982/0 (socket says 10.20.193.102:44982) 2026-02-21T22:27:36.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.028+0000 7f5d96d76640 1 -- 10.20.193.102:0/1525280446 learned_addr learned my addr 10.20.193.102:0/1525280446 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:36.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d96d76640 1 -- 10.20.193.102:0/1525280446 >> v1:10.20.193.102:6789/0 conn(0x7f5d981854c0 legacy=0x7f5d981a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:36.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d96575640 1 -- 10.20.193.102:0/1525280446 >> v1:10.20.193.102:6790/0 conn(0x7f5d98085970 legacy=0x7f5d9808e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191121465 0 0) 0x7f5d9808df30 con 0x7f5d981854c0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _negotiate 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): my global_id is 34632 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d84003620 con 0x7f5d981854c0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1582496675 0 0) 0x7f5d98056dd0 con 0x7f5d98085970 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _negotiate 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient(hunting): my global_id is 94145 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d9808df30 con 0x7f5d98085970 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 777433545 0 0) 0x7f5d84003620 con 0x7f5d981854c0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 >> v1:10.20.193.102:6790/0 conn(0x7f5d98085970 legacy=0x7f5d9808e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 >> v1:10.20.193.178:6789/0 conn(0x7f5d98085d40 legacy=0x7f5d98183da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: _finish_hunting 0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 monclient: found mon.noname-a 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9808d620 con 0x7f5d981854c0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: _finish_auth 0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: _check_auth_tickets 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d88004620 con 0x7f5d981854c0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: dump: 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.034 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 5 monclient: authenticate success, global_id 34632 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5d880048f0 con 0x7f5d981854c0 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 1 -- 10.20.193.102:0/1525280446 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d88006060 con 0x7f5d981854c0 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d95573640 10 monclient: dump: 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:36.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 4 set_mon_vals no callback set 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 10 monclient: shutdown 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1525280446 >> v1:10.20.193.102:6789/0 conn(0x7f5d981854c0 legacy=0x7f5d981a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1525280446 shutdown_connections 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1525280446 >> 10.20.193.102:0/1525280446 conn(0x7f5d98054e40 msgr2=0x7f5d98092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1525280446 shutdown_connections 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1525280446 wait complete. 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 10 monclient: build_initial_monmap 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 10 monclient: monmap: 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.032+0000 7f5d9c8c9640 1 Processor -- start 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- start start 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: init 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-c con 0x7f5d98085970 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-a con 0x7f5d98085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient(hunting): picked mon.noname-b con 0x7f5d981854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:36.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9807f880 con 0x7f5d98085d40 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9807cba0 con 0x7f5d981854c0 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d9807fb60 con 0x7f5d98085970 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient(hunting): authenticate will time out at 1865.272494s 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d96575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5d98085970 0x7f5d9807bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:51194/0 (socket says 10.20.193.102:51194) 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d96575640 1 -- 10.20.193.102:0/1220301722 learned_addr learned my addr 10.20.193.102:0/1220301722 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d95d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5d98085d40 0x7f5d9807c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44988/0 (socket says 10.20.193.102:44988) 2026-02-21T22:27:36.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d96d76640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f5d981854c0 legacy=0x7f5d98184ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d96575640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.102:6790/0 conn(0x7f5d98085970 legacy=0x7f5d9807bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d95d74640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.102:6789/0 conn(0x7f5d98085d40 legacy=0x7f5d9807c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935017577 0 0) 0x7f5d9807f880 con 0x7f5d98085d40 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): my global_id is 34635 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d64003620 con 0x7f5d98085d40 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144333380 0 0) 0x7f5d9807fb60 con 0x7f5d98085970 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _negotiate 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): my global_id is 94148 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d9807f880 con 0x7f5d98085970 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2939973419 0 0) 0x7f5d64003620 con 0x7f5d98085d40 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5d9807fb60 con 0x7f5d98085d40 2026-02-21T22:27:36.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d800044d0 con 0x7f5d98085d40 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): dump: 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 375265878 0 0) 0x7f5d9807f880 con 0x7f5d98085970 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5d64003620 con 0x7f5d98085970 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d8c003490 con 0x7f5d98085970 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient(hunting): dump: 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:36.039 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 20384111 0 0) 0x7f5d9807fb60 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.102:6790/0 conn(0x7f5d98085970 legacy=0x7f5d9807bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.178:6789/0 conn(0x7f5d981854c0 legacy=0x7f5d98184ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: _finish_hunting 0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 monclient: found mon.a 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d98089390 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: _finish_auth 0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: _check_auth_tickets 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 5 monclient: authenticate success, global_id 34635 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _renew_subs 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5d981852b0 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _renew_subs 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5d9807f5a0 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5d80004810 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5d80004b70 con 0x7f5d98085d40 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d7effd640 10 monclient: dump: 2026-02-21T22:27:36.040 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d94d72640 4 set_mon_vals no callback set 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.036+0000 7f5d9c8c9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d9c8c9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9808f100 con 0x7f5d98085d40 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5d80004e40 con 0x7f5d98085d40 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f5d8004c6d0 con 0x7f5d98085d40 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d96575640 10 monclient: get_auth_request con 0x7f5d6403d610 auth_method 0 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5d800176b0 con 0x7f5d98085d40 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d7effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:36.041 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.040+0000 7f5d7effd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:36.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5d98062ce0 con 0x7f5d98085d40 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d7effd640 1 -- 10.20.193.102:0/1220301722 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 798496654) 0x7f5d80016f90 con 0x7f5d98085d40 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d7effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d7effd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:36.158 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:27:36.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5d6403d610 legacy=0x7f5d6405dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:36.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 10 monclient: shutdown 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 >> v1:10.20.193.102:6789/0 conn(0x7f5d98085d40 legacy=0x7f5d9807c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 shutdown_connections 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 >> 10.20.193.102:0/1220301722 conn(0x7f5d98054e40 msgr2=0x7f5d98087d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 shutdown_connections 2026-02-21T22:27:36.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:36.156+0000 7f5d9c8c9640 1 -- 10.20.193.102:0/1220301722 wait complete. 2026-02-21T22:27:38.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:38.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:39.168 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: get_monmap_and_config 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: build_initial_monmap 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: monmap: 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:39.231525+0000 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:39.231525+0000 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:39.231 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 1 Processor -- start 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 1 -- start start 2026-02-21T22:27:39.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: init 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-a con 0x7fbb3c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-c con 0x7fbb3c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-b con 0x7fbb3c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c086a10 con 0x7fbb3c08ce00 2026-02-21T22:27:39.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c08b130 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c08bd60 con 0x7fbb3c08d8e0 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb41c5f640 10 monclient(hunting): authenticate will time out at 1868.467733s 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb3b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08ce00 0x7fbb3c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44998/0 (socket says 10.20.193.102:44998) 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb3b7fe640 1 -- 10.20.193.102:0/3279750992 learned_addr learned my addr 10.20.193.102:0/3279750992 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.228+0000 7fbb3affd640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fbb3c08b7f0 legacy=0x7fbb3c181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3b7fe640 1 -- 10.20.193.102:0/3279750992 >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08ce00 legacy=0x7fbb3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3bfff640 1 -- 10.20.193.102:0/3279750992 >> v1:10.20.193.102:6790/0 conn(0x7fbb3c08d8e0 legacy=0x7fbb3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:39.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471474784 0 0) 0x7fbb3c086a10 con 0x7fbb3c08ce00 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient(hunting): my global_id is 34638 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb20003620 con 0x7fbb3c08ce00 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 48111087 0 0) 0x7fbb20003620 con 0x7fbb3c08ce00 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 >> v1:10.20.193.102:6790/0 conn(0x7fbb3c08d8e0 legacy=0x7fbb3c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 >> v1:10.20.193.178:6789/0 conn(0x7fbb3c08b7f0 legacy=0x7fbb3c181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 monclient: found mon.noname-a 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:39.239 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c08b370 con 0x7fbb3c08ce00 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 5 monclient: authenticate success, global_id 34638 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbb28003110 con 0x7fbb3c08ce00 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: dump: 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:39.240 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbb280033e0 con 0x7fbb3c08ce00 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 1 -- 10.20.193.102:0/3279750992 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbb28004b50 con 0x7fbb3c08ce00 2026-02-21T22:27:39.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb3a7fc640 10 monclient: dump: 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:39.242 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 4 set_mon_vals no callback set 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 10 monclient: shutdown 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- 10.20.193.102:0/3279750992 >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08ce00 legacy=0x7fbb3c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- 10.20.193.102:0/3279750992 shutdown_connections 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- 10.20.193.102:0/3279750992 >> 10.20.193.102:0/3279750992 conn(0x7fbb3c091a50 msgr2=0x7fbb3c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- 10.20.193.102:0/3279750992 shutdown_connections 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- 10.20.193.102:0/3279750992 wait complete. 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 10 monclient: build_initial_monmap 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 10 monclient: monmap: 2026-02-21T22:27:39.243 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 Processor -- start 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 1 -- start start 2026-02-21T22:27:39.244 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.232+0000 7fbb41c5f640 10 monclient: init 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-a con 0x7fbb3c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-c con 0x7fbb3c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient(hunting): picked mon.noname-b con 0x7fbb3c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c082ea0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c058cd0 con 0x7fbb3c08d8e0 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3c088a70 con 0x7fbb3c08ce00 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient(hunting): authenticate will time out at 1868.472317s 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb3b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08b7f0 0x7fbb3c1535e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:45000/0 (socket says 10.20.193.102:45000) 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb3b7fe640 1 -- 10.20.193.102:0/1065168887 learned_addr learned my addr 10.20.193.102:0/1065168887 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:39.245 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb3bfff640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.178:6789/0 conn(0x7fbb3c08d8e0 legacy=0x7fbb3c1959b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:39.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb3b7fe640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08b7f0 legacy=0x7fbb3c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:39.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb3affd640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.102:6790/0 conn(0x7fbb3c08ce00 legacy=0x7fbb3c194290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:39.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032334733 0 0) 0x7fbb3c082ea0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): my global_id is 34641 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb0c003620 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 343530605 0 0) 0x7fbb0c003620 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbb3c082ea0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbb280048e0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): dump: 2026-02-21T22:27:39.249 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:39.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2982560818 0 0) 0x7fbb3c082ea0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.102:6790/0 conn(0x7fbb3c08ce00 legacy=0x7fbb3c194290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.178:6789/0 conn(0x7fbb3c08d8e0 legacy=0x7fbb3c1959b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: _finish_hunting 0 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 monclient: found mon.a 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c05d9b0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: _finish_auth 0 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: _check_auth_tickets 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 5 monclient: authenticate success, global_id 34641 2026-02-21T22:27:39.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _renew_subs 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c082bc0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _renew_subs 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbb3c153cf0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbb28003380 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbb28005320 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 10 monclient: dump: 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:39.252 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb39ffb640 4 set_mon_vals no callback set 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fbb280119b0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c0008e0 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.236+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7fbb2804c790 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.240+0000 7fbb3affd640 10 monclient: get_auth_request con 0x7fbb0c03d3e0 auth_method 0 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.240+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fbb28017770 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.240+0000 7fbb17fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:39.254 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.240+0000 7fbb17fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.352+0000 7fbb41c5f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbb3c090810 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb17fff640 1 -- 10.20.193.102:0/1065168887 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1240956590) 0x7fbb28017050 con 0x7fbb3c08b7f0 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb17fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:39.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb17fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:39.357 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.178:6800/2549949439 conn(0x7fbb0c03d3e0 legacy=0x7fbb0c05d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 10 monclient: shutdown 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 >> v1:10.20.193.102:6789/0 conn(0x7fbb3c08b7f0 legacy=0x7fbb3c1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 shutdown_connections 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 >> 10.20.193.102:0/1065168887 conn(0x7fbb3c091a50 msgr2=0x7fbb3c087b60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:39.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 shutdown_connections 2026-02-21T22:27:39.359 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:39.356+0000 7fbb41c5f640 1 -- 10.20.193.102:0/1065168887 wait complete. 2026-02-21T22:27:42.367 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:27:42.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: get_monmap_and_config 2026-02-21T22:27:42.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: build_initial_monmap 2026-02-21T22:27:42.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: monmap: 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:42.438634+0000 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:42.438634+0000 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:42.441 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:42.442 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:42.442 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:42.442 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:42.442 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:42.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 1 Processor -- start 2026-02-21T22:27:42.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 1 -- start start 2026-02-21T22:27:42.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: init 2026-02-21T22:27:42.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:42.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:42.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-c con 0x7f393c08e9d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-a con 0x7f393c08f4b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-b con 0x7f393c1854e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c091cd0 con 0x7f393c08f4b0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c05e2c0 con 0x7f393c08e9d0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c08cba0 con 0x7f393c1854e0 2026-02-21T22:27:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:42.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3942f5f640 10 monclient(hunting): authenticate will time out at 1871.674927s 2026-02-21T22:27:42.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3940cd4640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f393c08eda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:42.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3933fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f393c08f4b0 0x7f393c183dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44860/0 (socket says 10.20.193.102:44860) 2026-02-21T22:27:42.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.436+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 learned_addr learned my addr 10.20.193.102:0/4255846059 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:42.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.440+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f393c08f4b0 legacy=0x7f393c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:42.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.440+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f393c1a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:42.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:42.640+0000 7f3940cd4640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f393c08eda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:43.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:43.044+0000 7f3940cd4640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f393c08eda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:43.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:43.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:43.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:43.848+0000 7f3940cd4640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f393c08eda0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:45.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): tick 2026-02-21T22:27:45.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 monclient(hunting): continuing hunt 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f393c1a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f393c08eda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f393c08f4b0 legacy=0x7f393c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-b con 0x7f393c1854e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-c con 0x7f393c08e9d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-a con 0x7f393c08f4b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:45.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c08f8d0 con 0x7f393c08f4b0 2026-02-21T22:27:45.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c05e2c0 con 0x7f393c08e9d0 2026-02-21T22:27:45.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392006a090 con 0x7f393c1854e0 2026-02-21T22:27:45.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f39327fc640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:45.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.436+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:45.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.440+0000 7f3940cd4640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f39200238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:45.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.440+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f393c08f4b0 legacy=0x7f3920066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:45.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:45.640+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:46.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:46.044+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:46.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:46.848+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:48.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): tick 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 monclient(hunting): continuing hunt 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f39200238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f393c08f4b0 legacy=0x7f3920066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-a con 0x7f3920068160 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:48.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-b con 0x7f393c1854e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): picked mon.noname-c con 0x7f393c08e9d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392006b330 con 0x7f3920068160 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c05e2c0 con 0x7f393c08e9d0 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392006abe0 con 0x7f393c1854e0 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39327fc640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:48.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.436+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:48.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.440+0000 7f3940cd4640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f3920068160 legacy=0x7f392006a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:48.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.440+0000 7f3933fff640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f39200238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:48.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:48.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:48.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:48.640+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:49.044 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:49.044+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:49.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:49.848+0000 7f39414d5640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535457909 0 0) 0x7f392006abe0 con 0x7f393c1854e0 2026-02-21T22:27:50.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.247 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient(hunting): my global_id is 94151 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3928003620 con 0x7f393c1854e0 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1350998805 0 0) 0x7f3928003620 con 0x7f393c1854e0 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.178:6789/0 conn(0x7f393c08e9d0 legacy=0x7f39200450a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6789/0 conn(0x7f3920068160 legacy=0x7f392006a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 monclient: found mon.noname-b 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392006ae70 con 0x7f393c1854e0 2026-02-21T22:27:50.248 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 5 monclient: authenticate success, global_id 94151 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39240030f0 con 0x7f393c1854e0 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: dump: 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.249 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3933fff640 1 --1- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 0x7f39200238f0 :-1 s=OPENED pgs=403 gs=16 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f3924004b50 mon_map magic: 0 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3924003430 con 0x7f393c1854e0 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:50.250 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 4 set_mon_vals no callback set 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: shutdown 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- 10.20.193.102:0/4255846059 >> v1:10.20.193.102:6790/0 conn(0x7f393c1854e0 legacy=0x7f39200238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 1 -- 10.20.193.102:0/4255846059 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3924004b50 con 0x7f393c1854e0 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39337fe640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- 10.20.193.102:0/4255846059 shutdown_connections 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- 10.20.193.102:0/4255846059 >> 10.20.193.102:0/4255846059 conn(0x7f393c055730 msgr2=0x7f393c091010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- 10.20.193.102:0/4255846059 shutdown_connections 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- 10.20.193.102:0/4255846059 wait complete. 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: build_initial_monmap 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: monmap: 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:50.251 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 Processor -- start 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- start start 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: init 2026-02-21T22:27:50.252 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-b con 0x7f3920068160 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-c con 0x7f393c08e9d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient(hunting): picked mon.noname-a con 0x7f393c1854e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c12ce10 con 0x7f393c1854e0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c08bf60 con 0x7f3920068160 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c089460 con 0x7f393c08e9d0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3942f5f640 10 monclient(hunting): authenticate will time out at 1879.483047s 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f3940cd4640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f3920068160 legacy=0x7f393c130a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39414d5640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f393c1854e0 0x7f393c131af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44890/0 (socket says 10.20.193.102:44890) 2026-02-21T22:27:50.253 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.244+0000 7f39414d5640 1 -- 10.20.193.102:0/2768272982 learned_addr learned my addr 10.20.193.102:0/2768272982 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:50.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39414d5640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.102:6789/0 conn(0x7f393c1854e0 legacy=0x7f393c131af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.257 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3933fff640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.102:6790/0 conn(0x7f393c08e9d0 legacy=0x7f393c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933946198 0 0) 0x7f393c12ce10 con 0x7f393c1854e0 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): my global_id is 34644 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003620 con 0x7f393c1854e0 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057957923 0 0) 0x7f393c089460 con 0x7f393c08e9d0 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): my global_id is 94154 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f393c12ce10 con 0x7f393c08e9d0 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2312003555 0 0) 0x7f390c003620 con 0x7f393c1854e0 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f393c089460 con 0x7f393c1854e0 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3938004830 con 0x7f393c1854e0 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): dump: 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.259 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1288991396 0 0) 0x7f393c12ce10 con 0x7f393c08e9d0 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f390c003620 con 0x7f393c08e9d0 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39240030f0 con 0x7f393c08e9d0 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:50.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient(hunting): dump: 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3866372727 0 0) 0x7f393c089460 con 0x7f393c1854e0 2026-02-21T22:27:50.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.102:6790/0 conn(0x7f393c08e9d0 legacy=0x7f393c183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.178:6789/0 conn(0x7f3920068160 legacy=0x7f393c130a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 monclient: found mon.a 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c141270 con 0x7f393c1854e0 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 5 monclient: authenticate success, global_id 34644 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _renew_subs 2026-02-21T22:27:50.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f393c12cb30 con 0x7f393c1854e0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _renew_subs 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f393c12a000 con 0x7f393c1854e0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3938004ed0 con 0x7f393c1854e0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3938005230 con 0x7f393c1854e0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 10 monclient: dump: 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.263 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3932ffd640 4 set_mon_vals no callback set 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f39380054c0 con 0x7f393c1854e0 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393c108d10 con 0x7f393c1854e0 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f393804dae0 con 0x7f393c1854e0 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.248+0000 7f3940cd4640 10 monclient: get_auth_request con 0x7f390c03d570 auth_method 0 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.252+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f3938018a60 con 0x7f393c1854e0 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.252+0000 7f39317fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.252+0000 7f39317fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:50.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:27:50.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:50.368 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f393c108a30 con 0x7f393c1854e0 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f39317fa640 1 -- 10.20.193.102:0/2768272982 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 2224803118) 0x7f393801dcc0 con 0x7f393c1854e0 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f39317fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f39317fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:27:50.369 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.178:6800/2549949439 conn(0x7f390c03d570 legacy=0x7f390c05da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 10 monclient: shutdown 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 >> v1:10.20.193.102:6789/0 conn(0x7f393c1854e0 legacy=0x7f393c131af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 shutdown_connections 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 >> 10.20.193.102:0/2768272982 conn(0x7f393c055730 msgr2=0x7f393c109210 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 shutdown_connections 2026-02-21T22:27:50.370 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.368+0000 7f3942f5f640 1 -- 10.20.193.102:0/2768272982 wait complete. 2026-02-21T22:27:50.379 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:27:50.379 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:27:50.379 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: get_monmap_and_config 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: build_initial_monmap 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: monmap: 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.441 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:50.440893+0000 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:50.440893+0000 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 1 Processor -- start 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 1 -- start start 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: init 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:50.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient(hunting): picked mon.noname-b con 0x7f227408ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient(hunting): picked mon.noname-a con 0x7f227408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient(hunting): picked mon.noname-c con 0x7f227408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2274086a10 con 0x7f227408b7f0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227408b130 con 0x7f227408d8e0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227408bd60 con 0x7f227408ce00 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient(hunting): authenticate will time out at 1879.676823s 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227a5fc640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f227408d8e0 legacy=0x7f22741a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f22795fa640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f227408b7f0 0x7f2274181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44900/0 (socket says 10.20.193.102:44900) 2026-02-21T22:27:50.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f22795fa640 1 -- 10.20.193.102:0/1632311406 learned_addr learned my addr 10.20.193.102:0/1632311406 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:50.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2279dfb640 1 --1- 10.20.193.102:0/1632311406 >> v1:10.20.193.102:6790/0 conn(0x7f227408ce00 0x7f227408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60980/0 (socket says 10.20.193.102:60980) 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f22795fa640 1 -- 10.20.193.102:0/1632311406 >> v1:10.20.193.102:6789/0 conn(0x7f227408b7f0 legacy=0x7f2274181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2279dfb640 1 -- 10.20.193.102:0/1632311406 >> v1:10.20.193.102:6790/0 conn(0x7f227408ce00 legacy=0x7f227408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859788488 0 0) 0x7f2274086a10 con 0x7f227408b7f0 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): my global_id is 34647 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f225c003620 con 0x7f227408b7f0 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488507973 0 0) 0x7f227408bd60 con 0x7f227408ce00 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient(hunting): my global_id is 94157 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2274086a10 con 0x7f227408ce00 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 998669392 0 0) 0x7f225c003620 con 0x7f227408b7f0 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 >> v1:10.20.193.102:6790/0 conn(0x7f227408ce00 legacy=0x7f227408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 >> v1:10.20.193.178:6789/0 conn(0x7f227408d8e0 legacy=0x7f22741a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:50.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 monclient: found mon.noname-a 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227408b370 con 0x7f227408b7f0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 5 monclient: authenticate success, global_id 34647 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2268003110 con 0x7f227408b7f0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: dump: 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f22680033e0 con 0x7f227408b7f0 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 1 -- 10.20.193.102:0/1632311406 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2268004b50 con 0x7f227408b7f0 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f2278df9640 10 monclient: dump: 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.456 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.440+0000 7f227c086640 4 set_mon_vals no callback set 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: shutdown 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1632311406 >> v1:10.20.193.102:6789/0 conn(0x7f227408b7f0 legacy=0x7f2274181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1632311406 shutdown_connections 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1632311406 >> 10.20.193.102:0/1632311406 conn(0x7f2274091a50 msgr2=0x7f2274091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1632311406 shutdown_connections 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1632311406 wait complete. 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: build_initial_monmap 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: monmap: 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:50.458 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 Processor -- start 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- start start 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: init 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient(hunting): picked mon.noname-b con 0x7f227408b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient(hunting): picked mon.noname-a con 0x7f227408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient(hunting): picked mon.noname-c con 0x7f227408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2274080ec0 con 0x7f227408ce00 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227405e2d0 con 0x7f227408b7f0 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2274088a70 con 0x7f227408d8e0 2026-02-21T22:27:50.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient(hunting): authenticate will time out at 1879.681084s 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f2279dfb640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f227408b7f0 legacy=0x7f2274153030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22795fa640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f227408ce00 0x7f2274084750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44912/0 (socket says 10.20.193.102:44912) 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22795fa640 1 -- 10.20.193.102:0/1211224673 learned_addr learned my addr 10.20.193.102:0/1211224673 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22795fa640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227408ce00 legacy=0x7f2274084750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227a5fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6790/0 conn(0x7f227408d8e0 legacy=0x7f22741953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442551057 0 0) 0x7f2274080ec0 con 0x7f227408ce00 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): my global_id is 34650 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2244003620 con 0x7f227408ce00 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271112599 0 0) 0x7f2274088a70 con 0x7f227408d8e0 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): my global_id is 94160 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2274080ec0 con 0x7f227408d8e0 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 395522667 0 0) 0x7f2244003620 con 0x7f227408ce00 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2274088a70 con 0x7f227408ce00 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2268002b70 con 0x7f227408ce00 2026-02-21T22:27:50.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): dump: 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227a5fc640 1 --1- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6790/0 conn(0x7f227408d8e0 0x7f22741953a0 :-1 s=OPENED pgs=407 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f22700058f0 mon_map magic: 0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3805048753 0 0) 0x7f2274080ec0 con 0x7f227408d8e0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2244003620 con 0x7f227408d8e0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1087769398 0 0) 0x7f2274088a70 con 0x7f227408ce00 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6790/0 conn(0x7f227408d8e0 legacy=0x7f22741953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.178:6789/0 conn(0x7f227408b7f0 legacy=0x7f2274153030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 monclient: found mon.a 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2274083220 con 0x7f227408ce00 2026-02-21T22:27:50.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f22700058f0 con 0x7f227408d8e0 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f22627fc640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 5 monclient: authenticate success, global_id 34650 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _renew_subs 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2274082980 con 0x7f227408ce00 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _renew_subs 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.444+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2274084e60 con 0x7f227408ce00 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f22680033c0 con 0x7f227408ce00 2026-02-21T22:27:50.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2268005190 con 0x7f227408ce00 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: dump: 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.465 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f2263fff640 4 set_mon_vals no callback set 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f227c086640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f22740008e0 con 0x7f227407b4f0 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2268011820 con 0x7f227408ce00 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f226804c6a0 con 0x7f227408ce00 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f227a5fc640 10 monclient: get_auth_request con 0x7f224403d540 auth_method 0 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f2279dfb640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b4f0 legacy=0x7f227404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2879771598 0 0) 0x7f22740008e0 con 0x7f227407b4f0 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient(hunting): my global_id is 34653 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2274088a70 con 0x7f227407b4f0 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3690001868 0 0) 0x7f2274088a70 con 0x7f227407b4f0 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f22740008e0 con 0x7f227407b4f0 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2264003230 con 0x7f227407b4f0 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: dump: 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.467 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3164687093 0 0) 0x7f22740008e0 con 0x7f227407b4f0 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f227404d990 con 0x7f227407b4f0 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f22640048e0 con 0x7f227407b4f0 2026-02-21T22:27:50.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b4f0 legacy=0x7f227404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.448+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b4f0 legacy=0x7f227404d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:50.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.452+0000 7f227c086640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.452+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2274090810 con 0x7f227407b8c0 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f2279dfb640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b8c0 legacy=0x7f2274197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2635170827 0 0) 0x7f2274090810 con 0x7f227407b8c0 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient(hunting): my global_id is 34656 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22740008e0 con 0x7f227407b8c0 2026-02-21T22:27:50.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3691858059 0 0) 0x7f22740008e0 con 0x7f227407b8c0 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2274090810 con 0x7f227407b8c0 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2264004ee0 con 0x7f227407b8c0 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient: dump: 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.472 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1283580952 0 0) 0x7f2274090810 con 0x7f227407b8c0 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f22740909e0 con 0x7f227407b8c0 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 2128362251) 0x7f2264009250 con 0x7f227407b8c0 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b8c0 legacy=0x7f2274197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f22627fc640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227407b8c0 legacy=0x7f2274197ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 50, 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 619325, 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:50.474 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.475 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.476 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:50.477 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:50.478 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.479 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.480 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.481 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.482 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:50.483 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:27:50.484 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:50.485 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.178:6800/2549949439 conn(0x7f224403d540 legacy=0x7f224405d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 10 monclient: shutdown 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 >> v1:10.20.193.102:6789/0 conn(0x7f227408ce00 legacy=0x7f2274084750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227a5fc640 1 -- 10.20.193.102:0/1211224673 reap_dead start 2026-02-21T22:27:50.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227a5fc640 1 -- 10.20.193.102:0/1211224673 reap_dead start 2026-02-21T22:27:50.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 shutdown_connections 2026-02-21T22:27:50.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 >> 10.20.193.102:0/1211224673 conn(0x7f2274091a50 msgr2=0x7f227405c930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 shutdown_connections 2026-02-21T22:27:50.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.456+0000 7f227c086640 1 -- 10.20.193.102:0/1211224673 wait complete. 2026-02-21T22:27:50.487 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:27:50.487 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:27:50.487 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 10 monclient: get_monmap_and_config 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 10 monclient: build_initial_monmap 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 10 monclient: monmap: 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.548 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:27:50.548297+0000 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:27:50.548297+0000 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 1 Processor -- start 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 1 -- start start 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.544+0000 7f9c59b15640 10 monclient: init 2026-02-21T22:27:50.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-a con 0x7f9c5408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-b con 0x7f9c5408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-c con 0x7f9c5408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c54086a10 con 0x7f9c5408ce00 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5408b130 con 0x7f9c5408d8e0 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5408bd60 con 0x7f9c5408b7f0 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient(hunting): authenticate will time out at 1879.784193s 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c537fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 0x7f9c5408d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:44946/0 (socket says 10.20.193.102:44946) 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c52ffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 0x7f9c54181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60984/0 (socket says 10.20.193.102:60984) 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c52ffd640 1 -- 10.20.193.102:0/3232179789 learned_addr learned my addr 10.20.193.102:0/3232179789 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:50.550 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c53fff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f9c5408d8e0 legacy=0x7f9c541a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.556 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c537fe640 1 -- 10.20.193.102:0/3232179789 >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 legacy=0x7f9c5408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c52ffd640 1 -- 10.20.193.102:0/3232179789 >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 legacy=0x7f9c54181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1770237893 0 0) 0x7f9c54086a10 con 0x7f9c5408ce00 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): my global_id is 34659 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c38003620 con 0x7f9c5408ce00 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393339645 0 0) 0x7f9c5408bd60 con 0x7f9c5408b7f0 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.557 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient(hunting): my global_id is 94163 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c54086a10 con 0x7f9c5408b7f0 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341955600 0 0) 0x7f9c38003620 con 0x7f9c5408ce00 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 legacy=0x7f9c54181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 >> v1:10.20.193.178:6789/0 conn(0x7f9c5408d8e0 legacy=0x7f9c541a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 monclient: found mon.noname-a 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5408b370 con 0x7f9c5408ce00 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 5 monclient: authenticate success, global_id 34659 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c40003110 con 0x7f9c5408ce00 2026-02-21T22:27:50.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: dump: 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.559 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c400033e0 con 0x7f9c5408ce00 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 1 -- 10.20.193.102:0/3232179789 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c40004a90 con 0x7f9c5408ce00 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c527fc640 10 monclient: dump: 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.560 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: get_monmap_and_config success 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 4 set_mon_vals no callback set 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: shutdown 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- 10.20.193.102:0/3232179789 >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 legacy=0x7f9c5408d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- 10.20.193.102:0/3232179789 shutdown_connections 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- 10.20.193.102:0/3232179789 >> 10.20.193.102:0/3232179789 conn(0x7f9c54091a50 msgr2=0x7f9c54091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- 10.20.193.102:0/3232179789 shutdown_connections 2026-02-21T22:27:50.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- 10.20.193.102:0/3232179789 wait complete. 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: build_initial_monmap 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: monmap: 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 Processor -- start 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 1 -- start start 2026-02-21T22:27:50.562 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.548+0000 7f9c59b15640 10 monclient: init 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _reopen_session rank -1 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-c con 0x7f9c5408b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-a con 0x7f9c5408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient(hunting): picked mon.noname-b con 0x7f9c5408d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c54058410 con 0x7f9c5408ce00 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c54140180 con 0x7f9c5408d8e0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c540586f0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient(hunting): _renew_subs 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient(hunting): authenticate will time out at 1879.788264s 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c537fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 0x7f9c5413f360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:60992/0 (socket says 10.20.193.102:60992) 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c537fe640 1 -- 10.20.193.102:0/3372370234 learned_addr learned my addr 10.20.193.102:0/3372370234 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c53fff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.178:6789/0 conn(0x7f9c5408d8e0 legacy=0x7f9c54199fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c537fe640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 legacy=0x7f9c5413f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.563 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c52ffd640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 legacy=0x7f9c5413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355248371 0 0) 0x7f9c540586f0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): my global_id is 94166 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c24003620 con 0x7f9c5408b7f0 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657306926 0 0) 0x7f9c54058410 con 0x7f9c5408ce00 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): my global_id is 34662 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c540586f0 con 0x7f9c5408ce00 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c52ffd640 1 --1- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 0x7f9c5413fa70 :-1 s=OPENED pgs=509 gs=2 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f9c44002e70 mon_map magic: 0 2026-02-21T22:27:50.564 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 223473596 0 0) 0x7f9c540586f0 con 0x7f9c5408ce00 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c54058410 con 0x7f9c5408ce00 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 94735297 0 0) 0x7f9c24003620 con 0x7f9c5408b7f0 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c540586f0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c400048e0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): dump: 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.565 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c44002e70 con 0x7f9c5408ce00 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient(hunting): dump: 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.566 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4058281982 0 0) 0x7f9c540586f0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.178:6789/0 conn(0x7f9c5408d8e0 legacy=0x7f9c54199fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6789/0 conn(0x7f9c5408ce00 legacy=0x7f9c5413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: _finish_hunting 0 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 monclient: found mon.c 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c54089eb0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: _finish_auth 0 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: _check_auth_tickets 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 236553457 0 0) 0x7f9c54058410 con 0x7f9c5408ce00 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:27:50.567 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 5 monclient: authenticate success, global_id 94166 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _renew_subs 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5405e0d0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _renew_subs 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9c54058130 con 0x7f9c5408b7f0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9c40003380 con 0x7f9c5408b7f0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c40005220 con 0x7f9c5408b7f0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 10 monclient: dump: 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.568 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c51ffb640 4 set_mon_vals no callback set 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9c54085210 con 0x7f9c54084e40 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9c400118b0 con 0x7f9c5408b7f0 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.552+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(583..583 src has 395..583) ==== 3959+0+0 (unknown 1435650434 0 0) 0x7f9c4004c760 con 0x7f9c5408b7f0 2026-02-21T22:27:50.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c53fff640 10 monclient: get_auth_request con 0x7f9c2403d510 auth_method 0 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c52ffd640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c54084e40 legacy=0x7f9c5407b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040067240 0 0) 0x7f9c54085210 con 0x7f9c54084e40 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient(hunting): my global_id is 94169 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c54058410 con 0x7f9c54084e40 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 577085854 0 0) 0x7f9c54058410 con 0x7f9c54084e40 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c54085210 con 0x7f9c54084e40 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c44004f30 con 0x7f9c54084e40 2026-02-21T22:27:50.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient: dump: 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3634138764 0 0) 0x7f9c54085210 con 0x7f9c54084e40 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c54141530 con 0x7f9c54084e40 2026-02-21T22:27:50.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f9c44007250 con 0x7f9c54084e40 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c54084e40 legacy=0x7f9c5407b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.556+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c54084e40 legacy=0x7f9c5407b4f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.560+0000 7f9c59b15640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.560+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9c54062ce0 con 0x7f9c5404d280 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c52ffd640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c5404d280 legacy=0x7f9c541a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855786206 0 0) 0x7f9c54062ce0 con 0x7f9c5404d280 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient(hunting): _negotiate 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient(hunting): my global_id is 94172 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c54085210 con 0x7f9c5404d280 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3969231256 0 0) 0x7f9c54085210 con 0x7f9c5404d280 2026-02-21T22:27:50.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9c54062ce0 con 0x7f9c5404d280 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9c44005580 con 0x7f9c5404d280 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient: dump: 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:27:50.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 930961998 0 0) 0x7f9c54062ce0 con 0x7f9c5404d280 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9c54090810 con 0x7f9c5404d280 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4502 (unknown 0 0 2072904835) 0x7f9c44006900 con 0x7f9c5404d280 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c5404d280 legacy=0x7f9c541a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c2ffff640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c5404d280 legacy=0x7f9c541a3ce0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:27:50.576 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 50, 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 16839, 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.577 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.578 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:27:50.579 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:27:50.580 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.581 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.582 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:27:50.583 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.584 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:27:50.585 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9c2403d510 legacy=0x7f9c2405d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 10 monclient: shutdown 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 >> v1:10.20.193.102:6790/0 conn(0x7f9c5408b7f0 legacy=0x7f9c5413f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c53fff640 1 -- 10.20.193.102:0/3372370234 reap_dead start 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 shutdown_connections 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 >> 10.20.193.102:0/3372370234 conn(0x7f9c54091a50 msgr2=0x7f9c5405c800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 shutdown_connections 2026-02-21T22:27:50.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:27:50.564+0000 7f9c59b15640 1 -- 10.20.193.102:0/3372370234 wait complete. 2026-02-21T22:27:50.587 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:27:50.587 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:27:51.232 INFO:tasks.rados.rados.0.trial178.stdout:2761: done (3 left) 2026-02-21T22:27:51.232 INFO:tasks.rados.rados.0.trial178.stdout:2762: write (excl) oid 6 current snap is 280 2026-02-21T22:27:51.232 INFO:tasks.rados.rados.0.trial178.stdout:2762: seq_num 974 ranges {770018=585710,2046285=323733} 2026-02-21T22:27:51.237 INFO:tasks.rados.rados.0.trial178.stdout:2762: writing trial17818531-6 from 770018 to 1355728 tid 1 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2762: writing trial17818531-6 from 2046285 to 2370018 tid 2 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2763: copy_from oid 47 from oid 35 current snap is 280 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2758: finishing write tid 3 to trial17818531-38 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2764: delete oid 43 current snap is 280 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2758: finishing write tid 4 to trial17818531-38 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2758: oid 38 updating version 0 to 1169 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2758: oid 38 version 1169 is already newer than 1168 2026-02-21T22:27:51.242 INFO:tasks.rados.rados.0.trial178.stdout:2758: oid 38 version 1169 is already newer than 1167 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1169 (ObjNum 972 snap 280 seq_num 972) dirty exists 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2758: left oid 38 (ObjNum 972 snap 280 seq_num 972) 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: finishing write tid 1 to trial17818531-24 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: finishing write tid 2 to trial17818531-24 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: finishing write tid 3 to trial17818531-24 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: finishing write tid 4 to trial17818531-24 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: oid 24 updating version 0 to 1567 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: oid 24 updating version 1567 to 1568 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: oid 24 version 1568 is already newer than 1566 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1568 (ObjNum 973 snap 280 seq_num 973) dirty exists 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2759: left oid 24 (ObjNum 973 snap 280 seq_num 973) 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2760: finishing copy_from racing read to trial17818531-27 2026-02-21T22:27:51.243 INFO:tasks.rados.rados.0.trial178.stdout:2760: finishing copy_from to trial17818531-27 2026-02-21T22:27:51.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1026 (ObjNum 920 snap 270 seq_num 920) dirty exists 2026-02-21T22:27:51.244 INFO:tasks.rados.rados.0.trial178.stdout:2762: finishing write tid 1 to trial17818531-6 2026-02-21T22:27:51.244 INFO:tasks.rados.rados.0.trial178.stdout:2758: done (5 left) 2026-02-21T22:27:51.244 INFO:tasks.rados.rados.0.trial178.stdout:2759: done (4 left) 2026-02-21T22:27:51.244 INFO:tasks.rados.rados.0.trial178.stdout:2760: done (3 left) 2026-02-21T22:27:51.245 INFO:tasks.rados.rados.0.trial178.stdout:2764: done (2 left) 2026-02-21T22:27:51.245 INFO:tasks.rados.rados.0.trial178.stdout:2765: write oid 32 current snap is 280 2026-02-21T22:27:51.245 INFO:tasks.rados.rados.0.trial178.stdout:2765: seq_num 976 ranges {403632=736208,1763249=619999} 2026-02-21T22:27:51.250 INFO:tasks.rados.rados.0.trial178.stdout:2765: writing trial17818531-32 from 403632 to 1139840 tid 1 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2765: writing trial17818531-32 from 1763249 to 2383248 tid 2 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2766: read oid 49 snap -1 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2766: expect (ObjNum 945 snap 277 seq_num 945) 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2767: write (excl) oid 50 current snap is 280 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2762: finishing write tid 2 to trial17818531-6 2026-02-21T22:27:51.259 INFO:tasks.rados.rados.0.trial178.stdout:2762: finishing write tid 3 to trial17818531-6 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2762: finishing write tid 4 to trial17818531-6 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2762: oid 6 updating version 0 to 1177 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2762: oid 6 updating version 1177 to 1178 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2762: oid 6 version 1178 is already newer than 1176 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1178 (ObjNum 974 snap 280 seq_num 974) dirty exists 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2762: left oid 6 (ObjNum 974 snap 280 seq_num 974) 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2763: finishing copy_from racing read to trial17818531-47 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2765: finishing write tid 1 to trial17818531-32 2026-02-21T22:27:51.260 INFO:tasks.rados.rados.0.trial178.stdout:2767: seq_num 977 ranges {420439=411457,1421731=480010} 2026-02-21T22:27:51.264 INFO:tasks.rados.rados.0.trial178.stdout:2767: writing trial17818531-50 from 420439 to 831896 tid 1 2026-02-21T22:27:51.271 INFO:tasks.rados.rados.0.trial178.stdout:2767: writing trial17818531-50 from 1421731 to 1901741 tid 2 2026-02-21T22:27:51.272 INFO:tasks.rados.rados.0.trial178.stdout:2762: done (4 left) 2026-02-21T22:27:51.272 INFO:tasks.rados.rados.0.trial178.stdout:2768: write (excl) oid 29 current snap is 280 2026-02-21T22:27:51.272 INFO:tasks.rados.rados.0.trial178.stdout:2768: seq_num 978 ranges {437246=486706,1363860=623668} 2026-02-21T22:27:51.277 INFO:tasks.rados.rados.0.trial178.stdout:2768: writing trial17818531-29 from 437246 to 923952 tid 1 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2768: writing trial17818531-29 from 1363860 to 1987528 tid 2 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2769: read oid 36 snap -1 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2769: expect (ObjNum 964 snap 280 seq_num 964) 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2770: read oid 46 snap -1 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2770: expect (ObjNum 960 snap 279 seq_num 960) 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2771: write (excl) oid 12 current snap is 280 2026-02-21T22:27:51.282 INFO:tasks.rados.rados.0.trial178.stdout:2771: seq_num 979 ranges {454053=561955,1422342=483679,2368404=85649} 2026-02-21T22:27:51.285 INFO:tasks.rados.rados.0.trial178.stdout:2771: writing trial17818531-12 from 454053 to 1016008 tid 1 2026-02-21T22:27:51.289 INFO:tasks.rados.rados.0.trial178.stdout:2771: writing trial17818531-12 from 1422342 to 1906021 tid 2 2026-02-21T22:27:51.291 INFO:tasks.rados.rados.0.trial178.stdout:2771: writing trial17818531-12 from 2368404 to 2454053 tid 3 2026-02-21T22:27:51.291 INFO:tasks.rados.rados.0.trial178.stdout:2772: write (excl) oid 24 current snap is 280 2026-02-21T22:27:51.291 INFO:tasks.rados.rados.0.trial178.stdout:2772: seq_num 980 ranges {470860=637204,1880824=590036} 2026-02-21T22:27:51.295 INFO:tasks.rados.rados.0.trial178.stdout:2772: writing trial17818531-24 from 470860 to 1108064 tid 1 2026-02-21T22:27:51.301 INFO:tasks.rados.rados.0.trial178.stdout:2772: writing trial17818531-24 from 1880824 to 2470860 tid 2 2026-02-21T22:27:51.301 INFO:tasks.rados.rados.0.trial178.stdout:2773: read oid 25 snap -1 2026-02-21T22:27:51.301 INFO:tasks.rados.rados.0.trial178.stdout:2773: expect (ObjNum 961 snap 280 seq_num 961) 2026-02-21T22:27:51.301 INFO:tasks.rados.rados.0.trial178.stdout:2774: rollback oid 15 current snap is 280 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2774: done (10 left) 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2775: snap_create 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2763: finishing copy_from to trial17818531-47 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1315 (ObjNum 933 snap 275 seq_num 933) dirty exists 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: finishing write tid 2 to trial17818531-32 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: finishing write tid 3 to trial17818531-32 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: finishing write tid 4 to trial17818531-32 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: oid 32 updating version 0 to 1365 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: oid 32 version 1365 is already newer than 1364 2026-02-21T22:27:51.302 INFO:tasks.rados.rados.0.trial178.stdout:2765: oid 32 version 1365 is already newer than 1363 2026-02-21T22:27:51.303 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1365 (ObjNum 976 snap 280 seq_num 976) dirty exists 2026-02-21T22:27:51.303 INFO:tasks.rados.rados.0.trial178.stdout:2765: left oid 32 (ObjNum 976 snap 280 seq_num 976) 2026-02-21T22:27:51.303 INFO:tasks.rados.rados.0.trial178.stdout:2767: finishing write tid 1 to trial17818531-50 2026-02-21T22:27:51.303 INFO:tasks.rados.rados.0.trial178.stdout:2766: expect (ObjNum 945 snap 277 seq_num 945) 2026-02-21T22:27:51.324 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3361934 2026-02-21T22:27:51.330 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 312796 2026-02-21T22:27:51.391 INFO:tasks.rados.rados.0.trial178.stdout:2767: finishing write tid 2 to trial17818531-50 2026-02-21T22:27:51.391 INFO:tasks.rados.rados.0.trial178.stdout:2767: finishing write tid 3 to trial17818531-50 2026-02-21T22:27:51.391 INFO:tasks.rados.rados.0.trial178.stdout:2767: finishing write tid 4 to trial17818531-50 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2767: oid 50 updating version 0 to 1180 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2767: oid 50 updating version 1180 to 1181 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2767: oid 50 version 1181 is already newer than 1179 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1181 (ObjNum 977 snap 280 seq_num 977) dirty exists 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2767: left oid 50 (ObjNum 977 snap 280 seq_num 977) 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: finishing write tid 1 to trial17818531-29 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: finishing write tid 2 to trial17818531-29 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: finishing write tid 3 to trial17818531-29 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: finishing write tid 4 to trial17818531-29 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: oid 29 updating version 0 to 1172 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: oid 29 updating version 1172 to 1174 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: oid 29 version 1174 is already newer than 1173 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1174 (ObjNum 978 snap 280 seq_num 978) dirty exists 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2768: left oid 29 (ObjNum 978 snap 280 seq_num 978) 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:2769: expect (ObjNum 964 snap 280 seq_num 964) 2026-02-21T22:27:51.392 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:51.480 INFO:tasks.rados.rados.0.trial178.stdout:2770: expect (ObjNum 960 snap 279 seq_num 960) 2026-02-21T22:27:51.501 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: finishing write tid 1 to trial17818531-12 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: finishing write tid 2 to trial17818531-12 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: finishing write tid 3 to trial17818531-12 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: finishing write tid 4 to trial17818531-12 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: finishing write tid 5 to trial17818531-12 2026-02-21T22:27:51.555 INFO:tasks.rados.rados.0.trial178.stdout:2771: oid 12 updating version 0 to 1447 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2771: oid 12 updating version 1447 to 1448 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2771: oid 12 updating version 1448 to 1450 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2771: oid 12 version 1450 is already newer than 1449 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1450 (ObjNum 979 snap 280 seq_num 979) dirty exists 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2771: left oid 12 (ObjNum 979 snap 280 seq_num 979) 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2772: finishing write tid 1 to trial17818531-24 2026-02-21T22:27:51.556 INFO:tasks.rados.rados.0.trial178.stdout:2773: expect (ObjNum 961 snap 280 seq_num 961) 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: finishing write tid 2 to trial17818531-24 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: finishing write tid 3 to trial17818531-24 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: finishing write tid 4 to trial17818531-24 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: oid 24 updating version 0 to 1576 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: oid 24 updating version 1576 to 1577 2026-02-21T22:27:51.593 INFO:tasks.rados.rados.0.trial178.stdout:2772: oid 24 updating version 1577 to 1578 2026-02-21T22:27:51.594 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1578 (ObjNum 980 snap 280 seq_num 980) dirty exists 2026-02-21T22:27:51.594 INFO:tasks.rados.rados.0.trial178.stdout:2772: left oid 24 (ObjNum 980 snap 280 seq_num 980) 2026-02-21T22:27:52.374 INFO:tasks.rados.rados.0.trial178.stdout:2763: done (10 left) 2026-02-21T22:27:52.374 INFO:tasks.rados.rados.0.trial178.stdout:2765: done (9 left) 2026-02-21T22:27:52.374 INFO:tasks.rados.rados.0.trial178.stdout:2766: done (8 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2767: done (7 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2768: done (6 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2769: done (5 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2770: done (4 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2771: done (3 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2772: done (2 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2773: done (1 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2775: done (0 left) 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2776: write (excl) oid 4 current snap is 281 2026-02-21T22:27:52.375 INFO:tasks.rados.rados.0.trial178.stdout:2776: seq_num 981 ranges {487667=428806,1655659=770995} 2026-02-21T22:27:52.379 INFO:tasks.rados.rados.0.trial178.stdout:2776: writing trial17818531-4 from 487667 to 916473 tid 1 2026-02-21T22:27:52.384 INFO:tasks.rados.rados.0.trial178.stdout:2776: writing trial17818531-4 from 1655659 to 2426654 tid 2 2026-02-21T22:27:52.385 INFO:tasks.rados.rados.0.trial178.stdout:2777: write (excl) oid 45 current snap is 281 2026-02-21T22:27:52.385 INFO:tasks.rados.rados.0.trial178.stdout:2777: seq_num 982 ranges {504474=504055,1714141=631006} 2026-02-21T22:27:52.388 INFO:tasks.rados.rados.0.trial178.stdout:2777: writing trial17818531-45 from 504474 to 1008529 tid 1 2026-02-21T22:27:52.393 INFO:tasks.rados.rados.0.trial178.stdout:2777: writing trial17818531-45 from 1714141 to 2345147 tid 2 2026-02-21T22:27:52.394 INFO:tasks.rados.rados.0.trial178.stdout:2778: write (excl) oid 19 current snap is 281 2026-02-21T22:27:52.394 INFO:tasks.rados.rados.0.trial178.stdout:2778: seq_num 983 ranges {521281=579304,1656270=774664} 2026-02-21T22:27:52.398 INFO:tasks.rados.rados.0.trial178.stdout:2778: writing trial17818531-19 from 521281 to 1100585 tid 1 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2778: writing trial17818531-19 from 1656270 to 2430934 tid 2 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2779: delete oid 48 current snap is 281 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: finishing write tid 1 to trial17818531-4 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: finishing write tid 2 to trial17818531-4 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: finishing write tid 3 to trial17818531-4 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: finishing write tid 4 to trial17818531-4 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: oid 4 updating version 0 to 1453 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: oid 4 version 1453 is already newer than 1451 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: oid 4 version 1453 is already newer than 1452 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1453 (ObjNum 981 snap 281 seq_num 981) dirty exists 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2776: left oid 4 (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2777: finishing write tid 1 to trial17818531-45 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2777: finishing write tid 2 to trial17818531-45 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2777: finishing write tid 3 to trial17818531-45 2026-02-21T22:27:52.403 INFO:tasks.rados.rados.0.trial178.stdout:2777: finishing write tid 4 to trial17818531-45 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:2777: oid 45 updating version 0 to 1314 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:2777: oid 45 version 1314 is already newer than 1312 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:2777: oid 45 version 1314 is already newer than 1313 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1314 (ObjNum 982 snap 281 seq_num 982) dirty exists 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:2777: left oid 45 (ObjNum 982 snap 281 seq_num 982) 2026-02-21T22:27:52.404 INFO:tasks.rados.rados.0.trial178.stdout:2778: finishing write tid 1 to trial17818531-19 2026-02-21T22:27:52.405 INFO:tasks.rados.rados.0.trial178.stdout:2776: done (3 left) 2026-02-21T22:27:52.405 INFO:tasks.rados.rados.0.trial178.stdout:2777: done (2 left) 2026-02-21T22:27:52.405 INFO:tasks.rados.rados.0.trial178.stdout:2779: done (1 left) 2026-02-21T22:27:52.405 INFO:tasks.rados.rados.0.trial178.stdout:2780: write oid 39 current snap is 281 2026-02-21T22:27:52.405 INFO:tasks.rados.rados.0.trial178.stdout:2780: seq_num 985 ranges {554895=729802,1773234=778333} 2026-02-21T22:27:52.410 INFO:tasks.rados.rados.0.trial178.stdout:2780: writing trial17818531-39 from 554895 to 1284697 tid 1 2026-02-21T22:27:52.415 INFO:tasks.rados.rados.0.trial178.stdout:2780: writing trial17818531-39 from 1773234 to 2551567 tid 2 2026-02-21T22:27:52.415 INFO:tasks.rados.rados.0.trial178.stdout:2781: delete oid 3 current snap is 281 2026-02-21T22:27:52.415 INFO:tasks.rados.rados.0.trial178.stdout:2778: finishing write tid 2 to trial17818531-19 2026-02-21T22:27:52.415 INFO:tasks.rados.rados.0.trial178.stdout:2778: finishing write tid 3 to trial17818531-19 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2778: finishing write tid 4 to trial17818531-19 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2778: oid 19 updating version 0 to 1582 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2778: oid 19 version 1582 is already newer than 1580 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2778: oid 19 version 1582 is already newer than 1581 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1582 (ObjNum 983 snap 281 seq_num 983) dirty exists 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2778: left oid 19 (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:52.416 INFO:tasks.rados.rados.0.trial178.stdout:2780: finishing write tid 1 to trial17818531-39 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2778: done (2 left) 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2781: done (1 left) 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2782: read oid 16 snap -1 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2782: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2783: read oid 21 snap -1 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2783: expect (ObjNum 917 snap 270 seq_num 917) 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2784: read oid 42 snap 280 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2784: expect deleted 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2785: write (excl) oid 37 current snap is 281 2026-02-21T22:27:52.418 INFO:tasks.rados.rados.0.trial178.stdout:2785: seq_num 987 ranges {588509=480300,1773845=782002} 2026-02-21T22:27:52.422 INFO:tasks.rados.rados.0.trial178.stdout:2785: writing trial17818531-37 from 588509 to 1068809 tid 1 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2785: writing trial17818531-37 from 1773845 to 2555847 tid 2 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2786: copy_from oid 44 from oid 5 current snap is 281 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2780: finishing write tid 2 to trial17818531-39 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2780: finishing write tid 3 to trial17818531-39 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2780: finishing write tid 4 to trial17818531-39 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2780: oid 39 updating version 0 to 1207 2026-02-21T22:27:52.428 INFO:tasks.rados.rados.0.trial178.stdout:2780: oid 39 version 1207 is already newer than 1206 2026-02-21T22:27:52.429 INFO:tasks.rados.rados.0.trial178.stdout:2780: oid 39 version 1207 is already newer than 1205 2026-02-21T22:27:52.429 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1207 (ObjNum 985 snap 281 seq_num 985) dirty exists 2026-02-21T22:27:52.429 INFO:tasks.rados.rados.0.trial178.stdout:2780: left oid 39 (ObjNum 985 snap 281 seq_num 985) 2026-02-21T22:27:52.429 INFO:tasks.rados.rados.0.trial178.stdout:2782: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:52.429 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:52.480 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:52.511 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:52.542 INFO:tasks.rados.rados.0.trial178.stdout:2783: expect (ObjNum 917 snap 270 seq_num 917) 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2785: finishing write tid 1 to trial17818531-37 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2780: done (5 left) 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2782: done (4 left) 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2783: done (3 left) 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2784: done (2 left) 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2787: copy_from oid 50 from oid 19 current snap is 281 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2785: finishing write tid 2 to trial17818531-37 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2785: finishing write tid 3 to trial17818531-37 2026-02-21T22:27:52.663 INFO:tasks.rados.rados.0.trial178.stdout:2785: finishing write tid 4 to trial17818531-37 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2785: oid 37 updating version 0 to 1318 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2785: oid 37 version 1318 is already newer than 1317 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2785: oid 37 version 1318 is already newer than 1316 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1318 (ObjNum 987 snap 281 seq_num 987) dirty exists 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2785: left oid 37 (ObjNum 987 snap 281 seq_num 987) 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2786: finishing copy_from racing read to trial17818531-44 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2786: finishing copy_from to trial17818531-44 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1437 (ObjNum 908 snap 268 seq_num 908) dirty exists 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2785: done (2 left) 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2786: done (1 left) 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2788: write oid 36 current snap is 281 2026-02-21T22:27:52.664 INFO:tasks.rados.rados.0.trial178.stdout:2788: seq_num 988 ranges {605316=555549,1832327=642013} 2026-02-21T22:27:52.667 INFO:tasks.rados.rados.0.trial178.stdout:2788: writing trial17818531-36 from 605316 to 1160865 tid 1 2026-02-21T22:27:52.672 INFO:tasks.rados.rados.0.trial178.stdout:2788: writing trial17818531-36 from 1832327 to 2474340 tid 2 2026-02-21T22:27:52.672 INFO:tasks.rados.rados.0.trial178.stdout:2789: read oid 49 snap -1 2026-02-21T22:27:52.672 INFO:tasks.rados.rados.0.trial178.stdout:2789: expect (ObjNum 945 snap 277 seq_num 945) 2026-02-21T22:27:52.672 INFO:tasks.rados.rados.0.trial178.stdout:2788: finishing write tid 1 to trial17818531-36 2026-02-21T22:27:52.672 INFO:tasks.rados.rados.0.trial178.stdout:2790: snap_remove snap 280 2026-02-21T22:27:53.236 INFO:tasks.rados.rados.0.trial178.stdout:2790: done (3 left) 2026-02-21T22:27:53.236 INFO:tasks.rados.rados.0.trial178.stdout:2791: write oid 41 current snap is 281 2026-02-21T22:27:53.236 INFO:tasks.rados.rados.0.trial178.stdout:2791: seq_num 989 ranges {622123=747151,2007162=614961} 2026-02-21T22:27:53.245 INFO:tasks.rados.rados.0.trial178.stdout:2791: writing trial17818531-41 from 622123 to 1369274 tid 1 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2791: writing trial17818531-41 from 2007162 to 2622123 tid 2 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2792: read oid 13 snap -1 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2792: expect (ObjNum 956 snap 278 seq_num 956) 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2793: rollback oid 9 current snap is 281 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2793: done (5 left) 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2794: read oid 43 snap -1 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2794: expect deleted 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2795: read oid 26 snap -1 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2795: expect deleted 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2796: write (excl) oid 6 current snap is 281 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2787: finishing copy_from racing read to trial17818531-50 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2787: finishing copy_from to trial17818531-50 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1582 (ObjNum 983 snap 281 seq_num 983) dirty exists 2026-02-21T22:27:53.254 INFO:tasks.rados.rados.0.trial178.stdout:2789: expect (ObjNum 945 snap 277 seq_num 945) 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: finishing write tid 2 to trial17818531-36 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: finishing write tid 3 to trial17818531-36 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: finishing write tid 4 to trial17818531-36 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: oid 36 updating version 0 to 1099 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: oid 36 version 1099 is already newer than 1097 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: oid 36 version 1099 is already newer than 1098 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1099 (ObjNum 988 snap 281 seq_num 988) dirty exists 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2788: left oid 36 (ObjNum 988 snap 281 seq_num 988) 2026-02-21T22:27:53.317 INFO:tasks.rados.rados.0.trial178.stdout:2791: finishing write tid 1 to trial17818531-41 2026-02-21T22:27:53.318 INFO:tasks.rados.rados.0.trial178.stdout:2791: finishing write tid 2 to trial17818531-41 2026-02-21T22:27:53.318 INFO:tasks.rados.rados.0.trial178.stdout:2791: finishing write tid 3 to trial17818531-41 2026-02-21T22:27:53.318 INFO:tasks.rados.rados.0.trial178.stdout:2796: seq_num 990 ranges {638930=422400,1665644=645682} 2026-02-21T22:27:53.324 INFO:tasks.rados.rados.0.trial178.stdout:2796: writing trial17818531-6 from 638930 to 1061330 tid 1 2026-02-21T22:27:53.333 INFO:tasks.rados.rados.0.trial178.stdout:2796: writing trial17818531-6 from 1665644 to 2311326 tid 2 2026-02-21T22:27:53.333 INFO:tasks.rados.rados.0.trial178.stdout:2787: done (7 left) 2026-02-21T22:27:53.333 INFO:tasks.rados.rados.0.trial178.stdout:2788: done (6 left) 2026-02-21T22:27:53.333 INFO:tasks.rados.rados.0.trial178.stdout:2789: done (5 left) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2794: done (4 left) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2795: done (3 left) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2797: rollback oid 18 current snap is 281 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2797: done (3 left) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2798: rollback oid 10 current snap is 281 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2798: done (3 left) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2799: copy_from oid 17 from oid 25 current snap is 281 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2791: finishing write tid 4 to trial17818531-41 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2791: oid 41 updating version 0 to 1073 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2791: oid 41 updating version 1073 to 1075 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2791: oid 41 version 1075 is already newer than 1074 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1075 (ObjNum 989 snap 281 seq_num 989) dirty exists 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2791: left oid 41 (ObjNum 989 snap 281 seq_num 989) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:2792: expect (ObjNum 956 snap 278 seq_num 956) 2026-02-21T22:27:53.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:53.362 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: finishing write tid 1 to trial17818531-6 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: finishing write tid 2 to trial17818531-6 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: finishing write tid 3 to trial17818531-6 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: finishing write tid 4 to trial17818531-6 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: oid 6 updating version 0 to 1181 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: oid 6 version 1181 is already newer than 1179 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: oid 6 version 1181 is already newer than 1180 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1181 (ObjNum 990 snap 281 seq_num 990) dirty exists 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2796: left oid 6 (ObjNum 990 snap 281 seq_num 990) 2026-02-21T22:27:53.378 INFO:tasks.rados.rados.0.trial178.stdout:2799: finishing copy_from racing read to trial17818531-17 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2799: finishing copy_from to trial17818531-17 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1184 (ObjNum 961 snap 280 seq_num 961) dirty exists 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2791: done (3 left) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2792: done (2 left) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2796: done (1 left) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2799: done (0 left) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2800: read oid 27 snap -1 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2800: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2801: read oid 30 snap -1 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2801: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2802: rollback oid 44 current snap is 281 2026-02-21T22:27:53.379 INFO:tasks.rados.rados.0.trial178.stdout:2802: done (2 left) 2026-02-21T22:27:53.380 INFO:tasks.rados.rados.0.trial178.stdout:2803: write oid 15 current snap is 281 2026-02-21T22:27:53.380 INFO:tasks.rados.rados.0.trial178.stdout:2803: seq_num 991 ranges {655737=497649,1607773=789340} 2026-02-21T22:27:53.385 INFO:tasks.rados.rados.0.trial178.stdout:2803: writing trial17818531-15 from 655737 to 1153386 tid 1 2026-02-21T22:27:53.394 INFO:tasks.rados.rados.0.trial178.stdout:2803: writing trial17818531-15 from 1607773 to 2397113 tid 2 2026-02-21T22:27:53.394 INFO:tasks.rados.rados.0.trial178.stdout:2804: read oid 6 snap -1 2026-02-21T22:27:53.394 INFO:tasks.rados.rados.0.trial178.stdout:2804: expect (ObjNum 990 snap 281 seq_num 990) 2026-02-21T22:27:53.394 INFO:tasks.rados.rados.0.trial178.stdout:2805: read oid 5 snap -1 2026-02-21T22:27:53.394 INFO:tasks.rados.rados.0.trial178.stdout:2805: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:53.395 INFO:tasks.rados.rados.0.trial178.stdout:2806: write (excl) oid 2 current snap is 281 2026-02-21T22:27:53.395 INFO:tasks.rados.rados.0.trial178.stdout:2806: seq_num 992 ranges {672544=572898,1666255=532998} 2026-02-21T22:27:53.402 INFO:tasks.rados.rados.0.trial178.stdout:2806: writing trial17818531-2 from 672544 to 1245442 tid 1 2026-02-21T22:27:53.409 INFO:tasks.rados.rados.0.trial178.stdout:2806: writing trial17818531-2 from 1666255 to 2199253 tid 2 2026-02-21T22:27:53.409 INFO:tasks.rados.rados.0.trial178.stdout:2807: write (excl) oid 3 current snap is 281 2026-02-21T22:27:53.409 INFO:tasks.rados.rados.0.trial178.stdout:2807: seq_num 993 ranges {689351=648147,2124737=564614} 2026-02-21T22:27:53.418 INFO:tasks.rados.rados.0.trial178.stdout:2807: writing trial17818531-3 from 689351 to 1337498 tid 1 2026-02-21T22:27:53.423 INFO:tasks.rados.rados.0.trial178.stdout:2807: writing trial17818531-3 from 2124737 to 2689351 tid 2 2026-02-21T22:27:53.423 INFO:tasks.rados.rados.0.trial178.stdout:2808: copy_from oid 25 from oid 44 current snap is 281 2026-02-21T22:27:53.423 INFO:tasks.rados.rados.0.trial178.stdout:2801: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:53.503 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:53.537 INFO:tasks.rados.rados.0.trial178.stdout:2800: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:53.537 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:53.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:53.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:27:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:53.549 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:27:53.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:53.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:53.653 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3457396 2026-02-21T22:27:53.699 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3271077 2026-02-21T22:27:53.711 INFO:tasks.rados.rados.0.trial178.stdout:2803: finishing write tid 1 to trial17818531-15 2026-02-21T22:27:53.711 INFO:tasks.rados.rados.0.trial178.stdout:2804: expect (ObjNum 990 snap 281 seq_num 990) 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: finishing write tid 2 to trial17818531-15 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: finishing write tid 3 to trial17818531-15 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: finishing write tid 4 to trial17818531-15 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: oid 15 updating version 0 to 1185 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: oid 15 updating version 1185 to 1186 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: oid 15 updating version 1186 to 1187 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1187 (ObjNum 991 snap 281 seq_num 991) dirty exists 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2803: left oid 15 (ObjNum 991 snap 281 seq_num 991) 2026-02-21T22:27:53.792 INFO:tasks.rados.rados.0.trial178.stdout:2805: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:53.816 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2304042 2026-02-21T22:27:53.832 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2806: finishing write tid 1 to trial17818531-2 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2806: finishing write tid 2 to trial17818531-2 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2800: done (7 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2801: done (6 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2803: done (5 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2804: done (4 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2805: done (3 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2809: read oid 34 snap -1 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2809: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2810: read oid 30 snap -1 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2810: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2811: rollback oid 26 current snap is 281 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2811: done (5 left) 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2812: write oid 9 current snap is 281 2026-02-21T22:27:53.927 INFO:tasks.rados.rados.0.trial178.stdout:2812: seq_num 994 ranges {706158=723396,2183219=522939} 2026-02-21T22:27:53.932 INFO:tasks.rados.rados.0.trial178.stdout:2812: writing trial17818531-9 from 706158 to 1429554 tid 1 2026-02-21T22:27:53.937 INFO:tasks.rados.rados.0.trial178.stdout:2812: writing trial17818531-9 from 2183219 to 2706158 tid 2 2026-02-21T22:27:53.937 INFO:tasks.rados.rados.0.trial178.stdout:2813: rollback oid 13 current snap is 281 2026-02-21T22:27:53.937 INFO:tasks.rados.rados.0.trial178.stdout:2813: done (6 left) 2026-02-21T22:27:53.937 INFO:tasks.rados.rados.0.trial178.stdout:2814: write (excl) oid 48 current snap is 281 2026-02-21T22:27:53.937 INFO:tasks.rados.rados.0.trial178.stdout:2814: seq_num 995 ranges {722965=798645,2125348=597617} 2026-02-21T22:27:53.942 INFO:tasks.rados.rados.0.trial178.stdout:2814: writing trial17818531-48 from 722965 to 1521610 tid 1 2026-02-21T22:27:53.948 INFO:tasks.rados.rados.0.trial178.stdout:2814: writing trial17818531-48 from 2125348 to 2722965 tid 2 2026-02-21T22:27:53.948 INFO:tasks.rados.rados.0.trial178.stdout:2815: read oid 29 snap -1 2026-02-21T22:27:53.948 INFO:tasks.rados.rados.0.trial178.stdout:2815: expect (ObjNum 978 snap 280 seq_num 978) 2026-02-21T22:27:53.948 INFO:tasks.rados.rados.0.trial178.stdout:2816: snap_create 2026-02-21T22:27:53.948 INFO:tasks.rados.rados.0.trial178.stdout:2806: finishing write tid 3 to trial17818531-2 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2806: finishing write tid 4 to trial17818531-2 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2806: oid 2 updating version 0 to 1182 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2806: oid 2 updating version 1182 to 1183 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2806: oid 2 updating version 1183 to 1184 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1184 (ObjNum 992 snap 281 seq_num 992) dirty exists 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2806: left oid 2 (ObjNum 992 snap 281 seq_num 992) 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: finishing write tid 1 to trial17818531-3 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: finishing write tid 2 to trial17818531-3 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: finishing write tid 3 to trial17818531-3 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: finishing write tid 4 to trial17818531-3 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: oid 3 updating version 0 to 1370 2026-02-21T22:27:53.949 INFO:tasks.rados.rados.0.trial178.stdout:2807: oid 3 updating version 1370 to 1372 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:2807: oid 3 version 1372 is already newer than 1371 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1372 (ObjNum 993 snap 281 seq_num 993) dirty exists 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:2807: left oid 3 (ObjNum 993 snap 281 seq_num 993) 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:2808: finishing copy_from racing read to trial17818531-25 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:2808: finishing copy_from to trial17818531-25 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1437 (ObjNum 908 snap 268 seq_num 908) dirty exists 2026-02-21T22:27:53.950 INFO:tasks.rados.rados.0.trial178.stdout:2810: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:54.039 INFO:tasks.rados.rados.0.trial178.stdout:2809: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:54.056 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:54.087 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1299671 2026-02-21T22:27:54.103 INFO:tasks.rados.rados.0.trial178.stdout:2812: finishing write tid 1 to trial17818531-9 2026-02-21T22:27:54.103 INFO:tasks.rados.rados.0.trial178.stdout:2812: finishing write tid 2 to trial17818531-9 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: finishing write tid 3 to trial17818531-9 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: finishing write tid 4 to trial17818531-9 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: oid 9 updating version 0 to 1210 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: oid 9 version 1210 is already newer than 1209 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: oid 9 version 1210 is already newer than 1208 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1210 (ObjNum 994 snap 281 seq_num 994) dirty exists 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2812: left oid 9 (ObjNum 994 snap 281 seq_num 994) 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: finishing write tid 1 to trial17818531-48 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: finishing write tid 2 to trial17818531-48 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: finishing write tid 3 to trial17818531-48 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: finishing write tid 4 to trial17818531-48 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: oid 48 updating version 0 to 1319 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: oid 48 updating version 1319 to 1320 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: oid 48 updating version 1320 to 1321 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1321 (ObjNum 995 snap 281 seq_num 995) dirty exists 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2814: left oid 48 (ObjNum 995 snap 281 seq_num 995) 2026-02-21T22:27:54.104 INFO:tasks.rados.rados.0.trial178.stdout:2815: expect (ObjNum 978 snap 280 seq_num 978) 2026-02-21T22:27:54.123 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1082917 2026-02-21T22:27:54.135 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 310866 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2806: done (8 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2807: done (7 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2808: done (6 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2809: done (5 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2810: done (4 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2812: done (3 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2814: done (2 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2815: done (1 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2816: done (0 left) 2026-02-21T22:27:54.245 INFO:tasks.rados.rados.0.trial178.stdout:2817: read oid 41 snap -1 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2817: expect (ObjNum 989 snap 281 seq_num 989) 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2818: copy_from oid 45 from oid 34 current snap is 282 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2819: read oid 36 snap 281 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2819: expect (ObjNum 988 snap 281 seq_num 988) 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2820: write oid 2 current snap is 282 2026-02-21T22:27:54.246 INFO:tasks.rados.rados.0.trial178.stdout:2820: seq_num 996 ranges {739772=590247,1900183=540336} 2026-02-21T22:27:54.250 INFO:tasks.rados.rados.0.trial178.stdout:2820: writing trial17818531-2 from 739772 to 1330019 tid 1 2026-02-21T22:27:54.254 INFO:tasks.rados.rados.0.trial178.stdout:2820: writing trial17818531-2 from 1900183 to 2440519 tid 2 2026-02-21T22:27:54.254 INFO:tasks.rados.rados.0.trial178.stdout:2821: read oid 39 snap -1 2026-02-21T22:27:54.254 INFO:tasks.rados.rados.0.trial178.stdout:2821: expect (ObjNum 985 snap 281 seq_num 985) 2026-02-21T22:27:54.254 INFO:tasks.rados.rados.0.trial178.stdout:2822: write (excl) oid 31 current snap is 282 2026-02-21T22:27:54.254 INFO:tasks.rados.rados.0.trial178.stdout:2822: seq_num 997 ranges {756579=665496,1958665=400347} 2026-02-21T22:27:54.259 INFO:tasks.rados.rados.0.trial178.stdout:2822: writing trial17818531-31 from 756579 to 1422075 tid 1 2026-02-21T22:27:54.262 INFO:tasks.rados.rados.0.trial178.stdout:2822: writing trial17818531-31 from 1958665 to 2359012 tid 2 2026-02-21T22:27:54.263 INFO:tasks.rados.rados.0.trial178.stdout:2823: snap_create 2026-02-21T22:27:54.263 INFO:tasks.rados.rados.0.trial178.stdout:2819: expect (ObjNum 988 snap 281 seq_num 988) 2026-02-21T22:27:54.298 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:54.378 INFO:tasks.rados.rados.0.trial178.stdout:2817: expect (ObjNum 989 snap 281 seq_num 989) 2026-02-21T22:27:54.446 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1726128 2026-02-21T22:27:54.456 INFO:tasks.rados.rados.0.trial178.stdout:2820: finishing write tid 1 to trial17818531-2 2026-02-21T22:27:54.456 INFO:tasks.rados.rados.0.trial178.stdout:2820: finishing write tid 2 to trial17818531-2 2026-02-21T22:27:54.456 INFO:tasks.rados.rados.0.trial178.stdout:2820: finishing write tid 3 to trial17818531-2 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2820: finishing write tid 4 to trial17818531-2 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2820: oid 2 updating version 0 to 1185 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2820: oid 2 updating version 1185 to 1186 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2820: oid 2 updating version 1186 to 1187 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1187 (ObjNum 996 snap 282 seq_num 996) dirty exists 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2820: left oid 2 (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2818: finishing copy_from racing read to trial17818531-45 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2818: finishing copy_from to trial17818531-45 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1166 (ObjNum 927 snap 271 seq_num 927) dirty exists 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:2821: expect (ObjNum 985 snap 281 seq_num 985) 2026-02-21T22:27:54.457 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 457965 2026-02-21T22:27:54.498 INFO:tasks.rados.rados.0.trial178.stdout:2822: finishing write tid 1 to trial17818531-31 2026-02-21T22:27:54.498 INFO:tasks.rados.rados.0.trial178.stdout:2822: finishing write tid 2 to trial17818531-31 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: finishing write tid 3 to trial17818531-31 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: finishing write tid 4 to trial17818531-31 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: oid 31 updating version 0 to 1440 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: oid 31 version 1440 is already newer than 1439 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: oid 31 version 1440 is already newer than 1438 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1440 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:27:54.499 INFO:tasks.rados.rados.0.trial178.stdout:2822: left oid 31 (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2817: done (6 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2818: done (5 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2819: done (4 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2820: done (3 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2821: done (2 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2822: done (1 left) 2026-02-21T22:27:55.249 INFO:tasks.rados.rados.0.trial178.stdout:2823: done (0 left) 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2824: read oid 20 snap -1 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2824: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2825: read oid 44 snap 282 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2825: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2826: read oid 12 snap -1 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2826: expect (ObjNum 979 snap 280 seq_num 979) 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2827: write (excl) oid 33 current snap is 283 2026-02-21T22:27:55.250 INFO:tasks.rados.rados.0.trial178.stdout:2827: seq_num 998 ranges {773386=740745,2017147=544005} 2026-02-21T22:27:55.254 INFO:tasks.rados.rados.0.trial178.stdout:2827: writing trial17818531-33 from 773386 to 1514131 tid 1 2026-02-21T22:27:55.259 INFO:tasks.rados.rados.0.trial178.stdout:2827: writing trial17818531-33 from 2017147 to 2561152 tid 2 2026-02-21T22:27:55.259 INFO:tasks.rados.rados.0.trial178.stdout:2828: copy_from oid 21 from oid 43 current snap is 283 2026-02-21T22:27:55.259 INFO:tasks.rados.rados.0.trial178.stdout:2824: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:55.269 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1301942 2026-02-21T22:27:55.373 INFO:tasks.rados.rados.0.trial178.stdout:2825: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:27:55.373 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:55.476 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1498863 2026-02-21T22:27:55.503 INFO:tasks.rados.rados.0.trial178.stdout:2826: expect (ObjNum 979 snap 280 seq_num 979) 2026-02-21T22:27:55.521 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 89435 2026-02-21T22:27:55.556 INFO:tasks.rados.rados.0.trial178.stdout:2828: finishing copy_from to trial17818531-21 2026-02-21T22:27:55.556 INFO:tasks.rados.rados.0.trial178.stdout:2828: got expected ENOENT (src dne) 2026-02-21T22:27:55.556 INFO:tasks.rados.rados.0.trial178.stdout:2828: finishing copy_from racing read to trial17818531-21 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2827: finishing write tid 1 to trial17818531-33 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2827: finishing write tid 2 to trial17818531-33 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2827: finishing write tid 3 to trial17818531-33 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2824: done (4 left) 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2825: done (3 left) 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2826: done (2 left) 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2828: done (1 left) 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2829: write oid 39 current snap is 283 2026-02-21T22:27:55.557 INFO:tasks.rados.rados.0.trial178.stdout:2829: seq_num 999 ranges {790193=415994,1959276=404016} 2026-02-21T22:27:55.564 INFO:tasks.rados.rados.0.trial178.stdout:2829: writing trial17818531-39 from 790193 to 1206187 tid 1 2026-02-21T22:27:55.573 INFO:tasks.rados.rados.0.trial178.stdout:2829: writing trial17818531-39 from 1959276 to 2363292 tid 2 2026-02-21T22:27:55.573 INFO:tasks.rados.rados.0.trial178.stdout:2830: read oid 16 snap -1 2026-02-21T22:27:55.573 INFO:tasks.rados.rados.0.trial178.stdout:2830: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2831: read oid 46 snap -1 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2831: expect (ObjNum 960 snap 279 seq_num 960) 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2832: rollback oid 9 current snap is 283 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 281 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: finishing write tid 4 to trial17818531-33 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: oid 33 updating version 0 to 1172 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: oid 33 version 1172 is already newer than 1170 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: oid 33 version 1172 is already newer than 1171 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: left oid 33 (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2829: finishing write tid 1 to trial17818531-39 2026-02-21T22:27:55.574 INFO:tasks.rados.rados.0.trial178.stdout:2827: done (4 left) 2026-02-21T22:27:55.575 INFO:tasks.rados.rados.0.trial178.stdout:2833: delete oid 26 current snap is 283 2026-02-21T22:27:55.575 INFO:tasks.rados.rados.0.trial178.stdout:2833: done (4 left) 2026-02-21T22:27:55.575 INFO:tasks.rados.rados.0.trial178.stdout:2834: snap_remove snap 282 2026-02-21T22:27:56.253 INFO:tasks.rados.rados.0.trial178.stdout:2834: done (4 left) 2026-02-21T22:27:56.253 INFO:tasks.rados.rados.0.trial178.stdout:2835: read oid 50 snap 281 2026-02-21T22:27:56.253 INFO:tasks.rados.rados.0.trial178.stdout:2835: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:56.254 INFO:tasks.rados.rados.0.trial178.stdout:2836: rollback oid 28 current snap is 283 2026-02-21T22:27:56.254 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 281 2026-02-21T22:27:56.254 INFO:tasks.rados.rados.0.trial178.stdout:2831: expect (ObjNum 960 snap 279 seq_num 960) 2026-02-21T22:27:56.273 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.300 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.328 INFO:tasks.rados.rados.0.trial178.stdout:2830: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: finishing write tid 2 to trial17818531-39 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: finishing write tid 3 to trial17818531-39 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2832: finishing rollback tid 0 to trial17818531-9 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: finishing write tid 4 to trial17818531-39 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: oid 39 updating version 0 to 1211 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: oid 39 updating version 1211 to 1212 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: oid 39 updating version 1212 to 1213 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1213 (ObjNum 999 snap 283 seq_num 999) dirty exists 2026-02-21T22:27:56.409 INFO:tasks.rados.rados.0.trial178.stdout:2829: left oid 39 (ObjNum 999 snap 283 seq_num 999) 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2832: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2832: finishing rollback tid 2 to trial17818531-9 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1216 (ObjNum 994 snap 281 seq_num 994) dirty exists 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2836: finishing rollback tid 0 to trial17818531-28 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2836: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2836: finishing rollback tid 2 to trial17818531-28 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1175 (ObjNum 868 snap 252 seq_num 868) dirty exists 2026-02-21T22:27:56.410 INFO:tasks.rados.rados.0.trial178.stdout:2835: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:56.442 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 334479 2026-02-21T22:27:56.509 INFO:tasks.rados.rados.0.trial178.stdout:2829: done (5 left) 2026-02-21T22:27:56.509 INFO:tasks.rados.rados.0.trial178.stdout:2830: done (4 left) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2831: done (3 left) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2832: done (2 left) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2835: done (1 left) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2836: done (0 left) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2837: read oid 30 snap -1 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2837: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2838: read oid 50 snap -1 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2838: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2839: write oid 28 current snap is 283 2026-02-21T22:27:56.510 INFO:tasks.rados.rados.0.trial178.stdout:2839: seq_num 1001 ranges {423807=566492,1676240=407685,2739004=84803} 2026-02-21T22:27:56.515 INFO:tasks.rados.rados.0.trial178.stdout:2839: writing trial17818531-28 from 423807 to 990299 tid 1 2026-02-21T22:27:56.523 INFO:tasks.rados.rados.0.trial178.stdout:2839: writing trial17818531-28 from 1676240 to 2083925 tid 2 2026-02-21T22:27:56.528 INFO:tasks.rados.rados.0.trial178.stdout:2839: writing trial17818531-28 from 2739004 to 2823807 tid 3 2026-02-21T22:27:56.528 INFO:tasks.rados.rados.0.trial178.stdout:2840: write (excl) oid 29 current snap is 283 2026-02-21T22:27:56.528 INFO:tasks.rados.rados.0.trial178.stdout:2840: seq_num 1002 ranges {440614=641741,1734722=551343} 2026-02-21T22:27:56.535 INFO:tasks.rados.rados.0.trial178.stdout:2840: writing trial17818531-29 from 440614 to 1082355 tid 1 2026-02-21T22:27:56.544 INFO:tasks.rados.rados.0.trial178.stdout:2840: writing trial17818531-29 from 1734722 to 2286065 tid 2 2026-02-21T22:27:56.544 INFO:tasks.rados.rados.0.trial178.stdout:2841: copy_from oid 35 from oid 44 current snap is 283 2026-02-21T22:27:56.544 INFO:tasks.rados.rados.0.trial178.stdout:2838: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:56.545 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.598 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.665 INFO:tasks.rados.rados.0.trial178.stdout:2837: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:27:56.751 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2335363 2026-02-21T22:27:56.759 INFO:tasks.rados.rados.0.trial178.stdout:2839: finishing write tid 1 to trial17818531-28 2026-02-21T22:27:56.759 INFO:tasks.rados.rados.0.trial178.stdout:2839: finishing write tid 2 to trial17818531-28 2026-02-21T22:27:56.759 INFO:tasks.rados.rados.0.trial178.stdout:2839: finishing write tid 3 to trial17818531-28 2026-02-21T22:27:56.759 INFO:tasks.rados.rados.0.trial178.stdout:2837: done (4 left) 2026-02-21T22:27:56.759 INFO:tasks.rados.rados.0.trial178.stdout:2838: done (3 left) 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2842: read oid 47 snap -1 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2842: expect (ObjNum 933 snap 275 seq_num 933) 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2843: read oid 36 snap 281 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2843: expect (ObjNum 988 snap 281 seq_num 988) 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2844: read oid 42 snap -1 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2844: expect deleted 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2845: read oid 34 snap -1 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2845: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2846: delete oid 41 current snap is 283 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2839: finishing write tid 4 to trial17818531-28 2026-02-21T22:27:56.760 INFO:tasks.rados.rados.0.trial178.stdout:2839: finishing write tid 5 to trial17818531-28 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2839: oid 28 updating version 0 to 1176 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2839: oid 28 updating version 1176 to 1178 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2839: oid 28 version 1178 is already newer than 1177 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2839: oid 28 updating version 1178 to 1179 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1179 (ObjNum 1001 snap 283 seq_num 1001) dirty exists 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2839: left oid 28 (ObjNum 1001 snap 283 seq_num 1001) 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: finishing write tid 1 to trial17818531-29 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: finishing write tid 2 to trial17818531-29 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: finishing write tid 3 to trial17818531-29 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: finishing write tid 4 to trial17818531-29 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: oid 29 updating version 0 to 1177 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: oid 29 version 1177 is already newer than 1175 2026-02-21T22:27:56.761 INFO:tasks.rados.rados.0.trial178.stdout:2840: oid 29 version 1177 is already newer than 1176 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1177 (ObjNum 1002 snap 283 seq_num 1002) dirty exists 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:2840: left oid 29 (ObjNum 1002 snap 283 seq_num 1002) 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:2841: finishing copy_from racing read to trial17818531-35 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:2841: finishing copy_from to trial17818531-35 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1437 (ObjNum 908 snap 268 seq_num 908) dirty exists 2026-02-21T22:27:56.762 INFO:tasks.rados.rados.0.trial178.stdout:2843: expect (ObjNum 988 snap 281 seq_num 988) 2026-02-21T22:27:56.810 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 875998 2026-02-21T22:27:56.878 INFO:tasks.rados.rados.0.trial178.stdout:2845: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:27:56.878 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.904 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:56.957 INFO:tasks.rados.rados.0.trial178.stdout:2842: expect (ObjNum 933 snap 275 seq_num 933) 2026-02-21T22:27:56.969 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1585695 2026-02-21T22:27:57.039 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2684316 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2839: done (7 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2840: done (6 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2841: done (5 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2842: done (4 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2843: done (3 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2844: done (2 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2845: done (1 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2846: done (0 left) 2026-02-21T22:27:57.101 INFO:tasks.rados.rados.0.trial178.stdout:2847: read oid 20 snap -1 2026-02-21T22:27:57.102 INFO:tasks.rados.rados.0.trial178.stdout:2847: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:57.102 INFO:tasks.rados.rados.0.trial178.stdout:2848: rollback oid 47 current snap is 283 2026-02-21T22:27:57.102 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 281 2026-02-21T22:27:57.102 INFO:tasks.rados.rados.0.trial178.stdout:2849: delete oid 7 current snap is 283 2026-02-21T22:27:57.105 INFO:tasks.rados.rados.0.trial178.stdout:2847: expect (ObjNum 866 snap 251 seq_num 866) 2026-02-21T22:27:57.121 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1474195 2026-02-21T22:27:57.144 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 466262 2026-02-21T22:27:57.217 INFO:tasks.rados.rados.0.trial178.stdout:2848: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:27:57.217 INFO:tasks.rados.rados.0.trial178.stdout:2848: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:27:57.217 INFO:tasks.rados.rados.0.trial178.stdout:2848: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:27:57.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1588 (ObjNum 933 snap 275 seq_num 933) dirty exists 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2847: done (2 left) 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2848: done (1 left) 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2849: done (0 left) 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2850: read oid 16 snap -1 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2850: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2851: read oid 11 snap -1 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2851: expect (ObjNum 971 snap 280 seq_num 971) 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2852: write oid 15 current snap is 283 2026-02-21T22:27:57.218 INFO:tasks.rados.rados.0.trial178.stdout:2852: seq_num 1005 ranges {491035=583841,1510168=698670,2732343=158692} 2026-02-21T22:27:57.225 INFO:tasks.rados.rados.0.trial178.stdout:2852: writing trial17818531-15 from 491035 to 1074876 tid 1 2026-02-21T22:27:57.233 INFO:tasks.rados.rados.0.trial178.stdout:2852: writing trial17818531-15 from 1510168 to 2208838 tid 2 2026-02-21T22:27:57.238 INFO:tasks.rados.rados.0.trial178.stdout:2852: writing trial17818531-15 from 2732343 to 2891035 tid 3 2026-02-21T22:27:57.238 INFO:tasks.rados.rados.0.trial178.stdout:2853: delete oid 5 current snap is 283 2026-02-21T22:27:57.238 INFO:tasks.rados.rados.0.trial178.stdout:2851: expect (ObjNum 971 snap 280 seq_num 971) 2026-02-21T22:27:57.248 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1762930 2026-02-21T22:27:57.255 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:57.283 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 861530 2026-02-21T22:27:57.296 INFO:tasks.rados.rados.0.trial178.stdout:2850: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:27:57.354 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2852: finishing write tid 1 to trial17818531-15 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2852: finishing write tid 2 to trial17818531-15 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2852: finishing write tid 3 to trial17818531-15 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2850: done (3 left) 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2851: done (2 left) 2026-02-21T22:27:57.385 INFO:tasks.rados.rados.0.trial178.stdout:2853: done (1 left) 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2854: delete oid 17 current snap is 283 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: finishing write tid 4 to trial17818531-15 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: finishing write tid 5 to trial17818531-15 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: oid 15 updating version 0 to 1192 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: oid 15 version 1192 is already newer than 1191 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: oid 15 version 1192 is already newer than 1190 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: oid 15 version 1192 is already newer than 1189 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1192 (ObjNum 1005 snap 283 seq_num 1005) dirty exists 2026-02-21T22:27:57.386 INFO:tasks.rados.rados.0.trial178.stdout:2852: left oid 15 (ObjNum 1005 snap 283 seq_num 1005) 2026-02-21T22:27:57.387 INFO:tasks.rados.rados.0.trial178.stdout:2852: done (1 left) 2026-02-21T22:27:57.387 INFO:tasks.rados.rados.0.trial178.stdout:2854: done (0 left) 2026-02-21T22:27:57.387 INFO:tasks.rados.rados.0.trial178.stdout:2855: snap_remove snap 281 2026-02-21T22:27:58.256 INFO:tasks.rados.rados.0.trial178.stdout:2855: done (0 left) 2026-02-21T22:27:58.257 INFO:tasks.rados.rados.0.trial178.stdout:2856: delete oid 12 current snap is 283 2026-02-21T22:27:58.259 INFO:tasks.rados.rados.0.trial178.stdout:2856: done (0 left) 2026-02-21T22:27:58.259 INFO:tasks.rados.rados.0.trial178.stdout:2857: delete oid 36 current snap is 283 2026-02-21T22:27:58.263 INFO:tasks.rados.rados.0.trial178.stdout:2857: done (0 left) 2026-02-21T22:27:58.264 INFO:tasks.rados.rados.0.trial178.stdout:2858: write (excl) oid 19 current snap is 283 2026-02-21T22:27:58.264 INFO:tasks.rados.rados.0.trial178.stdout:2858: seq_num 1010 ranges {575070=560086,1686225=566019,2869458=105612} 2026-02-21T22:27:58.268 INFO:tasks.rados.rados.0.trial178.stdout:2858: writing trial17818531-19 from 575070 to 1135156 tid 1 2026-02-21T22:27:58.272 INFO:tasks.rados.rados.0.trial178.stdout:2858: writing trial17818531-19 from 1686225 to 2252244 tid 2 2026-02-21T22:27:58.275 INFO:tasks.rados.rados.0.trial178.stdout:2858: writing trial17818531-19 from 2869458 to 2975070 tid 3 2026-02-21T22:27:58.275 INFO:tasks.rados.rados.0.trial178.stdout:2859: read oid 27 snap -1 2026-02-21T22:27:58.275 INFO:tasks.rados.rados.0.trial178.stdout:2859: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:58.275 INFO:tasks.rados.rados.0.trial178.stdout:2860: read oid 15 snap -1 2026-02-21T22:27:58.276 INFO:tasks.rados.rados.0.trial178.stdout:2860: expect (ObjNum 1005 snap 283 seq_num 1005) 2026-02-21T22:27:58.276 INFO:tasks.rados.rados.0.trial178.stdout:2861: copy_from oid 4 from oid 5 current snap is 283 2026-02-21T22:27:58.276 INFO:tasks.rados.rados.0.trial178.stdout:2858: finishing write tid 1 to trial17818531-19 2026-02-21T22:27:58.276 INFO:tasks.rados.rados.0.trial178.stdout:2862: write oid 8 current snap is 283 2026-02-21T22:27:58.276 INFO:tasks.rados.rados.0.trial178.stdout:2862: seq_num 1011 ranges {591877=635335,1628354=709677} 2026-02-21T22:27:58.279 INFO:tasks.rados.rados.0.trial178.stdout:2862: writing trial17818531-8 from 591877 to 1227212 tid 1 2026-02-21T22:27:58.284 INFO:tasks.rados.rados.0.trial178.stdout:2862: writing trial17818531-8 from 1628354 to 2338031 tid 2 2026-02-21T22:27:58.284 INFO:tasks.rados.rados.0.trial178.stdout:2863: read oid 12 snap -1 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2863: expect deleted 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2864: read oid 50 snap -1 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2864: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2865: delete oid 1 current snap is 283 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2861: finishing copy_from to trial17818531-4 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2861: got expected ENOENT (src dne) 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2861: finishing copy_from racing read to trial17818531-4 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2858: finishing write tid 2 to trial17818531-19 2026-02-21T22:27:58.285 INFO:tasks.rados.rados.0.trial178.stdout:2860: expect (ObjNum 1005 snap 283 seq_num 1005) 2026-02-21T22:27:58.332 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2400773 2026-02-21T22:27:58.471 INFO:tasks.rados.rados.0.trial178.stdout:2858: finishing write tid 3 to trial17818531-19 2026-02-21T22:27:58.471 INFO:tasks.rados.rados.0.trial178.stdout:2858: finishing write tid 4 to trial17818531-19 2026-02-21T22:27:58.471 INFO:tasks.rados.rados.0.trial178.stdout:2859: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:27:58.530 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:58.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:27:58.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:27:58.637 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3257929 2026-02-21T22:27:58.650 INFO:tasks.rados.rados.0.trial178.stdout:2858: finishing write tid 5 to trial17818531-19 2026-02-21T22:27:58.650 INFO:tasks.rados.rados.0.trial178.stdout:2858: oid 19 updating version 0 to 1598 2026-02-21T22:27:58.650 INFO:tasks.rados.rados.0.trial178.stdout:2858: oid 19 version 1598 is already newer than 1597 2026-02-21T22:27:58.650 INFO:tasks.rados.rados.0.trial178.stdout:2858: oid 19 version 1598 is already newer than 1595 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2858: oid 19 version 1598 is already newer than 1596 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1598 (ObjNum 1010 snap 283 seq_num 1010) dirty exists 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2858: left oid 19 (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2862: finishing write tid 1 to trial17818531-8 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2858: done (7 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2859: done (6 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2860: done (5 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2861: done (4 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2863: done (3 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2865: done (2 left) 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2866: delete oid 46 current snap is 283 2026-02-21T22:27:58.651 INFO:tasks.rados.rados.0.trial178.stdout:2864: expect (ObjNum 983 snap 281 seq_num 983) 2026-02-21T22:27:58.652 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:58.688 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1594683 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: finishing write tid 2 to trial17818531-8 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: finishing write tid 3 to trial17818531-8 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: finishing write tid 4 to trial17818531-8 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: oid 8 updating version 0 to 1601 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: oid 8 version 1601 is already newer than 1600 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: oid 8 version 1601 is already newer than 1599 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1601 (ObjNum 1011 snap 283 seq_num 1011) dirty exists 2026-02-21T22:27:58.743 INFO:tasks.rados.rados.0.trial178.stdout:2862: left oid 8 (ObjNum 1011 snap 283 seq_num 1011) 2026-02-21T22:27:58.744 INFO:tasks.rados.rados.0.trial178.stdout:2862: done (2 left) 2026-02-21T22:27:58.744 INFO:tasks.rados.rados.0.trial178.stdout:2864: done (1 left) 2026-02-21T22:27:58.744 INFO:tasks.rados.rados.0.trial178.stdout:2866: done (0 left) 2026-02-21T22:27:58.744 INFO:tasks.rados.rados.0.trial178.stdout:2867: delete oid 38 current snap is 283 2026-02-21T22:27:58.745 INFO:tasks.rados.rados.0.trial178.stdout:2867: done (0 left) 2026-02-21T22:27:58.745 INFO:tasks.rados.rados.0.trial178.stdout:2868: copy_from oid 37 from oid 15 current snap is 283 2026-02-21T22:27:58.745 INFO:tasks.rados.rados.0.trial178.stdout:2869: write oid 11 current snap is 283 2026-02-21T22:27:58.745 INFO:tasks.rados.rados.0.trial178.stdout:2869: seq_num 1015 ranges {659105=652684,1862282=717015} 2026-02-21T22:27:58.754 INFO:tasks.rados.rados.0.trial178.stdout:2869: writing trial17818531-11 from 659105 to 1311789 tid 1 2026-02-21T22:27:58.763 INFO:tasks.rados.rados.0.trial178.stdout:2869: writing trial17818531-11 from 1862282 to 2579297 tid 2 2026-02-21T22:27:58.763 INFO:tasks.rados.rados.0.trial178.stdout:2870: rollback oid 44 current snap is 283 2026-02-21T22:27:58.764 INFO:tasks.rados.rados.0.trial178.stdout:2870: done (2 left) 2026-02-21T22:27:58.764 INFO:tasks.rados.rados.0.trial178.stdout:2871: write (excl) oid 49 current snap is 283 2026-02-21T22:27:58.764 INFO:tasks.rados.rados.0.trial178.stdout:2871: seq_num 1016 ranges {675912=727933,1920764=460673,2801290=274622} 2026-02-21T22:27:58.772 INFO:tasks.rados.rados.0.trial178.stdout:2871: writing trial17818531-49 from 675912 to 1403845 tid 1 2026-02-21T22:27:58.779 INFO:tasks.rados.rados.0.trial178.stdout:2871: writing trial17818531-49 from 1920764 to 2381437 tid 2 2026-02-21T22:27:58.783 INFO:tasks.rados.rados.0.trial178.stdout:2871: writing trial17818531-49 from 2801290 to 3075912 tid 3 2026-02-21T22:27:58.784 INFO:tasks.rados.rados.0.trial178.stdout:2872: write (excl) oid 3 current snap is 283 2026-02-21T22:27:58.784 INFO:tasks.rados.rados.0.trial178.stdout:2872: seq_num 1017 ranges {692719=403182,1579246=720684,2828713=264006} 2026-02-21T22:27:58.790 INFO:tasks.rados.rados.0.trial178.stdout:2872: writing trial17818531-3 from 692719 to 1095901 tid 1 2026-02-21T22:27:58.794 INFO:tasks.rados.rados.0.trial178.stdout:2872: writing trial17818531-3 from 1579246 to 2299930 tid 2 2026-02-21T22:27:58.797 INFO:tasks.rados.rados.0.trial178.stdout:2872: writing trial17818531-3 from 2828713 to 3092719 tid 3 2026-02-21T22:27:58.797 INFO:tasks.rados.rados.0.trial178.stdout:2873: write (excl) oid 35 current snap is 283 2026-02-21T22:27:58.797 INFO:tasks.rados.rados.0.trial178.stdout:2873: seq_num 1018 ranges {709526=478431,1637728=464342,2856136=253390} 2026-02-21T22:27:58.802 INFO:tasks.rados.rados.0.trial178.stdout:2873: writing trial17818531-35 from 709526 to 1187957 tid 1 2026-02-21T22:27:58.805 INFO:tasks.rados.rados.0.trial178.stdout:2873: writing trial17818531-35 from 1637728 to 2102070 tid 2 2026-02-21T22:27:58.809 INFO:tasks.rados.rados.0.trial178.stdout:2873: writing trial17818531-35 from 2856136 to 3109526 tid 3 2026-02-21T22:27:58.809 INFO:tasks.rados.rados.0.trial178.stdout:2874: write (excl) oid 34 current snap is 283 2026-02-21T22:27:58.809 INFO:tasks.rados.rados.0.trial178.stdout:2874: seq_num 1019 ranges {726333=670033,2096210=724353} 2026-02-21T22:27:58.814 INFO:tasks.rados.rados.0.trial178.stdout:2874: writing trial17818531-34 from 726333 to 1396366 tid 1 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2874: writing trial17818531-34 from 2096210 to 2820563 tid 2 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2875: copy_from oid 46 from oid 33 current snap is 283 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2868: finishing copy_from racing read to trial17818531-37 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2868: finishing copy_from to trial17818531-37 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1192 (ObjNum 1005 snap 283 seq_num 1005) dirty exists 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2869: finishing write tid 1 to trial17818531-11 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2869: finishing write tid 2 to trial17818531-11 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2869: finishing write tid 3 to trial17818531-11 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2868: done (6 left) 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2876: rollback oid 4 current snap is 283 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2876: done (6 left) 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2877: write (excl) oid 20 current snap is 283 2026-02-21T22:27:58.820 INFO:tasks.rados.rados.0.trial178.stdout:2877: seq_num 1020 ranges {743140=745282,2154692=468011} 2026-02-21T22:27:58.825 INFO:tasks.rados.rados.0.trial178.stdout:2877: writing trial17818531-20 from 743140 to 1488422 tid 1 2026-02-21T22:27:58.830 INFO:tasks.rados.rados.0.trial178.stdout:2877: writing trial17818531-20 from 2154692 to 2622703 tid 2 2026-02-21T22:27:58.830 INFO:tasks.rados.rados.0.trial178.stdout:2878: rollback oid 18 current snap is 283 2026-02-21T22:27:58.830 INFO:tasks.rados.rados.0.trial178.stdout:2878: done (7 left) 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2879: read oid 5 snap -1 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2879: expect deleted 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2880: delete oid 17 current snap is 283 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2869: finishing write tid 4 to trial17818531-11 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2869: oid 11 updating version 0 to 1440 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2869: oid 11 version 1440 is already newer than 1439 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2869: oid 11 version 1440 is already newer than 1438 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1440 (ObjNum 1015 snap 283 seq_num 1015) dirty exists 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2869: left oid 11 (ObjNum 1015 snap 283 seq_num 1015) 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: finishing write tid 1 to trial17818531-49 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: finishing write tid 2 to trial17818531-49 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: finishing write tid 3 to trial17818531-49 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: finishing write tid 4 to trial17818531-49 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: finishing write tid 5 to trial17818531-49 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: oid 49 updating version 0 to 1441 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: oid 49 updating version 1441 to 1444 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: oid 49 version 1444 is already newer than 1443 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:2871: oid 49 version 1444 is already newer than 1442 2026-02-21T22:27:58.831 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1444 (ObjNum 1016 snap 283 seq_num 1016) dirty exists 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2871: left oid 49 (ObjNum 1016 snap 283 seq_num 1016) 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: finishing write tid 1 to trial17818531-3 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: finishing write tid 2 to trial17818531-3 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: finishing write tid 3 to trial17818531-3 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: finishing write tid 4 to trial17818531-3 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: finishing write tid 5 to trial17818531-3 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: oid 3 updating version 0 to 1441 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: oid 3 updating version 1441 to 1444 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: oid 3 version 1444 is already newer than 1442 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: oid 3 version 1444 is already newer than 1443 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1444 (ObjNum 1017 snap 283 seq_num 1017) dirty exists 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2872: left oid 3 (ObjNum 1017 snap 283 seq_num 1017) 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: finishing write tid 1 to trial17818531-35 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: finishing write tid 2 to trial17818531-35 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: finishing write tid 3 to trial17818531-35 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: finishing write tid 4 to trial17818531-35 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: finishing write tid 5 to trial17818531-35 2026-02-21T22:27:58.832 INFO:tasks.rados.rados.0.trial178.stdout:2873: oid 35 updating version 0 to 1446 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2873: oid 35 version 1446 is already newer than 1445 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2873: oid 35 updating version 1446 to 1447 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2873: oid 35 updating version 1447 to 1448 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1448 (ObjNum 1018 snap 283 seq_num 1018) dirty exists 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2873: left oid 35 (ObjNum 1018 snap 283 seq_num 1018) 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: finishing write tid 1 to trial17818531-34 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: finishing write tid 2 to trial17818531-34 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: finishing write tid 3 to trial17818531-34 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: finishing write tid 4 to trial17818531-34 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: oid 34 updating version 0 to 1179 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: oid 34 version 1179 is already newer than 1178 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: oid 34 updating version 1179 to 1180 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1180 (ObjNum 1019 snap 283 seq_num 1019) dirty exists 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2874: left oid 34 (ObjNum 1019 snap 283 seq_num 1019) 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2875: finishing copy_from racing read to trial17818531-46 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2875: finishing copy_from to trial17818531-46 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:27:58.833 INFO:tasks.rados.rados.0.trial178.stdout:2877: finishing write tid 1 to trial17818531-20 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: finishing write tid 2 to trial17818531-20 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: finishing write tid 3 to trial17818531-20 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: finishing write tid 4 to trial17818531-20 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: oid 20 updating version 0 to 1194 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: oid 20 updating version 1194 to 1196 2026-02-21T22:27:58.834 INFO:tasks.rados.rados.0.trial178.stdout:2877: oid 20 version 1196 is already newer than 1195 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1196 (ObjNum 1020 snap 283 seq_num 1020) dirty exists 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2877: left oid 20 (ObjNum 1020 snap 283 seq_num 1020) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2869: done (8 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2871: done (7 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2872: done (6 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2873: done (5 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2874: done (4 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2875: done (3 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2877: done (2 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2879: done (1 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2880: done (0 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2881: read oid 32 snap -1 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2881: expect (ObjNum 976 snap 280 seq_num 976) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2882: rollback oid 8 current snap is 283 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2882: done (1 left) 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2883: write oid 14 current snap is 283 2026-02-21T22:27:58.835 INFO:tasks.rados.rados.0.trial178.stdout:2883: seq_num 1022 ranges {776754=495780,1871656=471680,2965828=210926} 2026-02-21T22:27:58.837 INFO:tasks.rados.rados.0.trial178.stdout:2883: writing trial17818531-14 from 776754 to 1272534 tid 1 2026-02-21T22:27:58.841 INFO:tasks.rados.rados.0.trial178.stdout:2883: writing trial17818531-14 from 1871656 to 2343336 tid 2 2026-02-21T22:27:58.845 INFO:tasks.rados.rados.0.trial178.stdout:2883: writing trial17818531-14 from 2965828 to 3176754 tid 3 2026-02-21T22:27:58.845 INFO:tasks.rados.rados.0.trial178.stdout:2884: read oid 46 snap -1 2026-02-21T22:27:58.845 INFO:tasks.rados.rados.0.trial178.stdout:2884: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:27:58.845 INFO:tasks.rados.rados.0.trial178.stdout:2885: rollback oid 49 current snap is 283 2026-02-21T22:27:58.845 INFO:tasks.rados.rados.0.trial178.stdout:2885: done (3 left) 2026-02-21T22:27:58.846 INFO:tasks.rados.rados.0.trial178.stdout:2886: read oid 13 snap -1 2026-02-21T22:27:58.846 INFO:tasks.rados.rados.0.trial178.stdout:2886: expect (ObjNum 956 snap 278 seq_num 956) 2026-02-21T22:27:58.846 INFO:tasks.rados.rados.0.trial178.stdout:2887: write oid 20 current snap is 283 2026-02-21T22:27:58.846 INFO:tasks.rados.rados.0.trial178.stdout:2887: seq_num 1023 ranges {793561=571029,1813785=731691,2993251=200310} 2026-02-21T22:27:58.850 INFO:tasks.rados.rados.0.trial178.stdout:2887: writing trial17818531-20 from 793561 to 1364590 tid 1 2026-02-21T22:27:58.854 INFO:tasks.rados.rados.0.trial178.stdout:2887: writing trial17818531-20 from 1813785 to 2545476 tid 2 2026-02-21T22:27:58.857 INFO:tasks.rados.rados.0.trial178.stdout:2887: writing trial17818531-20 from 2993251 to 3193561 tid 3 2026-02-21T22:27:58.857 INFO:tasks.rados.rados.0.trial178.stdout:2888: read oid 8 snap -1 2026-02-21T22:27:58.858 INFO:tasks.rados.rados.0.trial178.stdout:2888: expect (ObjNum 1011 snap 283 seq_num 1011) 2026-02-21T22:27:58.858 INFO:tasks.rados.rados.0.trial178.stdout:2889: delete oid 9 current snap is 283 2026-02-21T22:27:58.858 INFO:tasks.rados.rados.0.trial178.stdout:2881: expect (ObjNum 976 snap 280 seq_num 976) 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: finishing write tid 1 to trial17818531-14 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: finishing write tid 2 to trial17818531-14 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2881: done (6 left) 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2889: done (5 left) 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2890: copy_from oid 17 from oid 28 current snap is 283 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: finishing write tid 3 to trial17818531-14 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: finishing write tid 4 to trial17818531-14 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: finishing write tid 5 to trial17818531-14 2026-02-21T22:27:58.922 INFO:tasks.rados.rados.0.trial178.stdout:2883: oid 14 updating version 0 to 1220 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:2883: oid 14 version 1220 is already newer than 1218 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:2883: oid 14 version 1220 is already newer than 1217 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:2883: oid 14 version 1220 is already newer than 1219 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1220 (ObjNum 1022 snap 283 seq_num 1022) dirty exists 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:2883: left oid 14 (ObjNum 1022 snap 283 seq_num 1022) 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:2886: expect (ObjNum 956 snap 278 seq_num 956) 2026-02-21T22:27:58.923 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 663103 2026-02-21T22:27:58.963 INFO:tasks.rados.rados.0.trial178.stdout:2884: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:27:58.963 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:59.017 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1349166 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: finishing write tid 1 to trial17818531-20 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: finishing write tid 2 to trial17818531-20 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: finishing write tid 3 to trial17818531-20 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: finishing write tid 4 to trial17818531-20 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: finishing write tid 5 to trial17818531-20 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: oid 20 updating version 0 to 1198 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: oid 20 updating version 1198 to 1199 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: oid 20 version 1199 is already newer than 1197 2026-02-21T22:27:59.034 INFO:tasks.rados.rados.0.trial178.stdout:2887: oid 20 updating version 1199 to 1200 2026-02-21T22:27:59.035 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1200 (ObjNum 1023 snap 283 seq_num 1023) dirty exists 2026-02-21T22:27:59.035 INFO:tasks.rados.rados.0.trial178.stdout:2887: left oid 20 (ObjNum 1023 snap 283 seq_num 1023) 2026-02-21T22:27:59.035 INFO:tasks.rados.rados.0.trial178.stdout:2888: expect (ObjNum 1011 snap 283 seq_num 1011) 2026-02-21T22:27:59.038 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1175586 2026-02-21T22:27:59.053 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1234295 2026-02-21T22:27:59.068 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:59.090 INFO:tasks.rados.rados.0.trial178.stdout:2890: finishing copy_from racing read to trial17818531-17 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2890: finishing copy_from to trial17818531-17 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1179 (ObjNum 1001 snap 283 seq_num 1001) dirty exists 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2883: done (5 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2884: done (4 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2886: done (3 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2887: done (2 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2888: done (1 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2890: done (0 left) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2891: read oid 4 snap -1 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2891: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:27:59.091 INFO:tasks.rados.rados.0.trial178.stdout:2892: delete oid 24 current snap is 283 2026-02-21T22:27:59.092 INFO:tasks.rados.rados.0.trial178.stdout:2892: done (1 left) 2026-02-21T22:27:59.092 INFO:tasks.rados.rados.0.trial178.stdout:2893: snap_create 2026-02-21T22:27:59.095 INFO:tasks.rados.rados.0.trial178.stdout:2891: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:27:59.133 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:59.169 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:27:59.261 INFO:tasks.rados.rados.0.trial178.stdout:2891: done (1 left) 2026-02-21T22:27:59.262 INFO:tasks.rados.rados.0.trial178.stdout:2893: done (0 left) 2026-02-21T22:27:59.262 INFO:tasks.rados.rados.0.trial178.stdout:2894: snap_create 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2894: done (0 left) 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2895: rollback oid 14 current snap is 285 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 14 to 283 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2896: copy_from oid 13 from oid 30 current snap is 285 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2897: read oid 16 snap -1 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2897: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2898: copy_from oid 24 from oid 7 current snap is 285 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2899: read oid 42 snap -1 2026-02-21T22:28:00.267 INFO:tasks.rados.rados.0.trial178.stdout:2899: expect deleted 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2900: read oid 45 snap -1 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2900: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2901: rollback oid 48 current snap is 285 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 284 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2902: rollback oid 21 current snap is 285 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 283 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2903: write (excl) oid 26 current snap is 285 2026-02-21T22:28:00.268 INFO:tasks.rados.rados.0.trial178.stdout:2903: seq_num 1026 ranges {443982=796776,1989231=479018,2959167=284815} 2026-02-21T22:28:00.275 INFO:tasks.rados.rados.0.trial178.stdout:2903: writing trial17818531-26 from 443982 to 1240758 tid 1 2026-02-21T22:28:00.284 INFO:tasks.rados.rados.0.trial178.stdout:2903: writing trial17818531-26 from 1989231 to 2468249 tid 2 2026-02-21T22:28:00.289 INFO:tasks.rados.rados.0.trial178.stdout:2903: writing trial17818531-26 from 2959167 to 3243982 tid 3 2026-02-21T22:28:00.289 INFO:tasks.rados.rados.0.trial178.stdout:2904: copy_from oid 50 from oid 46 current snap is 285 2026-02-21T22:28:00.289 INFO:tasks.rados.rados.0.trial178.stdout:2897: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:00.314 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2349748 2026-02-21T22:28:00.390 INFO:tasks.rados.rados.0.trial178.stdout:2902: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2902: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2902: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1606 (ObjNum 917 snap 270 seq_num 917) dirty exists 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2895: finishing rollback tid 0 to trial17818531-14 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2895: finishing rollback tid 1 to trial17818531-14 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2897: done (9 left) 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2899: done (8 left) 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2902: done (7 left) 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2905: snap_create 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:2895: finishing rollback tid 2 to trial17818531-14 2026-02-21T22:28:00.391 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1224 (ObjNum 1022 snap 283 seq_num 1022) dirty exists 2026-02-21T22:28:00.392 INFO:tasks.rados.rados.0.trial178.stdout:2898: finishing copy_from to trial17818531-24 2026-02-21T22:28:00.392 INFO:tasks.rados.rados.0.trial178.stdout:2898: got expected ENOENT (src dne) 2026-02-21T22:28:00.392 INFO:tasks.rados.rados.0.trial178.stdout:2898: finishing copy_from racing read to trial17818531-24 2026-02-21T22:28:00.392 INFO:tasks.rados.rados.0.trial178.stdout:2900: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:28:00.392 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2896: finishing copy_from racing read to trial17818531-13 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2903: finishing write tid 1 to trial17818531-26 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2896: finishing copy_from to trial17818531-13 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 939 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2901: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2901: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:2901: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:28:00.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1325 (ObjNum 995 snap 281 seq_num 995) dirty exists 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: finishing write tid 2 to trial17818531-26 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: finishing write tid 3 to trial17818531-26 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: finishing write tid 4 to trial17818531-26 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: finishing write tid 5 to trial17818531-26 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: oid 26 updating version 0 to 1188 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: oid 26 updating version 1188 to 1190 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: oid 26 version 1190 is already newer than 1189 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: oid 26 updating version 1190 to 1191 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1191 (ObjNum 1026 snap 285 seq_num 1026) dirty exists 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2903: left oid 26 (ObjNum 1026 snap 285 seq_num 1026) 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2904: finishing copy_from racing read to trial17818531-50 2026-02-21T22:28:00.458 INFO:tasks.rados.rados.0.trial178.stdout:2904: finishing copy_from to trial17818531-50 2026-02-21T22:28:00.459 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:01.270 INFO:tasks.rados.rados.0.trial178.stdout:2895: done (7 left) 2026-02-21T22:28:01.270 INFO:tasks.rados.rados.0.trial178.stdout:2896: done (6 left) 2026-02-21T22:28:01.270 INFO:tasks.rados.rados.0.trial178.stdout:2898: done (5 left) 2026-02-21T22:28:01.270 INFO:tasks.rados.rados.0.trial178.stdout:2900: done (4 left) 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2901: done (3 left) 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2903: done (2 left) 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2904: done (1 left) 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2905: done (0 left) 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2906: read oid 42 snap -1 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2906: expect deleted 2026-02-21T22:28:01.271 INFO:tasks.rados.rados.0.trial178.stdout:2907: delete oid 41 current snap is 286 2026-02-21T22:28:01.274 INFO:tasks.rados.rados.0.trial178.stdout:2906: done (1 left) 2026-02-21T22:28:01.274 INFO:tasks.rados.rados.0.trial178.stdout:2907: done (0 left) 2026-02-21T22:28:01.274 INFO:tasks.rados.rados.0.trial178.stdout:2908: snap_create 2026-02-21T22:28:02.179 INFO:tasks.rados.rados.0.trial178.stdout:2908: done (0 left) 2026-02-21T22:28:02.179 INFO:tasks.rados.rados.0.trial178.stdout:2909: delete oid 38 current snap is 287 2026-02-21T22:28:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2909: done (0 left) 2026-02-21T22:28:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2910: read oid 35 snap -1 2026-02-21T22:28:02.183 INFO:tasks.rados.rados.0.trial178.stdout:2910: expect (ObjNum 1018 snap 283 seq_num 1018) 2026-02-21T22:28:02.184 INFO:tasks.rados.rados.0.trial178.stdout:2911: write oid 17 current snap is 287 2026-02-21T22:28:02.184 INFO:tasks.rados.rados.0.trial178.stdout:2911: seq_num 1029 ranges {494403=738876,1764677=626345,3041436=252967} 2026-02-21T22:28:02.191 INFO:tasks.rados.rados.0.trial178.stdout:2911: writing trial17818531-17 from 494403 to 1233279 tid 1 2026-02-21T22:28:02.199 INFO:tasks.rados.rados.0.trial178.stdout:2911: writing trial17818531-17 from 1764677 to 2391022 tid 2 2026-02-21T22:28:02.205 INFO:tasks.rados.rados.0.trial178.stdout:2911: writing trial17818531-17 from 3041436 to 3294403 tid 3 2026-02-21T22:28:02.205 INFO:tasks.rados.rados.0.trial178.stdout:2912: snap_remove snap 283 2026-02-21T22:28:03.182 INFO:tasks.rados.rados.0.trial178.stdout:2912: done (2 left) 2026-02-21T22:28:03.182 INFO:tasks.rados.rados.0.trial178.stdout:2913: copy_from oid 11 from oid 5 current snap is 287 2026-02-21T22:28:03.182 INFO:tasks.rados.rados.0.trial178.stdout:2910: expect (ObjNum 1018 snap 283 seq_num 1018) 2026-02-21T22:28:03.191 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 970206 2026-02-21T22:28:03.302 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:03.355 INFO:tasks.rados.rados.0.trial178.stdout:2911: finishing write tid 1 to trial17818531-17 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: finishing write tid 2 to trial17818531-17 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: finishing write tid 3 to trial17818531-17 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: finishing write tid 4 to trial17818531-17 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: finishing write tid 5 to trial17818531-17 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: oid 17 updating version 0 to 1204 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: oid 17 version 1204 is already newer than 1202 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: oid 17 version 1204 is already newer than 1203 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:2911: oid 17 version 1204 is already newer than 1201 2026-02-21T22:28:03.356 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1204 (ObjNum 1029 snap 287 seq_num 1029) dirty exists 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2911: left oid 17 (ObjNum 1029 snap 287 seq_num 1029) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2913: finishing copy_from to trial17818531-11 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2913: got expected ENOENT (src dne) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2913: finishing copy_from racing read to trial17818531-11 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2910: done (2 left) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2911: done (1 left) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2913: done (0 left) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2914: read oid 27 snap -1 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2914: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:28:03.357 INFO:tasks.rados.rados.0.trial178.stdout:2915: snap_remove snap 286 2026-02-21T22:28:03.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:03.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:04.279 INFO:tasks.rados.rados.0.trial178.stdout:2915: done (1 left) 2026-02-21T22:28:04.279 INFO:tasks.rados.rados.0.trial178.stdout:2916: delete oid 24 current snap is 287 2026-02-21T22:28:04.279 INFO:tasks.rados.rados.0.trial178.stdout:2914: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:28:04.460 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2870268 2026-02-21T22:28:04.479 INFO:tasks.rados.rados.0.trial178.stdout:2914: done (1 left) 2026-02-21T22:28:04.479 INFO:tasks.rados.rados.0.trial178.stdout:2916: done (0 left) 2026-02-21T22:28:04.479 INFO:tasks.rados.rados.0.trial178.stdout:2917: snap_create 2026-02-21T22:28:05.279 INFO:tasks.rados.rados.0.trial178.stdout:2917: done (0 left) 2026-02-21T22:28:05.279 INFO:tasks.rados.rados.0.trial178.stdout:2918: snap_remove snap 284 2026-02-21T22:28:06.284 INFO:tasks.rados.rados.0.trial178.stdout:2918: done (0 left) 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2919: read oid 48 snap 285 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2919: expect (ObjNum 995 snap 281 seq_num 995) 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2920: read oid 7 snap 285 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2920: expect deleted 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2921: write oid 17 current snap is 288 2026-02-21T22:28:06.285 INFO:tasks.rados.rados.0.trial178.stdout:2921: seq_num 1031 ranges {528017=489374,1481641=630014,2696282=400857} 2026-02-21T22:28:06.292 INFO:tasks.rados.rados.0.trial178.stdout:2921: writing trial17818531-17 from 528017 to 1017391 tid 1 2026-02-21T22:28:06.299 INFO:tasks.rados.rados.0.trial178.stdout:2921: writing trial17818531-17 from 1481641 to 2111655 tid 2 2026-02-21T22:28:06.306 INFO:tasks.rados.rados.0.trial178.stdout:2921: writing trial17818531-17 from 2696282 to 3097139 tid 3 2026-02-21T22:28:06.306 INFO:tasks.rados.rados.0.trial178.stdout:2922: rollback oid 15 current snap is 288 2026-02-21T22:28:06.306 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 287 2026-02-21T22:28:06.306 INFO:tasks.rados.rados.0.trial178.stdout:2919: expect (ObjNum 995 snap 281 seq_num 995) 2026-02-21T22:28:06.312 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 823311 2026-02-21T22:28:06.335 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:06.362 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:06.382 INFO:tasks.rados.rados.0.trial178.stdout:2921: finishing write tid 1 to trial17818531-17 2026-02-21T22:28:06.382 INFO:tasks.rados.rados.0.trial178.stdout:2921: finishing write tid 2 to trial17818531-17 2026-02-21T22:28:06.382 INFO:tasks.rados.rados.0.trial178.stdout:2921: finishing write tid 3 to trial17818531-17 2026-02-21T22:28:06.382 INFO:tasks.rados.rados.0.trial178.stdout:2921: finishing write tid 4 to trial17818531-17 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: finishing write tid 5 to trial17818531-17 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: oid 17 updating version 0 to 1206 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: oid 17 updating version 1206 to 1208 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: oid 17 version 1208 is already newer than 1205 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: oid 17 version 1208 is already newer than 1207 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1208 (ObjNum 1031 snap 288 seq_num 1031) dirty exists 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2921: left oid 17 (ObjNum 1031 snap 288 seq_num 1031) 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2922: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2922: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2922: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1211 (ObjNum 1005 snap 283 seq_num 1005) dirty exists 2026-02-21T22:28:06.383 INFO:tasks.rados.rados.0.trial178.stdout:2919: done (3 left) 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2920: done (2 left) 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2921: done (1 left) 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2922: done (0 left) 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2923: rollback oid 50 current snap is 288 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 50 to 285 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2924: read oid 44 snap -1 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2924: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:28:06.384 INFO:tasks.rados.rados.0.trial178.stdout:2925: snap_remove snap 287 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2925: done (2 left) 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2926: delete oid 37 current snap is 288 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2923: finishing rollback tid 0 to trial17818531-50 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2923: finishing rollback tid 1 to trial17818531-50 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2923: finishing rollback tid 2 to trial17818531-50 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1585 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:07.181 INFO:tasks.rados.rados.0.trial178.stdout:2924: expect (ObjNum 908 snap 268 seq_num 908) 2026-02-21T22:28:07.182 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:07.246 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:07.336 INFO:tasks.rados.rados.0.trial178.stdout:2923: done (2 left) 2026-02-21T22:28:07.336 INFO:tasks.rados.rados.0.trial178.stdout:2924: done (1 left) 2026-02-21T22:28:07.336 INFO:tasks.rados.rados.0.trial178.stdout:2926: done (0 left) 2026-02-21T22:28:07.336 INFO:tasks.rados.rados.0.trial178.stdout:2927: delete oid 26 current snap is 288 2026-02-21T22:28:07.337 INFO:tasks.rados.rados.0.trial178.stdout:2927: done (0 left) 2026-02-21T22:28:07.338 INFO:tasks.rados.rados.0.trial178.stdout:2928: snap_create 2026-02-21T22:28:08.292 INFO:tasks.rados.rados.0.trial178.stdout:2928: done (0 left) 2026-02-21T22:28:08.292 INFO:tasks.rados.rados.0.trial178.stdout:2929: read oid 2 snap -1 2026-02-21T22:28:08.292 INFO:tasks.rados.rados.0.trial178.stdout:2929: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:28:08.292 INFO:tasks.rados.rados.0.trial178.stdout:2930: write (excl) oid 39 current snap is 289 2026-02-21T22:28:08.292 INFO:tasks.rados.rados.0.trial178.stdout:2930: seq_num 1034 ranges {578438=431474,1657087=493694,2778551=599887} 2026-02-21T22:28:08.298 INFO:tasks.rados.rados.0.trial178.stdout:2930: writing trial17818531-39 from 578438 to 1009912 tid 1 2026-02-21T22:28:08.307 INFO:tasks.rados.rados.0.trial178.stdout:2930: writing trial17818531-39 from 1657087 to 2150781 tid 2 2026-02-21T22:28:08.316 INFO:tasks.rados.rados.0.trial178.stdout:2930: writing trial17818531-39 from 2778551 to 3378438 tid 3 2026-02-21T22:28:08.316 INFO:tasks.rados.rados.0.trial178.stdout:2931: snap_create 2026-02-21T22:28:08.316 INFO:tasks.rados.rados.0.trial178.stdout:2929: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:28:08.361 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:08.409 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:08.457 INFO:tasks.rados.rados.0.trial178.stdout:2930: finishing write tid 1 to trial17818531-39 2026-02-21T22:28:08.457 INFO:tasks.rados.rados.0.trial178.stdout:2930: finishing write tid 2 to trial17818531-39 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: finishing write tid 3 to trial17818531-39 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: finishing write tid 4 to trial17818531-39 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: finishing write tid 5 to trial17818531-39 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: oid 39 updating version 0 to 1226 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: oid 39 version 1226 is already newer than 1225 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: oid 39 updating version 1226 to 1227 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: oid 39 updating version 1227 to 1228 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1228 (ObjNum 1034 snap 289 seq_num 1034) dirty exists 2026-02-21T22:28:08.458 INFO:tasks.rados.rados.0.trial178.stdout:2930: left oid 39 (ObjNum 1034 snap 289 seq_num 1034) 2026-02-21T22:28:08.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:08.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:09.293 INFO:tasks.rados.rados.0.trial178.stdout:2929: done (2 left) 2026-02-21T22:28:09.294 INFO:tasks.rados.rados.0.trial178.stdout:2930: done (1 left) 2026-02-21T22:28:09.294 INFO:tasks.rados.rados.0.trial178.stdout:2931: done (0 left) 2026-02-21T22:28:09.294 INFO:tasks.rados.rados.0.trial178.stdout:2932: read oid 23 snap -1 2026-02-21T22:28:09.294 INFO:tasks.rados.rados.0.trial178.stdout:2932: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:28:09.294 INFO:tasks.rados.rados.0.trial178.stdout:2933: delete oid 46 current snap is 290 2026-02-21T22:28:09.302 INFO:tasks.rados.rados.0.trial178.stdout:2932: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:28:09.344 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:09.370 INFO:tasks.rados.rados.0.trial178.stdout:2932: done (1 left) 2026-02-21T22:28:09.371 INFO:tasks.rados.rados.0.trial178.stdout:2933: done (0 left) 2026-02-21T22:28:09.371 INFO:tasks.rados.rados.0.trial178.stdout:2934: copy_from oid 30 from oid 34 current snap is 290 2026-02-21T22:28:09.371 INFO:tasks.rados.rados.0.trial178.stdout:2935: read oid 12 snap -1 2026-02-21T22:28:09.371 INFO:tasks.rados.rados.0.trial178.stdout:2935: expect deleted 2026-02-21T22:28:09.371 INFO:tasks.rados.rados.0.trial178.stdout:2936: delete oid 42 current snap is 290 2026-02-21T22:28:09.374 INFO:tasks.rados.rados.0.trial178.stdout:2935: done (2 left) 2026-02-21T22:28:09.374 INFO:tasks.rados.rados.0.trial178.stdout:2936: done (1 left) 2026-02-21T22:28:09.374 INFO:tasks.rados.rados.0.trial178.stdout:2937: read oid 41 snap 288 2026-02-21T22:28:09.374 INFO:tasks.rados.rados.0.trial178.stdout:2937: expect deleted 2026-02-21T22:28:09.374 INFO:tasks.rados.rados.0.trial178.stdout:2938: snap_create 2026-02-21T22:28:09.386 INFO:tasks.rados.rados.0.trial178.stdout:2934: finishing copy_from racing read to trial17818531-30 2026-02-21T22:28:09.387 INFO:tasks.rados.rados.0.trial178.stdout:2934: finishing copy_from to trial17818531-30 2026-02-21T22:28:09.388 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1180 (ObjNum 1019 snap 283 seq_num 1019) dirty exists 2026-02-21T22:28:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2934: done (2 left) 2026-02-21T22:28:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2937: done (1 left) 2026-02-21T22:28:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2938: done (0 left) 2026-02-21T22:28:10.298 INFO:tasks.rados.rados.0.trial178.stdout:2939: delete oid 6 current snap is 291 2026-02-21T22:28:10.302 INFO:tasks.rados.rados.0.trial178.stdout:2939: done (0 left) 2026-02-21T22:28:10.302 INFO:tasks.rados.rados.0.trial178.stdout:2940: read oid 35 snap -1 2026-02-21T22:28:10.302 INFO:tasks.rados.rados.0.trial178.stdout:2940: expect (ObjNum 1018 snap 283 seq_num 1018) 2026-02-21T22:28:10.302 INFO:tasks.rados.rados.0.trial178.stdout:2941: snap_create 2026-02-21T22:28:10.310 INFO:tasks.rados.rados.0.trial178.stdout:2940: expect (ObjNum 1018 snap 283 seq_num 1018) 2026-02-21T22:28:10.310 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:10.459 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2998348 2026-02-21T22:28:11.361 INFO:tasks.rados.rados.0.trial178.stdout:2940: done (1 left) 2026-02-21T22:28:11.361 INFO:tasks.rados.rados.0.trial178.stdout:2941: done (0 left) 2026-02-21T22:28:11.361 INFO:tasks.rados.rados.0.trial178.stdout:2942: write oid 47 current snap is 292 2026-02-21T22:28:11.361 INFO:tasks.rados.rados.0.trial178.stdout:2942: seq_num 1038 ranges {645666=732470,2174662=501032,3288243=157423} 2026-02-21T22:28:11.370 INFO:tasks.rados.rados.0.trial178.stdout:2942: writing trial17818531-47 from 645666 to 1378136 tid 1 2026-02-21T22:28:11.378 INFO:tasks.rados.rados.0.trial178.stdout:2942: writing trial17818531-47 from 2174662 to 2675694 tid 2 2026-02-21T22:28:11.384 INFO:tasks.rados.rados.0.trial178.stdout:2942: writing trial17818531-47 from 3288243 to 3445666 tid 3 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2943: read oid 50 snap -1 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2943: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2944: read oid 16 snap -1 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2944: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2945: rollback oid 9 current snap is 292 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 9 to 289 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2942: finishing write tid 1 to trial17818531-47 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2942: finishing write tid 2 to trial17818531-47 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2946: write (excl) oid 7 current snap is 292 2026-02-21T22:28:11.385 INFO:tasks.rados.rados.0.trial178.stdout:2946: seq_num 1039 ranges {662473=407719,1833144=644690,3199313=263160} 2026-02-21T22:28:11.392 INFO:tasks.rados.rados.0.trial178.stdout:2946: writing trial17818531-7 from 662473 to 1070192 tid 1 2026-02-21T22:28:11.402 INFO:tasks.rados.rados.0.trial178.stdout:2946: writing trial17818531-7 from 1833144 to 2477834 tid 2 2026-02-21T22:28:11.409 INFO:tasks.rados.rados.0.trial178.stdout:2946: writing trial17818531-7 from 3199313 to 3462473 tid 3 2026-02-21T22:28:11.409 INFO:tasks.rados.rados.0.trial178.stdout:2947: snap_create 2026-02-21T22:28:11.409 INFO:tasks.rados.rados.0.trial178.stdout:2944: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:11.482 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:11.514 INFO:tasks.rados.rados.0.trial178.stdout:2942: finishing write tid 3 to trial17818531-47 2026-02-21T22:28:11.514 INFO:tasks.rados.rados.0.trial178.stdout:2942: finishing write tid 4 to trial17818531-47 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: finishing write tid 5 to trial17818531-47 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: oid 47 updating version 0 to 1616 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: oid 47 updating version 1616 to 1617 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: oid 47 version 1617 is already newer than 1614 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: oid 47 version 1617 is already newer than 1615 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1617 (ObjNum 1038 snap 292 seq_num 1038) dirty exists 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2942: left oid 47 (ObjNum 1038 snap 292 seq_num 1038) 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2945: finishing rollback tid 1 to trial17818531-9 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 0 (ObjNum 2625 snap 0 seq_num 2885558880) dirty dne 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:2943: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:11.515 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:11.594 INFO:tasks.rados.rados.0.trial178.stdout:2946: finishing write tid 1 to trial17818531-7 2026-02-21T22:28:11.594 INFO:tasks.rados.rados.0.trial178.stdout:2946: finishing write tid 2 to trial17818531-7 2026-02-21T22:28:11.594 INFO:tasks.rados.rados.0.trial178.stdout:2946: finishing write tid 3 to trial17818531-7 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: finishing write tid 4 to trial17818531-7 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: finishing write tid 5 to trial17818531-7 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: oid 7 updating version 0 to 1621 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: oid 7 version 1621 is already newer than 1619 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: oid 7 version 1621 is already newer than 1620 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: oid 7 version 1621 is already newer than 1618 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1621 (ObjNum 1039 snap 292 seq_num 1039) dirty exists 2026-02-21T22:28:11.595 INFO:tasks.rados.rados.0.trial178.stdout:2946: left oid 7 (ObjNum 1039 snap 292 seq_num 1039) 2026-02-21T22:28:12.181 INFO:tasks.rados.rados.0.trial178.stdout:2942: done (5 left) 2026-02-21T22:28:12.181 INFO:tasks.rados.rados.0.trial178.stdout:2943: done (4 left) 2026-02-21T22:28:12.181 INFO:tasks.rados.rados.0.trial178.stdout:2944: done (3 left) 2026-02-21T22:28:12.182 INFO:tasks.rados.rados.0.trial178.stdout:2945: done (2 left) 2026-02-21T22:28:12.182 INFO:tasks.rados.rados.0.trial178.stdout:2946: done (1 left) 2026-02-21T22:28:12.182 INFO:tasks.rados.rados.0.trial178.stdout:2947: done (0 left) 2026-02-21T22:28:12.182 INFO:tasks.rados.rados.0.trial178.stdout:2948: delete oid 9 current snap is 293 2026-02-21T22:28:12.183 INFO:tasks.rados.rados.0.trial178.stdout:2948: done (0 left) 2026-02-21T22:28:12.184 INFO:tasks.rados.rados.0.trial178.stdout:2949: snap_remove snap 290 2026-02-21T22:28:13.187 INFO:tasks.rados.rados.0.trial178.stdout:2949: done (0 left) 2026-02-21T22:28:13.187 INFO:tasks.rados.rados.0.trial178.stdout:2950: read oid 19 snap 288 2026-02-21T22:28:13.187 INFO:tasks.rados.rados.0.trial178.stdout:2950: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:28:13.187 INFO:tasks.rados.rados.0.trial178.stdout:2951: snap_remove snap 292 2026-02-21T22:28:13.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:13.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:14.188 INFO:tasks.rados.rados.0.trial178.stdout:2951: done (1 left) 2026-02-21T22:28:14.188 INFO:tasks.rados.rados.0.trial178.stdout:2952: snap_create 2026-02-21T22:28:14.188 INFO:tasks.rados.rados.0.trial178.stdout:2950: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:28:14.304 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:15.189 INFO:tasks.rados.rados.0.trial178.stdout:2950: done (1 left) 2026-02-21T22:28:15.189 INFO:tasks.rados.rados.0.trial178.stdout:2952: done (0 left) 2026-02-21T22:28:15.189 INFO:tasks.rados.rados.0.trial178.stdout:2953: delete oid 43 current snap is 294 2026-02-21T22:28:15.190 INFO:tasks.rados.rados.0.trial178.stdout:2953: done (0 left) 2026-02-21T22:28:15.190 INFO:tasks.rados.rados.0.trial178.stdout:2954: snap_remove snap 288 2026-02-21T22:28:16.194 INFO:tasks.rados.rados.0.trial178.stdout:2954: done (0 left) 2026-02-21T22:28:16.194 INFO:tasks.rados.rados.0.trial178.stdout:2955: write (excl) oid 29 current snap is 294 2026-02-21T22:28:16.195 INFO:tasks.rados.rados.0.trial178.stdout:2955: seq_num 1042 ranges {712894=749819,2008590=792017,3281582=231312} 2026-02-21T22:28:16.204 INFO:tasks.rados.rados.0.trial178.stdout:2955: writing trial17818531-29 from 712894 to 1462713 tid 1 2026-02-21T22:28:16.213 INFO:tasks.rados.rados.0.trial178.stdout:2955: writing trial17818531-29 from 2008590 to 2800607 tid 2 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2955: writing trial17818531-29 from 3281582 to 3512894 tid 3 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2956: read oid 23 snap 289 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2956: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2957: read oid 17 snap -1 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2957: expect (ObjNum 1031 snap 288 seq_num 1031) 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2958: write (excl) oid 32 current snap is 294 2026-02-21T22:28:16.218 INFO:tasks.rados.rados.0.trial178.stdout:2958: seq_num 1043 ranges {729701=425068,1667072=652028,2909005=485180} 2026-02-21T22:28:16.225 INFO:tasks.rados.rados.0.trial178.stdout:2958: writing trial17818531-32 from 729701 to 1154769 tid 1 2026-02-21T22:28:16.233 INFO:tasks.rados.rados.0.trial178.stdout:2958: writing trial17818531-32 from 1667072 to 2319100 tid 2 2026-02-21T22:28:16.239 INFO:tasks.rados.rados.0.trial178.stdout:2958: writing trial17818531-32 from 2909005 to 3394185 tid 3 2026-02-21T22:28:16.239 INFO:tasks.rados.rados.0.trial178.stdout:2959: read oid 27 snap -1 2026-02-21T22:28:16.239 INFO:tasks.rados.rados.0.trial178.stdout:2959: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:28:16.239 INFO:tasks.rados.rados.0.trial178.stdout:2960: snap_remove snap 285 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2960: done (5 left) 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2961: snap_create 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: finishing write tid 1 to trial17818531-29 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: finishing write tid 2 to trial17818531-29 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: finishing write tid 3 to trial17818531-29 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: finishing write tid 4 to trial17818531-29 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: finishing write tid 5 to trial17818531-29 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: oid 29 updating version 0 to 1184 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: oid 29 version 1184 is already newer than 1182 2026-02-21T22:28:17.184 INFO:tasks.rados.rados.0.trial178.stdout:2955: oid 29 version 1184 is already newer than 1181 2026-02-21T22:28:17.185 INFO:tasks.rados.rados.0.trial178.stdout:2955: oid 29 version 1184 is already newer than 1183 2026-02-21T22:28:17.185 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1184 (ObjNum 1042 snap 294 seq_num 1042) dirty exists 2026-02-21T22:28:17.185 INFO:tasks.rados.rados.0.trial178.stdout:2955: left oid 29 (ObjNum 1042 snap 294 seq_num 1042) 2026-02-21T22:28:17.185 INFO:tasks.rados.rados.0.trial178.stdout:2956: expect (ObjNum 691 snap 190 seq_num 691) 2026-02-21T22:28:17.202 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2583995 2026-02-21T22:28:17.217 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1017362 2026-02-21T22:28:17.244 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:17.270 INFO:tasks.rados.rados.0.trial178.stdout:2957: expect (ObjNum 1031 snap 288 seq_num 1031) 2026-02-21T22:28:17.329 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 514821 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: finishing write tid 1 to trial17818531-32 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: finishing write tid 2 to trial17818531-32 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: finishing write tid 3 to trial17818531-32 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: finishing write tid 4 to trial17818531-32 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: finishing write tid 5 to trial17818531-32 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: oid 32 updating version 0 to 1451 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: oid 32 version 1451 is already newer than 1449 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: oid 32 version 1451 is already newer than 1450 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: oid 32 updating version 1451 to 1452 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1452 (ObjNum 1043 snap 294 seq_num 1043) dirty exists 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2958: left oid 32 (ObjNum 1043 snap 294 seq_num 1043) 2026-02-21T22:28:17.449 INFO:tasks.rados.rados.0.trial178.stdout:2959: expect (ObjNum 920 snap 270 seq_num 920) 2026-02-21T22:28:17.453 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 946358 2026-02-21T22:28:17.510 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:18.187 INFO:tasks.rados.rados.0.trial178.stdout:2955: done (5 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2956: done (4 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2957: done (3 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2958: done (2 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2959: done (1 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2961: done (0 left) 2026-02-21T22:28:18.188 INFO:tasks.rados.rados.0.trial178.stdout:2962: snap_create 2026-02-21T22:28:18.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:18.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:19.188 INFO:tasks.rados.rados.0.trial178.stdout:2962: done (0 left) 2026-02-21T22:28:19.189 INFO:tasks.rados.rados.0.trial178.stdout:2963: snap_remove snap 294 2026-02-21T22:28:20.193 INFO:tasks.rados.rados.0.trial178.stdout:2963: done (0 left) 2026-02-21T22:28:20.193 INFO:tasks.rados.rados.0.trial178.stdout:2964: snap_remove snap 295 2026-02-21T22:28:21.195 INFO:tasks.rados.rados.0.trial178.stdout:2964: done (0 left) 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2965: copy_from oid 43 from oid 33 current snap is 296 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2966: rollback oid 37 current snap is 296 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 291 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2967: copy_from oid 35 from oid 14 current snap is 296 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2968: read oid 45 snap 293 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2968: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2969: rollback oid 41 current snap is 296 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 41 to 289 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2970: read oid 5 snap -1 2026-02-21T22:28:21.196 INFO:tasks.rados.rados.0.trial178.stdout:2970: expect deleted 2026-02-21T22:28:21.197 INFO:tasks.rados.rados.0.trial178.stdout:2971: rollback oid 15 current snap is 296 2026-02-21T22:28:21.197 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 15 to 289 2026-02-21T22:28:21.197 INFO:tasks.rados.rados.0.trial178.stdout:2972: snap_create 2026-02-21T22:28:21.197 INFO:tasks.rados.rados.0.trial178.stdout:2966: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:28:21.197 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 0 (ObjNum 81 snap 0 seq_num 1397764501) dirty dne 2026-02-21T22:28:21.198 INFO:tasks.rados.rados.0.trial178.stdout:2969: finishing rollback tid 1 to trial17818531-41 2026-02-21T22:28:21.198 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 0 (ObjNum 81 snap 0 seq_num 1397681757) dirty dne 2026-02-21T22:28:21.203 INFO:tasks.rados.rados.0.trial178.stdout:2968: expect (ObjNum 927 snap 271 seq_num 927) 2026-02-21T22:28:21.245 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:21.280 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2013722 2026-02-21T22:28:21.291 INFO:tasks.rados.rados.0.trial178.stdout:2971: finishing rollback tid 0 to trial17818531-15 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2971: finishing rollback tid 1 to trial17818531-15 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2971: finishing rollback tid 2 to trial17818531-15 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1214 (ObjNum 1005 snap 283 seq_num 1005) dirty exists 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2965: finishing copy_from racing read to trial17818531-43 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2965: finishing copy_from to trial17818531-43 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2967: finishing copy_from racing read to trial17818531-35 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:2967: finishing copy_from to trial17818531-35 2026-02-21T22:28:21.292 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1224 (ObjNum 1022 snap 283 seq_num 1022) dirty exists 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2965: done (7 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2966: done (6 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2967: done (5 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2968: done (4 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2969: done (3 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2970: done (2 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2971: done (1 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2972: done (0 left) 2026-02-21T22:28:22.186 INFO:tasks.rados.rados.0.trial178.stdout:2973: read oid 6 snap -1 2026-02-21T22:28:22.187 INFO:tasks.rados.rados.0.trial178.stdout:2973: expect deleted 2026-02-21T22:28:22.187 INFO:tasks.rados.rados.0.trial178.stdout:2974: write oid 34 current snap is 297 2026-02-21T22:28:22.187 INFO:tasks.rados.rados.0.trial178.stdout:2974: seq_num 1044 ranges {746508=500317,2009201=795686,3220075=326433} 2026-02-21T22:28:22.193 INFO:tasks.rados.rados.0.trial178.stdout:2974: writing trial17818531-34 from 746508 to 1246825 tid 1 2026-02-21T22:28:22.204 INFO:tasks.rados.rados.0.trial178.stdout:2974: writing trial17818531-34 from 2009201 to 2804887 tid 2 2026-02-21T22:28:22.209 INFO:tasks.rados.rados.0.trial178.stdout:2974: writing trial17818531-34 from 3220075 to 3546508 tid 3 2026-02-21T22:28:22.209 INFO:tasks.rados.rados.0.trial178.stdout:2975: snap_create 2026-02-21T22:28:22.209 INFO:tasks.rados.rados.0.trial178.stdout:2974: finishing write tid 1 to trial17818531-34 2026-02-21T22:28:22.209 INFO:tasks.rados.rados.0.trial178.stdout:2974: finishing write tid 2 to trial17818531-34 2026-02-21T22:28:22.211 INFO:tasks.rados.rados.0.trial178.stdout:2974: finishing write tid 3 to trial17818531-34 2026-02-21T22:28:22.211 INFO:tasks.rados.rados.0.trial178.stdout:2974: finishing write tid 4 to trial17818531-34 2026-02-21T22:28:22.211 INFO:tasks.rados.rados.0.trial178.stdout:2974: finishing write tid 5 to trial17818531-34 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:2974: oid 34 updating version 0 to 1185 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:2974: oid 34 updating version 1185 to 1187 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:2974: oid 34 version 1187 is already newer than 1186 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:2974: oid 34 updating version 1187 to 1188 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1188 (ObjNum 1044 snap 297 seq_num 1044) dirty exists 2026-02-21T22:28:22.212 INFO:tasks.rados.rados.0.trial178.stdout:2974: left oid 34 (ObjNum 1044 snap 297 seq_num 1044) 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:2973: done (2 left) 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:2974: done (1 left) 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:2975: done (0 left) 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:2976: rollback oid 4 current snap is 298 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 289 2026-02-21T22:28:23.201 INFO:tasks.rados.rados.0.trial178.stdout:2977: delete oid 34 current snap is 298 2026-02-21T22:28:23.203 INFO:tasks.rados.rados.0.trial178.stdout:2977: done (1 left) 2026-02-21T22:28:23.203 INFO:tasks.rados.rados.0.trial178.stdout:2978: rollback oid 39 current snap is 298 2026-02-21T22:28:23.203 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 296 2026-02-21T22:28:23.203 INFO:tasks.rados.rados.0.trial178.stdout:2979: snap_create 2026-02-21T22:28:23.205 INFO:tasks.rados.rados.0.trial178.stdout:2976: finishing rollback tid 0 to trial17818531-4 2026-02-21T22:28:23.205 INFO:tasks.rados.rados.0.trial178.stdout:2976: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:28:23.205 INFO:tasks.rados.rados.0.trial178.stdout:2976: finishing rollback tid 2 to trial17818531-4 2026-02-21T22:28:23.205 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1457 (ObjNum 981 snap 281 seq_num 981) dirty exists 2026-02-21T22:28:23.206 INFO:tasks.rados.rados.0.trial178.stdout:2978: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:28:23.206 INFO:tasks.rados.rados.0.trial178.stdout:2978: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:28:23.206 INFO:tasks.rados.rados.0.trial178.stdout:2978: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:28:23.206 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1231 (ObjNum 1034 snap 289 seq_num 1034) dirty exists 2026-02-21T22:28:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:28:23] 2026-02-21T22:28:23.535 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:28:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:28:23, mode upmap, max misplaced 0.050000 2026-02-21T22:28:23.536 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:28:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:28:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'unique_pool_0', 'rbd'] 2026-02-21T22:28:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:28:23.537 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:28:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:23.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:28:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:28:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:28:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:28:23.565 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:28:23.566 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:28:23.567 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:28:23.568 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:28:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:28:23.570 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:28:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:28:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:28:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:28:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:28:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:28:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:28:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:28:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:28:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:28:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:28:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:28:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:28:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:28:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:28:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:28:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:28:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:28:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:28:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:28:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:28:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "616", 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:28:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:28:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:28:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:28:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:28:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:28:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:28:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 616, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 304, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:28:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:28:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:28:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:28:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:28:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0002524764449508102 of space, bias 1.0, pg target 0.07574293348524307 quantized to 32 (current 16) 2026-02-21T22:28:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:28:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:28:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:28:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:24.206 INFO:tasks.rados.rados.0.trial178.stdout:2976: done (2 left) 2026-02-21T22:28:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2978: done (1 left) 2026-02-21T22:28:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2979: done (0 left) 2026-02-21T22:28:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2980: read oid 37 snap 291 2026-02-21T22:28:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2980: expect deleted 2026-02-21T22:28:24.207 INFO:tasks.rados.rados.0.trial178.stdout:2981: snap_create 2026-02-21T22:28:25.207 INFO:tasks.rados.rados.0.trial178.stdout:2980: done (1 left) 2026-02-21T22:28:25.207 INFO:tasks.rados.rados.0.trial178.stdout:2981: done (0 left) 2026-02-21T22:28:25.207 INFO:tasks.rados.rados.0.trial178.stdout:2982: write oid 7 current snap is 300 2026-02-21T22:28:25.207 INFO:tasks.rados.rados.0.trial178.stdout:2982: seq_num 1046 ranges {780122=650815,2126165=799355} 2026-02-21T22:28:25.218 INFO:tasks.rados.rados.0.trial178.stdout:2982: writing trial17818531-7 from 780122 to 1430937 tid 1 2026-02-21T22:28:25.228 INFO:tasks.rados.rados.0.trial178.stdout:2982: writing trial17818531-7 from 2126165 to 2925520 tid 2 2026-02-21T22:28:25.228 INFO:tasks.rados.rados.0.trial178.stdout:2983: copy_from oid 28 from oid 6 current snap is 300 2026-02-21T22:28:25.228 INFO:tasks.rados.rados.0.trial178.stdout:2982: finishing write tid 1 to trial17818531-7 2026-02-21T22:28:25.229 INFO:tasks.rados.rados.0.trial178.stdout:2984: copy_from oid 49 from oid 34 current snap is 300 2026-02-21T22:28:25.229 INFO:tasks.rados.rados.0.trial178.stdout:2985: write (excl) oid 27 current snap is 300 2026-02-21T22:28:25.229 INFO:tasks.rados.rados.0.trial178.stdout:2985: seq_num 1047 ranges {796929=726064,2184647=659366,3302344=294585} 2026-02-21T22:28:25.238 INFO:tasks.rados.rados.0.trial178.stdout:2985: writing trial17818531-27 from 796929 to 1522993 tid 1 2026-02-21T22:28:25.247 INFO:tasks.rados.rados.0.trial178.stdout:2985: writing trial17818531-27 from 2184647 to 2844013 tid 2 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2985: writing trial17818531-27 from 3302344 to 3596929 tid 3 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2986: rollback oid 3 current snap is 300 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 289 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2983: finishing copy_from to trial17818531-28 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2983: got expected ENOENT (src dne) 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2983: finishing copy_from racing read to trial17818531-28 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2984: finishing copy_from to trial17818531-49 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2984: got expected ENOENT (src dne) 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2983: done (4 left) 2026-02-21T22:28:25.253 INFO:tasks.rados.rados.0.trial178.stdout:2987: read oid 48 snap -1 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2987: expect (ObjNum 995 snap 281 seq_num 995) 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2988: snap_create 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2984: finishing copy_from racing read to trial17818531-49 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: finishing write tid 2 to trial17818531-7 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: finishing write tid 3 to trial17818531-7 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: finishing write tid 4 to trial17818531-7 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: oid 7 updating version 0 to 1630 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: oid 7 updating version 1630 to 1631 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: oid 7 version 1631 is already newer than 1629 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1631 (ObjNum 1046 snap 300 seq_num 1046) dirty exists 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2982: left oid 7 (ObjNum 1046 snap 300 seq_num 1046) 2026-02-21T22:28:25.254 INFO:tasks.rados.rados.0.trial178.stdout:2985: finishing write tid 1 to trial17818531-27 2026-02-21T22:28:25.255 INFO:tasks.rados.rados.0.trial178.stdout:2985: finishing write tid 2 to trial17818531-27 2026-02-21T22:28:25.256 INFO:tasks.rados.rados.0.trial178.stdout:2986: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:28:25.256 INFO:tasks.rados.rados.0.trial178.stdout:2986: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:28:25.256 INFO:tasks.rados.rados.0.trial178.stdout:2986: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:28:25.256 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1455 (ObjNum 1017 snap 283 seq_num 1017) dirty exists 2026-02-21T22:28:25.257 INFO:tasks.rados.rados.0.trial178.stdout:2987: expect (ObjNum 995 snap 281 seq_num 995) 2026-02-21T22:28:25.257 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:25.301 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1472226 2026-02-21T22:28:25.330 INFO:tasks.rados.rados.0.trial178.stdout:2985: finishing write tid 3 to trial17818531-27 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: finishing write tid 4 to trial17818531-27 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: finishing write tid 5 to trial17818531-27 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: oid 27 updating version 0 to 1234 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: oid 27 updating version 1234 to 1235 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: oid 27 version 1235 is already newer than 1232 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: oid 27 version 1235 is already newer than 1233 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1235 (ObjNum 1047 snap 300 seq_num 1047) dirty exists 2026-02-21T22:28:25.331 INFO:tasks.rados.rados.0.trial178.stdout:2985: left oid 27 (ObjNum 1047 snap 300 seq_num 1047) 2026-02-21T22:28:26.214 INFO:tasks.rados.rados.0.trial178.stdout:2982: done (5 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2984: done (4 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2985: done (3 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2986: done (2 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2987: done (1 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2988: done (0 left) 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2989: rollback oid 12 current snap is 301 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 293 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2990: read oid 38 snap -1 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2990: expect deleted 2026-02-21T22:28:26.215 INFO:tasks.rados.rados.0.trial178.stdout:2991: snap_remove snap 296 2026-02-21T22:28:27.184 INFO:tasks.rados.rados.0.trial178.stdout:2991: done (2 left) 2026-02-21T22:28:27.184 INFO:tasks.rados.rados.0.trial178.stdout:2992: snap_create 2026-02-21T22:28:27.184 INFO:tasks.rados.rados.0.trial178.stdout:2989: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:28:27.184 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 0 (ObjNum 33 snap 0 seq_num 101485296) dirty dne 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2989: done (2 left) 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2990: done (1 left) 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2992: done (0 left) 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2993: read oid 31 snap -1 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2993: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:28:28.188 INFO:tasks.rados.rados.0.trial178.stdout:2994: snap_create 2026-02-21T22:28:28.197 INFO:tasks.rados.rados.0.trial178.stdout:2993: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:28:28.197 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:28.276 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1423588 2026-02-21T22:28:28.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:28.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:29.188 INFO:tasks.rados.rados.0.trial178.stdout:2993: done (1 left) 2026-02-21T22:28:29.189 INFO:tasks.rados.rados.0.trial178.stdout:2994: done (0 left) 2026-02-21T22:28:29.189 INFO:tasks.rados.rados.0.trial178.stdout:2995: snap_remove snap 293 2026-02-21T22:28:30.194 INFO:tasks.rados.rados.0.trial178.stdout:2995: done (0 left) 2026-02-21T22:28:30.194 INFO:tasks.rados.rados.0.trial178.stdout:2996: write (excl) oid 15 current snap is 303 2026-02-21T22:28:30.194 INFO:tasks.rados.rados.0.trial178.stdout:2996: seq_num 1048 ranges {413736=401313,1326776=403024,2413414=457893,3438716=175020} 2026-02-21T22:28:30.200 INFO:tasks.rados.rados.0.trial178.stdout:2996: writing trial17818531-15 from 413736 to 815049 tid 1 2026-02-21T22:28:30.206 INFO:tasks.rados.rados.0.trial178.stdout:2996: writing trial17818531-15 from 1326776 to 1729800 tid 2 2026-02-21T22:28:30.214 INFO:tasks.rados.rados.0.trial178.stdout:2996: writing trial17818531-15 from 2413414 to 2871307 tid 3 2026-02-21T22:28:30.219 INFO:tasks.rados.rados.0.trial178.stdout:2996: writing trial17818531-15 from 3438716 to 3613736 tid 4 2026-02-21T22:28:30.219 INFO:tasks.rados.rados.0.trial178.stdout:2997: write oid 22 current snap is 303 2026-02-21T22:28:30.219 INFO:tasks.rados.rados.0.trial178.stdout:2997: seq_num 1049 ranges {430543=476562,1385258=663035,2840837=669165} 2026-02-21T22:28:30.225 INFO:tasks.rados.rados.0.trial178.stdout:2997: writing trial17818531-22 from 430543 to 907105 tid 1 2026-02-21T22:28:30.233 INFO:tasks.rados.rados.0.trial178.stdout:2997: writing trial17818531-22 from 1385258 to 2048293 tid 2 2026-02-21T22:28:30.241 INFO:tasks.rados.rados.0.trial178.stdout:2997: writing trial17818531-22 from 2840837 to 3510002 tid 3 2026-02-21T22:28:30.241 INFO:tasks.rados.rados.0.trial178.stdout:2998: write (excl) oid 34 current snap is 303 2026-02-21T22:28:30.241 INFO:tasks.rados.rados.0.trial178.stdout:2998: seq_num 1050 ranges {447350=668164,1560093=406693,2584613=480437} 2026-02-21T22:28:30.245 INFO:tasks.rados.rados.0.trial178.stdout:2998: writing trial17818531-34 from 447350 to 1115514 tid 1 2026-02-21T22:28:30.248 INFO:tasks.rados.rados.0.trial178.stdout:2998: writing trial17818531-34 from 1560093 to 1966786 tid 2 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2998: writing trial17818531-34 from 2584613 to 3065050 tid 3 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2999: snap_create 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 1 to trial17818531-15 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 2 to trial17818531-15 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 3 to trial17818531-15 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 4 to trial17818531-15 2026-02-21T22:28:30.253 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 5 to trial17818531-15 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: finishing write tid 6 to trial17818531-15 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: oid 15 updating version 0 to 1217 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: oid 15 updating version 1217 to 1218 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: oid 15 version 1218 is already newer than 1216 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: oid 15 updating version 1218 to 1219 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: oid 15 version 1219 is already newer than 1215 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1219 (ObjNum 1048 snap 303 seq_num 1048) dirty exists 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2996: left oid 15 (ObjNum 1048 snap 303 seq_num 1048) 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2997: finishing write tid 1 to trial17818531-22 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2997: finishing write tid 2 to trial17818531-22 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2997: finishing write tid 3 to trial17818531-22 2026-02-21T22:28:30.254 INFO:tasks.rados.rados.0.trial178.stdout:2997: finishing write tid 4 to trial17818531-22 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: finishing write tid 5 to trial17818531-22 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: oid 22 updating version 0 to 1586 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: oid 22 updating version 1586 to 1589 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: oid 22 version 1589 is already newer than 1588 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: oid 22 version 1589 is already newer than 1587 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1589 (ObjNum 1049 snap 303 seq_num 1049) dirty exists 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2997: left oid 22 (ObjNum 1049 snap 303 seq_num 1049) 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2998: finishing write tid 1 to trial17818531-34 2026-02-21T22:28:30.255 INFO:tasks.rados.rados.0.trial178.stdout:2998: finishing write tid 2 to trial17818531-34 2026-02-21T22:28:30.256 INFO:tasks.rados.rados.0.trial178.stdout:2998: finishing write tid 3 to trial17818531-34 2026-02-21T22:28:30.256 INFO:tasks.rados.rados.0.trial178.stdout:2998: finishing write tid 4 to trial17818531-34 2026-02-21T22:28:30.256 INFO:tasks.rados.rados.0.trial178.stdout:2998: finishing write tid 5 to trial17818531-34 2026-02-21T22:28:30.256 INFO:tasks.rados.rados.0.trial178.stdout:2998: oid 34 updating version 0 to 1193 2026-02-21T22:28:30.256 INFO:tasks.rados.rados.0.trial178.stdout:2998: oid 34 version 1193 is already newer than 1190 2026-02-21T22:28:30.257 INFO:tasks.rados.rados.0.trial178.stdout:2998: oid 34 version 1193 is already newer than 1191 2026-02-21T22:28:30.257 INFO:tasks.rados.rados.0.trial178.stdout:2998: oid 34 version 1193 is already newer than 1192 2026-02-21T22:28:30.257 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1193 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:28:30.257 INFO:tasks.rados.rados.0.trial178.stdout:2998: left oid 34 (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:28:31.198 INFO:tasks.rados.rados.0.trial178.stdout:2996: done (3 left) 2026-02-21T22:28:31.199 INFO:tasks.rados.rados.0.trial178.stdout:2997: done (2 left) 2026-02-21T22:28:31.199 INFO:tasks.rados.rados.0.trial178.stdout:2998: done (1 left) 2026-02-21T22:28:31.199 INFO:tasks.rados.rados.0.trial178.stdout:2999: done (0 left) 2026-02-21T22:28:31.199 INFO:tasks.rados.rados.0.trial178.stdout:3000: write oid 18 current snap is 304 2026-02-21T22:28:31.199 INFO:tasks.rados.rados.0.trial178.stdout:3000: seq_num 1051 ranges {464157=743413,1618575=666704,3012036=408062} 2026-02-21T22:28:31.204 INFO:tasks.rados.rados.0.trial178.stdout:3000: writing trial17818531-18 from 464157 to 1207570 tid 1 2026-02-21T22:28:31.209 INFO:tasks.rados.rados.0.trial178.stdout:3000: writing trial17818531-18 from 1618575 to 2285279 tid 2 2026-02-21T22:28:31.213 INFO:tasks.rados.rados.0.trial178.stdout:3000: writing trial17818531-18 from 3012036 to 3420098 tid 3 2026-02-21T22:28:31.213 INFO:tasks.rados.rados.0.trial178.stdout:3001: read oid 17 snap -1 2026-02-21T22:28:31.213 INFO:tasks.rados.rados.0.trial178.stdout:3001: expect (ObjNum 1031 snap 288 seq_num 1031) 2026-02-21T22:28:31.213 INFO:tasks.rados.rados.0.trial178.stdout:3002: snap_remove snap 289 2026-02-21T22:28:32.185 INFO:tasks.rados.rados.0.trial178.stdout:3002: done (2 left) 2026-02-21T22:28:32.185 INFO:tasks.rados.rados.0.trial178.stdout:3003: delete oid 49 current snap is 304 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: finishing write tid 1 to trial17818531-18 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: finishing write tid 2 to trial17818531-18 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: finishing write tid 3 to trial17818531-18 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: finishing write tid 4 to trial17818531-18 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: finishing write tid 5 to trial17818531-18 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: oid 18 updating version 0 to 1442 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: oid 18 version 1442 is already newer than 1441 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: oid 18 version 1442 is already newer than 1440 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: oid 18 version 1442 is already newer than 1439 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1442 (ObjNum 1051 snap 304 seq_num 1051) dirty exists 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3000: left oid 18 (ObjNum 1051 snap 304 seq_num 1051) 2026-02-21T22:28:32.186 INFO:tasks.rados.rados.0.trial178.stdout:3001: expect (ObjNum 1031 snap 288 seq_num 1031) 2026-02-21T22:28:32.272 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:32.390 INFO:tasks.rados.rados.0.trial178.stdout:3000: done (2 left) 2026-02-21T22:28:32.390 INFO:tasks.rados.rados.0.trial178.stdout:3001: done (1 left) 2026-02-21T22:28:32.390 INFO:tasks.rados.rados.0.trial178.stdout:3003: done (0 left) 2026-02-21T22:28:32.390 INFO:tasks.rados.rados.0.trial178.stdout:3004: snap_create 2026-02-21T22:28:33.467 INFO:tasks.rados.rados.0.trial178.stdout:3004: done (0 left) 2026-02-21T22:28:33.468 INFO:tasks.rados.rados.0.trial178.stdout:3005: snap_remove snap 291 2026-02-21T22:28:33.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:33.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:34.249 INFO:tasks.rados.rados.0.trial178.stdout:3005: done (0 left) 2026-02-21T22:28:34.249 INFO:tasks.rados.rados.0.trial178.stdout:3006: delete oid 20 current snap is 305 2026-02-21T22:28:34.253 INFO:tasks.rados.rados.0.trial178.stdout:3006: done (0 left) 2026-02-21T22:28:34.253 INFO:tasks.rados.rados.0.trial178.stdout:3007: delete oid 1 current snap is 305 2026-02-21T22:28:34.255 INFO:tasks.rados.rados.0.trial178.stdout:3007: done (0 left) 2026-02-21T22:28:34.255 INFO:tasks.rados.rados.0.trial178.stdout:3008: copy_from oid 40 from oid 33 current snap is 305 2026-02-21T22:28:34.255 INFO:tasks.rados.rados.0.trial178.stdout:3009: write (excl) oid 41 current snap is 305 2026-02-21T22:28:34.255 INFO:tasks.rados.rados.0.trial178.stdout:3009: seq_num 1055 ranges {531385=644409,1736150=557689,3005375=569503} 2026-02-21T22:28:34.263 INFO:tasks.rados.rados.0.trial178.stdout:3009: writing trial17818531-41 from 531385 to 1175794 tid 1 2026-02-21T22:28:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3009: writing trial17818531-41 from 1736150 to 2293839 tid 2 2026-02-21T22:28:34.279 INFO:tasks.rados.rados.0.trial178.stdout:3009: writing trial17818531-41 from 3005375 to 3574878 tid 3 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:3010: delete oid 3 current snap is 305 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:3008: finishing copy_from racing read to trial17818531-40 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:3008: finishing copy_from to trial17818531-40 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:3009: finishing write tid 1 to trial17818531-41 2026-02-21T22:28:34.280 INFO:tasks.rados.rados.0.trial178.stdout:3009: finishing write tid 2 to trial17818531-41 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: finishing write tid 3 to trial17818531-41 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: finishing write tid 4 to trial17818531-41 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: finishing write tid 5 to trial17818531-41 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: oid 41 updating version 0 to 1221 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: oid 41 updating version 1221 to 1223 2026-02-21T22:28:34.282 INFO:tasks.rados.rados.0.trial178.stdout:3009: oid 41 updating version 1223 to 1224 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3009: oid 41 version 1224 is already newer than 1222 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1224 (ObjNum 1055 snap 305 seq_num 1055) dirty exists 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3009: left oid 41 (ObjNum 1055 snap 305 seq_num 1055) 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3008: done (2 left) 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3009: done (1 left) 2026-02-21T22:28:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3010: done (0 left) 2026-02-21T22:28:34.284 INFO:tasks.rados.rados.0.trial178.stdout:3011: snap_remove snap 301 2026-02-21T22:28:35.251 INFO:tasks.rados.rados.0.trial178.stdout:3011: done (0 left) 2026-02-21T22:28:35.251 INFO:tasks.rados.rados.0.trial178.stdout:3012: read oid 26 snap 304 2026-02-21T22:28:35.251 INFO:tasks.rados.rados.0.trial178.stdout:3012: expect deleted 2026-02-21T22:28:35.251 INFO:tasks.rados.rados.0.trial178.stdout:3013: read oid 40 snap 298 2026-02-21T22:28:35.252 INFO:tasks.rados.rados.0.trial178.stdout:3013: expect (ObjNum 970 snap 280 seq_num 970) 2026-02-21T22:28:35.252 INFO:tasks.rados.rados.0.trial178.stdout:3014: snap_create 2026-02-21T22:28:35.254 INFO:tasks.rados.rados.0.trial178.stdout:3013: expect (ObjNum 970 snap 280 seq_num 970) 2026-02-21T22:28:35.259 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1636368 2026-02-21T22:28:36.256 INFO:tasks.rados.rados.0.trial178.stdout:3012: done (2 left) 2026-02-21T22:28:36.256 INFO:tasks.rados.rados.0.trial178.stdout:3013: done (1 left) 2026-02-21T22:28:36.257 INFO:tasks.rados.rados.0.trial178.stdout:3014: done (0 left) 2026-02-21T22:28:36.257 INFO:tasks.rados.rados.0.trial178.stdout:3015: write (excl) oid 22 current snap is 306 2026-02-21T22:28:36.257 INFO:tasks.rados.rados.0.trial178.stdout:3015: seq_num 1057 ranges {564999=511260,1853114=561358,3060221=592047} 2026-02-21T22:28:36.264 INFO:tasks.rados.rados.0.trial178.stdout:3015: writing trial17818531-22 from 564999 to 1076259 tid 1 2026-02-21T22:28:36.273 INFO:tasks.rados.rados.0.trial178.stdout:3015: writing trial17818531-22 from 1853114 to 2414472 tid 2 2026-02-21T22:28:36.282 INFO:tasks.rados.rados.0.trial178.stdout:3015: writing trial17818531-22 from 3060221 to 3652268 tid 3 2026-02-21T22:28:36.282 INFO:tasks.rados.rados.0.trial178.stdout:3016: snap_remove snap 298 2026-02-21T22:28:37.186 INFO:tasks.rados.rados.0.trial178.stdout:3016: done (1 left) 2026-02-21T22:28:37.186 INFO:tasks.rados.rados.0.trial178.stdout:3017: snap_create 2026-02-21T22:28:37.186 INFO:tasks.rados.rados.0.trial178.stdout:3015: finishing write tid 1 to trial17818531-22 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: finishing write tid 2 to trial17818531-22 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: finishing write tid 3 to trial17818531-22 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: finishing write tid 4 to trial17818531-22 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: finishing write tid 5 to trial17818531-22 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: oid 22 updating version 0 to 1592 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: oid 22 version 1592 is already newer than 1591 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: oid 22 updating version 1592 to 1593 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: oid 22 version 1593 is already newer than 1590 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1593 (ObjNum 1057 snap 306 seq_num 1057) dirty exists 2026-02-21T22:28:37.187 INFO:tasks.rados.rados.0.trial178.stdout:3015: left oid 22 (ObjNum 1057 snap 306 seq_num 1057) 2026-02-21T22:28:38.190 INFO:tasks.rados.rados.0.trial178.stdout:3015: done (1 left) 2026-02-21T22:28:38.190 INFO:tasks.rados.rados.0.trial178.stdout:3017: done (0 left) 2026-02-21T22:28:38.190 INFO:tasks.rados.rados.0.trial178.stdout:3018: write oid 48 current snap is 307 2026-02-21T22:28:38.190 INFO:tasks.rados.rados.0.trial178.stdout:3018: seq_num 1058 ranges {581806=586509,1911596=421369,3087644=403319} 2026-02-21T22:28:38.197 INFO:tasks.rados.rados.0.trial178.stdout:3018: writing trial17818531-48 from 581806 to 1168315 tid 1 2026-02-21T22:28:38.205 INFO:tasks.rados.rados.0.trial178.stdout:3018: writing trial17818531-48 from 1911596 to 2332965 tid 2 2026-02-21T22:28:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3018: writing trial17818531-48 from 3087644 to 3490963 tid 3 2026-02-21T22:28:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3019: write (excl) oid 20 current snap is 307 2026-02-21T22:28:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3019: seq_num 1059 ranges {598613=661758,1970078=565027,3115067=614591} 2026-02-21T22:28:38.221 INFO:tasks.rados.rados.0.trial178.stdout:3019: writing trial17818531-20 from 598613 to 1260371 tid 1 2026-02-21T22:28:38.230 INFO:tasks.rados.rados.0.trial178.stdout:3019: writing trial17818531-20 from 1970078 to 2535105 tid 2 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3019: writing trial17818531-20 from 3115067 to 3729658 tid 3 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3020: copy_from oid 6 from oid 3 current snap is 307 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: finishing write tid 1 to trial17818531-48 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: finishing write tid 2 to trial17818531-48 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: finishing write tid 3 to trial17818531-48 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: finishing write tid 4 to trial17818531-48 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: finishing write tid 5 to trial17818531-48 2026-02-21T22:28:38.237 INFO:tasks.rados.rados.0.trial178.stdout:3018: oid 48 updating version 0 to 1330 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3018: oid 48 version 1330 is already newer than 1327 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3018: oid 48 version 1330 is already newer than 1328 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3018: oid 48 version 1330 is already newer than 1329 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1330 (ObjNum 1058 snap 307 seq_num 1058) dirty exists 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3018: left oid 48 (ObjNum 1058 snap 307 seq_num 1058) 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3019: finishing write tid 1 to trial17818531-20 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3019: finishing write tid 2 to trial17818531-20 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3018: done (2 left) 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3021: read oid 28 snap 304 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3021: expect (ObjNum 1001 snap 283 seq_num 1001) 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3022: delete oid 9 current snap is 307 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3020: finishing copy_from to trial17818531-6 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3020: got expected ENOENT (src dne) 2026-02-21T22:28:38.238 INFO:tasks.rados.rados.0.trial178.stdout:3020: finishing copy_from racing read to trial17818531-6 2026-02-21T22:28:38.239 INFO:tasks.rados.rados.0.trial178.stdout:3019: finishing write tid 3 to trial17818531-20 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: finishing write tid 4 to trial17818531-20 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: finishing write tid 5 to trial17818531-20 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: oid 20 updating version 0 to 1228 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: oid 20 version 1228 is already newer than 1225 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: oid 20 version 1228 is already newer than 1227 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: oid 20 version 1228 is already newer than 1226 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1228 (ObjNum 1059 snap 307 seq_num 1059) dirty exists 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: left oid 20 (ObjNum 1059 snap 307 seq_num 1059) 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3019: done (3 left) 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3020: done (2 left) 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3022: done (1 left) 2026-02-21T22:28:38.240 INFO:tasks.rados.rados.0.trial178.stdout:3023: read oid 46 snap 302 2026-02-21T22:28:38.241 INFO:tasks.rados.rados.0.trial178.stdout:3023: expect deleted 2026-02-21T22:28:38.241 INFO:tasks.rados.rados.0.trial178.stdout:3024: snap_create 2026-02-21T22:28:38.241 INFO:tasks.rados.rados.0.trial178.stdout:3021: expect (ObjNum 1001 snap 283 seq_num 1001) 2026-02-21T22:28:38.241 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:38.389 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2439979 2026-02-21T22:28:38.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:38.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:39.193 INFO:tasks.rados.rados.0.trial178.stdout:3021: done (2 left) 2026-02-21T22:28:39.193 INFO:tasks.rados.rados.0.trial178.stdout:3023: done (1 left) 2026-02-21T22:28:39.193 INFO:tasks.rados.rados.0.trial178.stdout:3024: done (0 left) 2026-02-21T22:28:39.193 INFO:tasks.rados.rados.0.trial178.stdout:3025: write oid 31 current snap is 308 2026-02-21T22:28:39.193 INFO:tasks.rados.rados.0.trial178.stdout:3025: seq_num 1061 ranges {632227=412256,1570689=568696,2653560=753488} 2026-02-21T22:28:39.199 INFO:tasks.rados.rados.0.trial178.stdout:3025: writing trial17818531-31 from 632227 to 1044483 tid 1 2026-02-21T22:28:39.207 INFO:tasks.rados.rados.0.trial178.stdout:3025: writing trial17818531-31 from 1570689 to 2139385 tid 2 2026-02-21T22:28:39.216 INFO:tasks.rados.rados.0.trial178.stdout:3025: writing trial17818531-31 from 2653560 to 3407048 tid 3 2026-02-21T22:28:39.216 INFO:tasks.rados.rados.0.trial178.stdout:3026: read oid 3 snap -1 2026-02-21T22:28:39.216 INFO:tasks.rados.rados.0.trial178.stdout:3026: expect deleted 2026-02-21T22:28:39.216 INFO:tasks.rados.rados.0.trial178.stdout:3027: write (excl) oid 7 current snap is 308 2026-02-21T22:28:39.217 INFO:tasks.rados.rados.0.trial178.stdout:3025: finishing write tid 1 to trial17818531-31 2026-02-21T22:28:39.217 INFO:tasks.rados.rados.0.trial178.stdout:3025: finishing write tid 2 to trial17818531-31 2026-02-21T22:28:39.217 INFO:tasks.rados.rados.0.trial178.stdout:3027: seq_num 1062 ranges {649034=487505,1629171=428707,2680983=564760,3798768=50266} 2026-02-21T22:28:39.224 INFO:tasks.rados.rados.0.trial178.stdout:3027: writing trial17818531-7 from 649034 to 1136539 tid 1 2026-02-21T22:28:39.230 INFO:tasks.rados.rados.0.trial178.stdout:3027: writing trial17818531-7 from 1629171 to 2057878 tid 2 2026-02-21T22:28:39.238 INFO:tasks.rados.rados.0.trial178.stdout:3027: writing trial17818531-7 from 2680983 to 3245743 tid 3 2026-02-21T22:28:39.242 INFO:tasks.rados.rados.0.trial178.stdout:3027: writing trial17818531-7 from 3798768 to 3849034 tid 4 2026-02-21T22:28:39.242 INFO:tasks.rados.rados.0.trial178.stdout:3028: snap_create 2026-02-21T22:28:39.242 INFO:tasks.rados.rados.0.trial178.stdout:3025: finishing write tid 3 to trial17818531-31 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: finishing write tid 4 to trial17818531-31 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: finishing write tid 5 to trial17818531-31 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: oid 31 updating version 0 to 1447 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: oid 31 version 1447 is already newer than 1446 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: oid 31 updating version 1447 to 1448 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: oid 31 updating version 1448 to 1449 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1449 (ObjNum 1061 snap 308 seq_num 1061) dirty exists 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3025: left oid 31 (ObjNum 1061 snap 308 seq_num 1061) 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 1 to trial17818531-7 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 2 to trial17818531-7 2026-02-21T22:28:39.243 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 3 to trial17818531-7 2026-02-21T22:28:39.244 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 4 to trial17818531-7 2026-02-21T22:28:39.244 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 5 to trial17818531-7 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: finishing write tid 6 to trial17818531-7 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: oid 7 updating version 0 to 1647 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: oid 7 version 1647 is already newer than 1645 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: oid 7 version 1647 is already newer than 1644 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: oid 7 version 1647 is already newer than 1646 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: oid 7 version 1647 is already newer than 1643 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1647 (ObjNum 1062 snap 308 seq_num 1062) dirty exists 2026-02-21T22:28:39.245 INFO:tasks.rados.rados.0.trial178.stdout:3027: left oid 7 (ObjNum 1062 snap 308 seq_num 1062) 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3025: done (3 left) 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3026: done (2 left) 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3027: done (1 left) 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3028: done (0 left) 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3029: rollback oid 19 current snap is 309 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 308 2026-02-21T22:28:40.197 INFO:tasks.rados.rados.0.trial178.stdout:3030: delete oid 24 current snap is 309 2026-02-21T22:28:40.202 INFO:tasks.rados.rados.0.trial178.stdout:3029: finishing rollback tid 0 to trial17818531-19 2026-02-21T22:28:40.202 INFO:tasks.rados.rados.0.trial178.stdout:3029: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:28:40.202 INFO:tasks.rados.rados.0.trial178.stdout:3029: finishing rollback tid 2 to trial17818531-19 2026-02-21T22:28:40.203 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1651 (ObjNum 1010 snap 283 seq_num 1010) dirty exists 2026-02-21T22:28:40.203 INFO:tasks.rados.rados.0.trial178.stdout:3029: done (1 left) 2026-02-21T22:28:40.203 INFO:tasks.rados.rados.0.trial178.stdout:3030: done (0 left) 2026-02-21T22:28:40.203 INFO:tasks.rados.rados.0.trial178.stdout:3031: write oid 3 current snap is 309 2026-02-21T22:28:40.203 INFO:tasks.rados.rados.0.trial178.stdout:3031: seq_num 1064 ranges {682648=638003,2029782=432376,3019476=587304} 2026-02-21T22:28:40.211 INFO:tasks.rados.rados.0.trial178.stdout:3031: writing trial17818531-3 from 682648 to 1320651 tid 1 2026-02-21T22:28:40.219 INFO:tasks.rados.rados.0.trial178.stdout:3031: writing trial17818531-3 from 2029782 to 2462158 tid 2 2026-02-21T22:28:40.227 INFO:tasks.rados.rados.0.trial178.stdout:3031: writing trial17818531-3 from 3019476 to 3606780 tid 3 2026-02-21T22:28:40.227 INFO:tasks.rados.rados.0.trial178.stdout:3032: delete oid 44 current snap is 309 2026-02-21T22:28:40.227 INFO:tasks.rados.rados.0.trial178.stdout:3031: finishing write tid 1 to trial17818531-3 2026-02-21T22:28:40.227 INFO:tasks.rados.rados.0.trial178.stdout:3031: finishing write tid 2 to trial17818531-3 2026-02-21T22:28:40.229 INFO:tasks.rados.rados.0.trial178.stdout:3031: finishing write tid 3 to trial17818531-3 2026-02-21T22:28:40.229 INFO:tasks.rados.rados.0.trial178.stdout:3031: finishing write tid 4 to trial17818531-3 2026-02-21T22:28:40.229 INFO:tasks.rados.rados.0.trial178.stdout:3031: finishing write tid 5 to trial17818531-3 2026-02-21T22:28:40.229 INFO:tasks.rados.rados.0.trial178.stdout:3031: oid 3 updating version 0 to 1458 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3031: oid 3 version 1458 is already newer than 1457 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3031: oid 3 updating version 1458 to 1459 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3031: oid 3 updating version 1459 to 1460 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1460 (ObjNum 1064 snap 309 seq_num 1064) dirty exists 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3031: left oid 3 (ObjNum 1064 snap 309 seq_num 1064) 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3031: done (1 left) 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3032: done (0 left) 2026-02-21T22:28:40.230 INFO:tasks.rados.rados.0.trial178.stdout:3033: snap_remove snap 299 2026-02-21T22:28:41.197 INFO:tasks.rados.rados.0.trial178.stdout:3033: done (0 left) 2026-02-21T22:28:41.197 INFO:tasks.rados.rados.0.trial178.stdout:3034: snap_remove snap 300 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3034: done (0 left) 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3035: read oid 11 snap -1 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3035: expect (ObjNum 1015 snap 283 seq_num 1015) 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3036: read oid 40 snap 304 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3036: expect (ObjNum 970 snap 280 seq_num 970) 2026-02-21T22:28:42.749 INFO:tasks.rados.rados.0.trial178.stdout:3037: read oid 49 snap -1 2026-02-21T22:28:42.750 INFO:tasks.rados.rados.0.trial178.stdout:3037: expect deleted 2026-02-21T22:28:42.750 INFO:tasks.rados.rados.0.trial178.stdout:3038: write (excl) oid 8 current snap is 309 2026-02-21T22:28:42.750 INFO:tasks.rados.rados.0.trial178.stdout:3038: seq_num 1066 ranges {716262=504854,1863099=719692,3074322=726201} 2026-02-21T22:28:42.757 INFO:tasks.rados.rados.0.trial178.stdout:3038: writing trial17818531-8 from 716262 to 1221116 tid 1 2026-02-21T22:28:42.766 INFO:tasks.rados.rados.0.trial178.stdout:3038: writing trial17818531-8 from 1863099 to 2582791 tid 2 2026-02-21T22:28:42.775 INFO:tasks.rados.rados.0.trial178.stdout:3038: writing trial17818531-8 from 3074322 to 3800523 tid 3 2026-02-21T22:28:42.775 INFO:tasks.rados.rados.0.trial178.stdout:3039: copy_from oid 17 from oid 33 current snap is 309 2026-02-21T22:28:42.775 INFO:tasks.rados.rados.0.trial178.stdout:3035: expect (ObjNum 1015 snap 283 seq_num 1015) 2026-02-21T22:28:42.775 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:42.833 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:42.902 INFO:tasks.rados.rados.0.trial178.stdout:3036: expect (ObjNum 970 snap 280 seq_num 970) 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: finishing write tid 1 to trial17818531-8 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: finishing write tid 2 to trial17818531-8 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: finishing write tid 3 to trial17818531-8 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: finishing write tid 4 to trial17818531-8 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: finishing write tid 5 to trial17818531-8 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: oid 8 updating version 0 to 1663 2026-02-21T22:28:42.943 INFO:tasks.rados.rados.0.trial178.stdout:3038: oid 8 updating version 1663 to 1665 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3038: oid 8 version 1665 is already newer than 1664 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3038: oid 8 version 1665 is already newer than 1658 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1665 (ObjNum 1066 snap 309 seq_num 1066) dirty exists 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3038: left oid 8 (ObjNum 1066 snap 309 seq_num 1066) 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3039: finishing copy_from racing read to trial17818531-17 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3039: finishing copy_from to trial17818531-17 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1172 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3035: done (4 left) 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3036: done (3 left) 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3037: done (2 left) 2026-02-21T22:28:42.944 INFO:tasks.rados.rados.0.trial178.stdout:3038: done (1 left) 2026-02-21T22:28:42.945 INFO:tasks.rados.rados.0.trial178.stdout:3039: done (0 left) 2026-02-21T22:28:42.945 INFO:tasks.rados.rados.0.trial178.stdout:3040: copy_from oid 5 from oid 50 current snap is 309 2026-02-21T22:28:42.945 INFO:tasks.rados.rados.0.trial178.stdout:3041: write (excl) oid 33 current snap is 309 2026-02-21T22:28:42.945 INFO:tasks.rados.rados.0.trial178.stdout:3041: seq_num 1067 ranges {733069=580103,1921581=579703,3218098=537473} 2026-02-21T22:28:42.951 INFO:tasks.rados.rados.0.trial178.stdout:3041: writing trial17818531-33 from 733069 to 1313172 tid 1 2026-02-21T22:28:42.959 INFO:tasks.rados.rados.0.trial178.stdout:3041: writing trial17818531-33 from 1921581 to 2501284 tid 2 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3041: writing trial17818531-33 from 3218098 to 3755571 tid 3 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3042: read oid 4 snap 308 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3042: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3043: snap_create 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3040: finishing copy_from racing read to trial17818531-5 2026-02-21T22:28:42.969 INFO:tasks.rados.rados.0.trial178.stdout:3040: finishing copy_from to trial17818531-5 2026-02-21T22:28:42.970 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1585 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:42.970 INFO:tasks.rados.rados.0.trial178.stdout:3041: finishing write tid 1 to trial17818531-33 2026-02-21T22:28:42.970 INFO:tasks.rados.rados.0.trial178.stdout:3041: finishing write tid 2 to trial17818531-33 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: finishing write tid 3 to trial17818531-33 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: finishing write tid 4 to trial17818531-33 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: finishing write tid 5 to trial17818531-33 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: oid 33 updating version 0 to 1184 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: oid 33 version 1184 is already newer than 1183 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: oid 33 version 1184 is already newer than 1181 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:3041: oid 33 version 1184 is already newer than 1182 2026-02-21T22:28:42.971 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1184 (ObjNum 1067 snap 309 seq_num 1067) dirty exists 2026-02-21T22:28:42.972 INFO:tasks.rados.rados.0.trial178.stdout:3041: left oid 33 (ObjNum 1067 snap 309 seq_num 1067) 2026-02-21T22:28:42.973 INFO:tasks.rados.rados.0.trial178.stdout:3042: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:28:42.975 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 538346 2026-02-21T22:28:43.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:43.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:43.818 INFO:tasks.rados.rados.0.trial178.stdout:3040: done (3 left) 2026-02-21T22:28:43.818 INFO:tasks.rados.rados.0.trial178.stdout:3041: done (2 left) 2026-02-21T22:28:43.818 INFO:tasks.rados.rados.0.trial178.stdout:3042: done (1 left) 2026-02-21T22:28:43.818 INFO:tasks.rados.rados.0.trial178.stdout:3043: done (0 left) 2026-02-21T22:28:43.819 INFO:tasks.rados.rados.0.trial178.stdout:3044: read oid 8 snap -1 2026-02-21T22:28:43.819 INFO:tasks.rados.rados.0.trial178.stdout:3044: expect (ObjNum 1066 snap 309 seq_num 1066) 2026-02-21T22:28:43.819 INFO:tasks.rados.rados.0.trial178.stdout:3045: write (excl) oid 27 current snap is 310 2026-02-21T22:28:43.819 INFO:tasks.rados.rados.0.trial178.stdout:3045: seq_num 1068 ranges {749876=655352,1863710=723361,3012815=748745} 2026-02-21T22:28:43.828 INFO:tasks.rados.rados.0.trial178.stdout:3045: writing trial17818531-27 from 749876 to 1405228 tid 1 2026-02-21T22:28:43.836 INFO:tasks.rados.rados.0.trial178.stdout:3045: writing trial17818531-27 from 1863710 to 2587071 tid 2 2026-02-21T22:28:43.844 INFO:tasks.rados.rados.0.trial178.stdout:3045: writing trial17818531-27 from 3012815 to 3761560 tid 3 2026-02-21T22:28:43.844 INFO:tasks.rados.rados.0.trial178.stdout:3046: snap_remove snap 306 2026-02-21T22:28:44.820 INFO:tasks.rados.rados.0.trial178.stdout:3046: done (2 left) 2026-02-21T22:28:44.820 INFO:tasks.rados.rados.0.trial178.stdout:3047: rollback oid 39 current snap is 310 2026-02-21T22:28:44.820 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 39 to 308 2026-02-21T22:28:44.820 INFO:tasks.rados.rados.0.trial178.stdout:3044: expect (ObjNum 1066 snap 309 seq_num 1066) 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: finishing write tid 1 to trial17818531-27 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: finishing write tid 2 to trial17818531-27 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: finishing write tid 3 to trial17818531-27 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: finishing write tid 4 to trial17818531-27 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: finishing write tid 5 to trial17818531-27 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: oid 27 updating version 0 to 1238 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: oid 27 version 1238 is already newer than 1236 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: oid 27 updating version 1238 to 1239 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:3045: oid 27 version 1239 is already newer than 1237 2026-02-21T22:28:44.934 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1239 (ObjNum 1068 snap 310 seq_num 1068) dirty exists 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3045: left oid 27 (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3047: finishing rollback tid 0 to trial17818531-39 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3047: finishing rollback tid 1 to trial17818531-39 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3047: finishing rollback tid 2 to trial17818531-39 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1242 (ObjNum 1034 snap 289 seq_num 1034) dirty exists 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3044: done (2 left) 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3045: done (1 left) 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3047: done (0 left) 2026-02-21T22:28:44.935 INFO:tasks.rados.rados.0.trial178.stdout:3048: read oid 12 snap -1 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3048: expect deleted 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3049: read oid 19 snap -1 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3049: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3050: copy_from oid 14 from oid 4 current snap is 310 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3051: rollback oid 48 current snap is 310 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 303 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3052: rollback oid 5 current snap is 310 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 309 2026-02-21T22:28:44.936 INFO:tasks.rados.rados.0.trial178.stdout:3053: snap_create 2026-02-21T22:28:44.937 INFO:tasks.rados.rados.0.trial178.stdout:3052: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:28:44.937 INFO:tasks.rados.rados.0.trial178.stdout:3052: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:28:44.937 INFO:tasks.rados.rados.0.trial178.stdout:3052: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:28:44.938 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1588 (ObjNum 998 snap 283 seq_num 998) dirty exists 2026-02-21T22:28:44.938 INFO:tasks.rados.rados.0.trial178.stdout:3051: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:28:44.938 INFO:tasks.rados.rados.0.trial178.stdout:3051: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:28:44.938 INFO:tasks.rados.rados.0.trial178.stdout:3051: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:28:44.938 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1333 (ObjNum 995 snap 281 seq_num 995) dirty exists 2026-02-21T22:28:44.941 INFO:tasks.rados.rados.0.trial178.stdout:3049: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:28:45.045 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 870561 2026-02-21T22:28:45.080 INFO:tasks.rados.rados.0.trial178.stdout:3050: finishing copy_from racing read to trial17818531-14 2026-02-21T22:28:45.080 INFO:tasks.rados.rados.0.trial178.stdout:3050: finishing copy_from to trial17818531-14 2026-02-21T22:28:45.080 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1457 (ObjNum 981 snap 281 seq_num 981) dirty exists 2026-02-21T22:28:46.542 INFO:tasks.rados.rados.0.trial178.stdout:3048: done (5 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3049: done (4 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3050: done (3 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3051: done (2 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3052: done (1 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3053: done (0 left) 2026-02-21T22:28:46.543 INFO:tasks.rados.rados.0.trial178.stdout:3054: snap_remove snap 307 2026-02-21T22:28:46.828 INFO:tasks.rados.rados.0.trial178.stdout:3054: done (0 left) 2026-02-21T22:28:46.829 INFO:tasks.rados.rados.0.trial178.stdout:3055: delete oid 46 current snap is 311 2026-02-21T22:28:46.830 INFO:tasks.rados.rados.0.trial178.stdout:3055: done (0 left) 2026-02-21T22:28:46.830 INFO:tasks.rados.rados.0.trial178.stdout:3056: delete oid 45 current snap is 311 2026-02-21T22:28:46.833 INFO:tasks.rados.rados.0.trial178.stdout:3056: done (0 left) 2026-02-21T22:28:46.833 INFO:tasks.rados.rados.0.trial178.stdout:3057: rollback oid 1 current snap is 311 2026-02-21T22:28:46.833 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 1 to 308 2026-02-21T22:28:46.833 INFO:tasks.rados.rados.0.trial178.stdout:3058: snap_remove snap 310 2026-02-21T22:28:47.188 INFO:tasks.rados.rados.0.trial178.stdout:3058: done (1 left) 2026-02-21T22:28:47.188 INFO:tasks.rados.rados.0.trial178.stdout:3059: delete oid 22 current snap is 311 2026-02-21T22:28:47.188 INFO:tasks.rados.rados.0.trial178.stdout:3057: finishing rollback tid 1 to trial17818531-1 2026-02-21T22:28:47.188 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3057: done (1 left) 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3059: done (0 left) 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3060: read oid 34 snap -1 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3060: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3061: read oid 3 snap -1 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3061: expect (ObjNum 1064 snap 309 seq_num 1064) 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3062: write oid 1 current snap is 311 2026-02-21T22:28:47.192 INFO:tasks.rados.rados.0.trial178.stdout:3062: seq_num 1072 ranges {417104=672701,1813991=203113} 2026-02-21T22:28:47.196 INFO:tasks.rados.rados.0.trial178.stdout:3062: writing trial17818531-1 from 417104 to 1089805 tid 1 2026-02-21T22:28:47.199 INFO:tasks.rados.rados.0.trial178.stdout:3062: writing trial17818531-1 from 1813991 to 2017104 tid 2 2026-02-21T22:28:47.199 INFO:tasks.rados.rados.0.trial178.stdout:3063: delete oid 45 current snap is 311 2026-02-21T22:28:47.200 INFO:tasks.rados.rados.0.trial178.stdout:3060: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:28:47.247 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:47.301 INFO:tasks.rados.rados.0.trial178.stdout:3061: expect (ObjNum 1064 snap 309 seq_num 1064) 2026-02-21T22:28:47.308 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2113357 2026-02-21T22:28:47.360 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3864330 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: finishing write tid 1 to trial17818531-1 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: finishing write tid 2 to trial17818531-1 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: finishing write tid 3 to trial17818531-1 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: finishing write tid 4 to trial17818531-1 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: oid 1 updating version 0 to 1337 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: oid 1 version 1337 is already newer than 1336 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:3062: oid 1 version 1337 is already newer than 1335 2026-02-21T22:28:47.363 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1337 (ObjNum 1072 snap 311 seq_num 1072) dirty exists 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3062: left oid 1 (ObjNum 1072 snap 311 seq_num 1072) 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3060: done (3 left) 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3061: done (2 left) 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3062: done (1 left) 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3063: done (0 left) 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3064: write (excl) oid 20 current snap is 311 2026-02-21T22:28:47.364 INFO:tasks.rados.rados.0.trial178.stdout:3064: seq_num 1074 ranges {450718=423199,1414602=636116} 2026-02-21T22:28:47.369 INFO:tasks.rados.rados.0.trial178.stdout:3064: writing trial17818531-20 from 450718 to 873917 tid 1 2026-02-21T22:28:47.378 INFO:tasks.rados.rados.0.trial178.stdout:3064: writing trial17818531-20 from 1414602 to 2050718 tid 2 2026-02-21T22:28:47.378 INFO:tasks.rados.rados.0.trial178.stdout:3065: copy_from oid 40 from oid 27 current snap is 311 2026-02-21T22:28:47.378 INFO:tasks.rados.rados.0.trial178.stdout:3064: finishing write tid 1 to trial17818531-20 2026-02-21T22:28:47.378 INFO:tasks.rados.rados.0.trial178.stdout:3066: write (excl) oid 13 current snap is 311 2026-02-21T22:28:47.378 INFO:tasks.rados.rados.0.trial178.stdout:3066: seq_num 1075 ranges {467525=498448,1473084=594379} 2026-02-21T22:28:47.384 INFO:tasks.rados.rados.0.trial178.stdout:3066: writing trial17818531-13 from 467525 to 965973 tid 1 2026-02-21T22:28:47.393 INFO:tasks.rados.rados.0.trial178.stdout:3066: writing trial17818531-13 from 1473084 to 2067463 tid 2 2026-02-21T22:28:47.393 INFO:tasks.rados.rados.0.trial178.stdout:3067: copy_from oid 24 from oid 10 current snap is 311 2026-02-21T22:28:47.393 INFO:tasks.rados.rados.0.trial178.stdout:3064: finishing write tid 2 to trial17818531-20 2026-02-21T22:28:47.393 INFO:tasks.rados.rados.0.trial178.stdout:3064: finishing write tid 3 to trial17818531-20 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: finishing write tid 4 to trial17818531-20 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: oid 20 updating version 0 to 1230 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: oid 20 version 1230 is already newer than 1229 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: oid 20 updating version 1230 to 1231 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1231 (ObjNum 1074 snap 311 seq_num 1074) dirty exists 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: left oid 20 (ObjNum 1074 snap 311 seq_num 1074) 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3066: finishing write tid 1 to trial17818531-13 2026-02-21T22:28:47.394 INFO:tasks.rados.rados.0.trial178.stdout:3064: done (3 left) 2026-02-21T22:28:47.395 INFO:tasks.rados.rados.0.trial178.stdout:3068: write (excl) oid 19 current snap is 311 2026-02-21T22:28:47.395 INFO:tasks.rados.rados.0.trial178.stdout:3068: seq_num 1076 ranges {484332=573697,1531566=552766} 2026-02-21T22:28:47.400 INFO:tasks.rados.rados.0.trial178.stdout:3068: writing trial17818531-19 from 484332 to 1058029 tid 1 2026-02-21T22:28:47.407 INFO:tasks.rados.rados.0.trial178.stdout:3068: writing trial17818531-19 from 1531566 to 2084332 tid 2 2026-02-21T22:28:47.407 INFO:tasks.rados.rados.0.trial178.stdout:3069: snap_remove snap 302 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3069: done (4 left) 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3070: copy_from oid 47 from oid 9 current snap is 311 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3067: finishing copy_from to trial17818531-24 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3067: got expected ENOENT (src dne) 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3067: finishing copy_from racing read to trial17818531-24 2026-02-21T22:28:48.193 INFO:tasks.rados.rados.0.trial178.stdout:3066: finishing write tid 2 to trial17818531-13 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3065: finishing copy_from racing read to trial17818531-40 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: finishing write tid 3 to trial17818531-13 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: finishing write tid 4 to trial17818531-13 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: oid 13 updating version 0 to 1462 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: oid 13 version 1462 is already newer than 1461 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: oid 13 updating version 1462 to 1463 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1463 (ObjNum 1075 snap 311 seq_num 1075) dirty exists 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: left oid 13 (ObjNum 1075 snap 311 seq_num 1075) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3065: finishing copy_from to trial17818531-40 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1239 (ObjNum 1068 snap 310 seq_num 1068) dirty exists 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: finishing write tid 1 to trial17818531-19 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3065: done (4 left) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3066: done (3 left) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3067: done (2 left) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3071: rollback oid 23 current snap is 311 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 308 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: finishing write tid 2 to trial17818531-19 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: finishing write tid 3 to trial17818531-19 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: finishing write tid 4 to trial17818531-19 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: oid 19 updating version 0 to 1680 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: oid 19 version 1680 is already newer than 1679 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: oid 19 updating version 1680 to 1681 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1681 (ObjNum 1076 snap 311 seq_num 1076) dirty exists 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: left oid 19 (ObjNum 1076 snap 311 seq_num 1076) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3068: done (2 left) 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3072: rollback oid 42 current snap is 311 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 304 2026-02-21T22:28:48.194 INFO:tasks.rados.rados.0.trial178.stdout:3073: snap_create 2026-02-21T22:28:48.195 INFO:tasks.rados.rados.0.trial178.stdout:3072: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:28:48.195 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 0 (ObjNum 33 snap 0 seq_num 101485296) dirty dne 2026-02-21T22:28:48.197 INFO:tasks.rados.rados.0.trial178.stdout:3070: finishing copy_from to trial17818531-47 2026-02-21T22:28:48.197 INFO:tasks.rados.rados.0.trial178.stdout:3070: got expected ENOENT (src dne) 2026-02-21T22:28:48.197 INFO:tasks.rados.rados.0.trial178.stdout:3070: finishing copy_from racing read to trial17818531-47 2026-02-21T22:28:48.198 INFO:tasks.rados.rados.0.trial178.stdout:3071: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:28:48.198 INFO:tasks.rados.rados.0.trial178.stdout:3071: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:28:48.199 INFO:tasks.rados.rados.0.trial178.stdout:3071: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:28:48.199 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1385 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:28:48.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:48.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:49.194 INFO:tasks.rados.rados.0.trial178.stdout:3070: done (3 left) 2026-02-21T22:28:49.194 INFO:tasks.rados.rados.0.trial178.stdout:3071: done (2 left) 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3072: done (1 left) 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3073: done (0 left) 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3074: read oid 30 snap -1 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3074: expect (ObjNum 1019 snap 283 seq_num 1019) 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3075: read oid 8 snap -1 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3075: expect (ObjNum 1066 snap 309 seq_num 1066) 2026-02-21T22:28:49.195 INFO:tasks.rados.rados.0.trial178.stdout:3076: snap_remove snap 311 2026-02-21T22:28:50.200 INFO:tasks.rados.rados.0.trial178.stdout:3076: done (2 left) 2026-02-21T22:28:50.200 INFO:tasks.rados.rados.0.trial178.stdout:3077: read oid 20 snap -1 2026-02-21T22:28:50.200 INFO:tasks.rados.rados.0.trial178.stdout:3077: expect (ObjNum 1074 snap 311 seq_num 1074) 2026-02-21T22:28:50.200 INFO:tasks.rados.rados.0.trial178.stdout:3078: snap_remove snap 297 2026-02-21T22:28:51.202 INFO:tasks.rados.rados.0.trial178.stdout:3078: done (3 left) 2026-02-21T22:28:51.202 INFO:tasks.rados.rados.0.trial178.stdout:3079: delete oid 28 current snap is 312 2026-02-21T22:28:51.202 INFO:tasks.rados.rados.0.trial178.stdout:3074: expect (ObjNum 1019 snap 283 seq_num 1019) 2026-02-21T22:28:51.202 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 470188 2026-02-21T22:28:51.274 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:51.305 INFO:tasks.rados.rados.0.trial178.stdout:3075: expect (ObjNum 1066 snap 309 seq_num 1066) 2026-02-21T22:28:51.306 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:51.348 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:51.432 INFO:tasks.rados.rados.0.trial178.stdout:3077: expect (ObjNum 1074 snap 311 seq_num 1074) 2026-02-21T22:28:51.455 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1270159 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3074: done (3 left) 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3075: done (2 left) 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3077: done (1 left) 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3079: done (0 left) 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3080: read oid 21 snap -1 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3080: expect (ObjNum 917 snap 270 seq_num 917) 2026-02-21T22:28:51.482 INFO:tasks.rados.rados.0.trial178.stdout:3081: snap_create 2026-02-21T22:28:51.491 INFO:tasks.rados.rados.0.trial178.stdout:3080: expect (ObjNum 917 snap 270 seq_num 917) 2026-02-21T22:28:52.190 INFO:tasks.rados.rados.0.trial178.stdout:3080: done (1 left) 2026-02-21T22:28:52.190 INFO:tasks.rados.rados.0.trial178.stdout:3081: done (0 left) 2026-02-21T22:28:52.190 INFO:tasks.rados.rados.0.trial178.stdout:3082: write oid 1 current snap is 313 2026-02-21T22:28:52.190 INFO:tasks.rados.rados.0.trial178.stdout:3082: seq_num 1078 ranges {517946=724195,1932177=185769} 2026-02-21T22:28:52.198 INFO:tasks.rados.rados.0.trial178.stdout:3082: writing trial17818531-1 from 517946 to 1242141 tid 1 2026-02-21T22:28:52.204 INFO:tasks.rados.rados.0.trial178.stdout:3082: writing trial17818531-1 from 1932177 to 2117946 tid 2 2026-02-21T22:28:52.204 INFO:tasks.rados.rados.0.trial178.stdout:3083: delete oid 37 current snap is 313 2026-02-21T22:28:52.204 INFO:tasks.rados.rados.0.trial178.stdout:3082: finishing write tid 1 to trial17818531-1 2026-02-21T22:28:52.206 INFO:tasks.rados.rados.0.trial178.stdout:3082: finishing write tid 2 to trial17818531-1 2026-02-21T22:28:52.206 INFO:tasks.rados.rados.0.trial178.stdout:3082: finishing write tid 3 to trial17818531-1 2026-02-21T22:28:52.206 INFO:tasks.rados.rados.0.trial178.stdout:3082: finishing write tid 4 to trial17818531-1 2026-02-21T22:28:52.206 INFO:tasks.rados.rados.0.trial178.stdout:3082: oid 1 updating version 0 to 1339 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3082: oid 1 version 1339 is already newer than 1338 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3082: oid 1 updating version 1339 to 1340 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1340 (ObjNum 1078 snap 313 seq_num 1078) dirty exists 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3082: left oid 1 (ObjNum 1078 snap 313 seq_num 1078) 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3082: done (1 left) 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3083: done (0 left) 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3084: write (excl) oid 49 current snap is 313 2026-02-21T22:28:52.207 INFO:tasks.rados.rados.0.trial178.stdout:3084: seq_num 1080 ranges {551560=591046,1765494=386066} 2026-02-21T22:28:52.214 INFO:tasks.rados.rados.0.trial178.stdout:3084: writing trial17818531-49 from 551560 to 1142606 tid 1 2026-02-21T22:28:52.221 INFO:tasks.rados.rados.0.trial178.stdout:3084: writing trial17818531-49 from 1765494 to 2151560 tid 2 2026-02-21T22:28:52.221 INFO:tasks.rados.rados.0.trial178.stdout:3085: copy_from oid 19 from oid 38 current snap is 313 2026-02-21T22:28:52.222 INFO:tasks.rados.rados.0.trial178.stdout:3084: finishing write tid 1 to trial17818531-49 2026-02-21T22:28:52.222 INFO:tasks.rados.rados.0.trial178.stdout:3086: write (excl) oid 6 current snap is 313 2026-02-21T22:28:52.222 INFO:tasks.rados.rados.0.trial178.stdout:3086: seq_num 1081 ranges {568367=666295,1707623=460744} 2026-02-21T22:28:52.229 INFO:tasks.rados.rados.0.trial178.stdout:3086: writing trial17818531-6 from 568367 to 1234662 tid 1 2026-02-21T22:28:52.236 INFO:tasks.rados.rados.0.trial178.stdout:3086: writing trial17818531-6 from 1707623 to 2168367 tid 2 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3087: rollback oid 23 current snap is 313 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 303 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3085: finishing copy_from to trial17818531-19 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3085: got expected ENOENT (src dne) 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3085: finishing copy_from racing read to trial17818531-19 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3084: finishing write tid 2 to trial17818531-49 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3084: finishing write tid 3 to trial17818531-49 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3084: finishing write tid 4 to trial17818531-49 2026-02-21T22:28:52.237 INFO:tasks.rados.rados.0.trial178.stdout:3084: oid 49 updating version 0 to 1452 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3084: oid 49 version 1452 is already newer than 1451 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3084: oid 49 version 1452 is already newer than 1450 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1452 (ObjNum 1080 snap 313 seq_num 1080) dirty exists 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3084: left oid 49 (ObjNum 1080 snap 313 seq_num 1080) 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3086: finishing write tid 1 to trial17818531-6 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3084: done (3 left) 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3085: done (2 left) 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3088: read oid 44 snap -1 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3088: expect deleted 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3089: write oid 12 current snap is 313 2026-02-21T22:28:52.238 INFO:tasks.rados.rados.0.trial178.stdout:3089: seq_num 1082 ranges {585174=741544,1766105=419069} 2026-02-21T22:28:52.245 INFO:tasks.rados.rados.0.trial178.stdout:3089: writing trial17818531-12 from 585174 to 1326718 tid 1 2026-02-21T22:28:52.251 INFO:tasks.rados.rados.0.trial178.stdout:3089: writing trial17818531-12 from 1766105 to 2185174 tid 2 2026-02-21T22:28:52.251 INFO:tasks.rados.rados.0.trial178.stdout:3090: write (excl) oid 36 current snap is 313 2026-02-21T22:28:52.251 INFO:tasks.rados.rados.0.trial178.stdout:3090: seq_num 1083 ranges {601981=416793,1424587=492702} 2026-02-21T22:28:52.254 INFO:tasks.rados.rados.0.trial178.stdout:3090: writing trial17818531-36 from 601981 to 1018774 tid 1 2026-02-21T22:28:52.258 INFO:tasks.rados.rados.0.trial178.stdout:3090: writing trial17818531-36 from 1424587 to 1917289 tid 2 2026-02-21T22:28:52.258 INFO:tasks.rados.rados.0.trial178.stdout:3091: snap_remove snap 312 2026-02-21T22:28:53.213 INFO:tasks.rados.rados.0.trial178.stdout:3091: done (5 left) 2026-02-21T22:28:53.213 INFO:tasks.rados.rados.0.trial178.stdout:3092: write oid 31 current snap is 313 2026-02-21T22:28:53.213 INFO:tasks.rados.rados.0.trial178.stdout:3092: seq_num 1084 ranges {618788=492042,1883069=335719} 2026-02-21T22:28:53.218 INFO:tasks.rados.rados.0.trial178.stdout:3092: writing trial17818531-31 from 618788 to 1110830 tid 1 2026-02-21T22:28:53.222 INFO:tasks.rados.rados.0.trial178.stdout:3092: writing trial17818531-31 from 1883069 to 2218788 tid 2 2026-02-21T22:28:53.222 INFO:tasks.rados.rados.0.trial178.stdout:3093: write oid 33 current snap is 313 2026-02-21T22:28:53.222 INFO:tasks.rados.rados.0.trial178.stdout:3093: seq_num 1085 ranges {635595=567291,1825198=410397} 2026-02-21T22:28:53.226 INFO:tasks.rados.rados.0.trial178.stdout:3093: writing trial17818531-33 from 635595 to 1202886 tid 1 2026-02-21T22:28:53.231 INFO:tasks.rados.rados.0.trial178.stdout:3093: writing trial17818531-33 from 1825198 to 2235595 tid 2 2026-02-21T22:28:53.231 INFO:tasks.rados.rados.0.trial178.stdout:3094: delete oid 15 current snap is 313 2026-02-21T22:28:53.231 INFO:tasks.rados.rados.0.trial178.stdout:3086: finishing write tid 2 to trial17818531-6 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: finishing write tid 3 to trial17818531-6 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: finishing write tid 4 to trial17818531-6 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: oid 6 updating version 0 to 1196 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: oid 6 version 1196 is already newer than 1195 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: oid 6 version 1196 is already newer than 1194 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1196 (ObjNum 1081 snap 313 seq_num 1081) dirty exists 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3086: left oid 6 (ObjNum 1081 snap 313 seq_num 1081) 2026-02-21T22:28:53.232 INFO:tasks.rados.rados.0.trial178.stdout:3087: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3087: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3087: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1388 (ObjNum 691 snap 190 seq_num 691) dirty exists 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3089: finishing write tid 1 to trial17818531-12 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3089: finishing write tid 2 to trial17818531-12 2026-02-21T22:28:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3089: finishing write tid 3 to trial17818531-12 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3089: finishing write tid 4 to trial17818531-12 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3089: oid 12 updating version 0 to 1460 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3089: oid 12 updating version 1460 to 1461 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3089: oid 12 version 1461 is already newer than 1459 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1461 (ObjNum 1082 snap 313 seq_num 1082) dirty exists 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3089: left oid 12 (ObjNum 1082 snap 313 seq_num 1082) 2026-02-21T22:28:53.234 INFO:tasks.rados.rados.0.trial178.stdout:3090: finishing write tid 1 to trial17818531-36 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: finishing write tid 2 to trial17818531-36 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: finishing write tid 3 to trial17818531-36 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: finishing write tid 4 to trial17818531-36 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: oid 36 updating version 0 to 1102 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: oid 36 version 1102 is already newer than 1101 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: oid 36 updating version 1102 to 1103 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1103 (ObjNum 1083 snap 313 seq_num 1083) dirty exists 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3090: left oid 36 (ObjNum 1083 snap 313 seq_num 1083) 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3092: finishing write tid 1 to trial17818531-31 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3092: finishing write tid 2 to trial17818531-31 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3092: finishing write tid 3 to trial17818531-31 2026-02-21T22:28:53.235 INFO:tasks.rados.rados.0.trial178.stdout:3092: finishing write tid 4 to trial17818531-31 2026-02-21T22:28:53.236 INFO:tasks.rados.rados.0.trial178.stdout:3092: oid 31 updating version 0 to 1453 2026-02-21T22:28:53.236 INFO:tasks.rados.rados.0.trial178.stdout:3092: oid 31 updating version 1453 to 1454 2026-02-21T22:28:53.236 INFO:tasks.rados.rados.0.trial178.stdout:3092: oid 31 updating version 1454 to 1455 2026-02-21T22:28:53.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1455 (ObjNum 1084 snap 313 seq_num 1084) dirty exists 2026-02-21T22:28:53.236 INFO:tasks.rados.rados.0.trial178.stdout:3092: left oid 31 (ObjNum 1084 snap 313 seq_num 1084) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3086: done (7 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3087: done (6 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3088: done (5 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3089: done (4 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3090: done (3 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3092: done (2 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3094: done (1 left) 2026-02-21T22:28:53.237 INFO:tasks.rados.rados.0.trial178.stdout:3095: copy_from oid 25 from oid 41 current snap is 313 2026-02-21T22:28:53.238 INFO:tasks.rados.rados.0.trial178.stdout:3096: read oid 16 snap -1 2026-02-21T22:28:53.238 INFO:tasks.rados.rados.0.trial178.stdout:3096: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:53.238 INFO:tasks.rados.rados.0.trial178.stdout:3097: write oid 8 current snap is 313 2026-02-21T22:28:53.238 INFO:tasks.rados.rados.0.trial178.stdout:3097: seq_num 1087 ranges {669209=717789,1942162=327047} 2026-02-21T22:28:53.239 INFO:tasks.rados.rados.0.trial178.stdout:3097: writing trial17818531-8 from 669209 to 1386998 tid 1 2026-02-21T22:28:53.242 INFO:tasks.rados.rados.0.trial178.stdout:3097: writing trial17818531-8 from 1942162 to 2269209 tid 2 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3098: read oid 42 snap -1 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3098: expect deleted 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3099: read oid 10 snap -1 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3099: expect deleted 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3100: delete oid 2 current snap is 313 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3096: expect (ObjNum 890 snap 261 seq_num 890) 2026-02-21T22:28:53.243 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:53.296 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 625195 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: finishing write tid 1 to trial17818531-33 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: finishing write tid 2 to trial17818531-33 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: finishing write tid 3 to trial17818531-33 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: finishing write tid 4 to trial17818531-33 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: oid 33 updating version 0 to 1187 2026-02-21T22:28:53.356 INFO:tasks.rados.rados.0.trial178.stdout:3093: oid 33 version 1187 is already newer than 1186 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3093: oid 33 updating version 1187 to 1188 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1188 (ObjNum 1085 snap 313 seq_num 1085) dirty exists 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3093: left oid 33 (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: finishing write tid 1 to trial17818531-8 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: finishing write tid 2 to trial17818531-8 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3093: done (6 left) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3096: done (5 left) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3099: done (4 left) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3100: done (3 left) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3101: delete oid 29 current snap is 313 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: finishing write tid 3 to trial17818531-8 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: finishing write tid 4 to trial17818531-8 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: oid 8 updating version 0 to 1695 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: oid 8 updating version 1695 to 1696 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: oid 8 updating version 1696 to 1697 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1697 (ObjNum 1087 snap 313 seq_num 1087) dirty exists 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3097: left oid 8 (ObjNum 1087 snap 313 seq_num 1087) 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3095: finishing copy_from racing read to trial17818531-25 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:3095: finishing copy_from to trial17818531-25 2026-02-21T22:28:53.357 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1224 (ObjNum 1055 snap 305 seq_num 1055) dirty exists 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3095: done (3 left) 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3097: done (2 left) 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3098: done (1 left) 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3101: done (0 left) 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3102: write oid 19 current snap is 313 2026-02-21T22:28:53.358 INFO:tasks.rados.rados.0.trial178.stdout:3102: seq_num 1090 ranges {719630=659889,2117608=202022} 2026-02-21T22:28:53.368 INFO:tasks.rados.rados.0.trial178.stdout:3102: writing trial17818531-19 from 719630 to 1379519 tid 1 2026-02-21T22:28:53.374 INFO:tasks.rados.rados.0.trial178.stdout:3102: writing trial17818531-19 from 2117608 to 2319630 tid 2 2026-02-21T22:28:53.374 INFO:tasks.rados.rados.0.trial178.stdout:3103: snap_remove snap 304 2026-02-21T22:28:53.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:53.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:53.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:53.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:28:53.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:53.551 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:28:53.581 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:53.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3103: done (1 left) 2026-02-21T22:28:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3104: read oid 2 snap -1 2026-02-21T22:28:54.219 INFO:tasks.rados.rados.0.trial178.stdout:3104: expect deleted 2026-02-21T22:28:54.219 INFO:tasks.rados.rados.0.trial178.stdout:3105: write oid 25 current snap is 313 2026-02-21T22:28:54.219 INFO:tasks.rados.rados.0.trial178.stdout:3105: seq_num 1091 ranges {736437=735138,2176090=160347} 2026-02-21T22:28:54.227 INFO:tasks.rados.rados.0.trial178.stdout:3105: writing trial17818531-25 from 736437 to 1471575 tid 1 2026-02-21T22:28:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3105: writing trial17818531-25 from 2176090 to 2336437 tid 2 2026-02-21T22:28:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3106: write (excl) oid 3 current snap is 313 2026-02-21T22:28:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3106: seq_num 1092 ranges {753244=410387,1834572=518672} 2026-02-21T22:28:54.241 INFO:tasks.rados.rados.0.trial178.stdout:3106: writing trial17818531-3 from 753244 to 1163631 tid 1 2026-02-21T22:28:54.249 INFO:tasks.rados.rados.0.trial178.stdout:3106: writing trial17818531-3 from 1834572 to 2353244 tid 2 2026-02-21T22:28:54.249 INFO:tasks.rados.rados.0.trial178.stdout:3107: copy_from oid 16 from oid 34 current snap is 313 2026-02-21T22:28:54.249 INFO:tasks.rados.rados.0.trial178.stdout:3102: finishing write tid 1 to trial17818531-19 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: finishing write tid 2 to trial17818531-19 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: finishing write tid 3 to trial17818531-19 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: finishing write tid 4 to trial17818531-19 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: oid 19 updating version 0 to 1699 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: oid 19 updating version 1699 to 1700 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: oid 19 version 1700 is already newer than 1698 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1700 (ObjNum 1090 snap 313 seq_num 1090) dirty exists 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: left oid 19 (ObjNum 1090 snap 313 seq_num 1090) 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3105: finishing write tid 1 to trial17818531-25 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3102: done (4 left) 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3104: done (3 left) 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3108: write (excl) oid 20 current snap is 313 2026-02-21T22:28:54.250 INFO:tasks.rados.rados.0.trial178.stdout:3108: seq_num 1093 ranges {770051=485636,1776701=511047} 2026-02-21T22:28:54.258 INFO:tasks.rados.rados.0.trial178.stdout:3108: writing trial17818531-20 from 770051 to 1255687 tid 1 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3108: writing trial17818531-20 from 1776701 to 2287748 tid 2 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3109: read oid 35 snap 309 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3109: expect (ObjNum 1022 snap 283 seq_num 1022) 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3110: rollback oid 13 current snap is 313 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 305 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: finishing write tid 2 to trial17818531-25 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: finishing write tid 3 to trial17818531-25 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: finishing write tid 4 to trial17818531-25 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: oid 25 updating version 0 to 1457 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: oid 25 version 1457 is already newer than 1456 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: oid 25 updating version 1457 to 1458 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1458 (ObjNum 1091 snap 313 seq_num 1091) dirty exists 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: left oid 25 (ObjNum 1091 snap 313 seq_num 1091) 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: finishing write tid 1 to trial17818531-3 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: finishing write tid 2 to trial17818531-3 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: finishing write tid 3 to trial17818531-3 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: finishing write tid 4 to trial17818531-3 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: oid 3 updating version 0 to 1466 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: oid 3 version 1466 is already newer than 1464 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: oid 3 version 1466 is already newer than 1465 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1466 (ObjNum 1092 snap 313 seq_num 1092) dirty exists 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: left oid 3 (ObjNum 1092 snap 313 seq_num 1092) 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3108: finishing write tid 1 to trial17818531-20 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3107: finishing copy_from racing read to trial17818531-16 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3107: finishing copy_from to trial17818531-16 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1193 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3105: done (5 left) 2026-02-21T22:28:54.265 INFO:tasks.rados.rados.0.trial178.stdout:3106: done (4 left) 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3107: done (3 left) 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3111: read oid 8 snap -1 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3111: expect (ObjNum 1087 snap 313 seq_num 1087) 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3112: read oid 19 snap -1 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3112: expect (ObjNum 1090 snap 313 seq_num 1090) 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3113: write (excl) oid 26 current snap is 313 2026-02-21T22:28:54.266 INFO:tasks.rados.rados.0.trial178.stdout:3113: seq_num 1094 ranges {786858=560885,1835183=551675} 2026-02-21T22:28:54.274 INFO:tasks.rados.rados.0.trial178.stdout:3113: writing trial17818531-26 from 786858 to 1347743 tid 1 2026-02-21T22:28:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3113: writing trial17818531-26 from 1835183 to 2386858 tid 2 2026-02-21T22:28:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3114: write oid 42 current snap is 313 2026-02-21T22:28:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3114: seq_num 1095 ranges {403665=752487,1610018=514716} 2026-02-21T22:28:54.284 INFO:tasks.rados.rados.0.trial178.stdout:3114: writing trial17818531-42 from 403665 to 1156152 tid 1 2026-02-21T22:28:54.288 INFO:tasks.rados.rados.0.trial178.stdout:3114: writing trial17818531-42 from 1610018 to 2124734 tid 2 2026-02-21T22:28:54.288 INFO:tasks.rados.rados.0.trial178.stdout:3115: read oid 17 snap -1 2026-02-21T22:28:54.289 INFO:tasks.rados.rados.0.trial178.stdout:3115: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:54.289 INFO:tasks.rados.rados.0.trial178.stdout:3116: write (excl) oid 14 current snap is 313 2026-02-21T22:28:54.289 INFO:tasks.rados.rados.0.trial178.stdout:3116: seq_num 1096 ranges {420472=427736,1268500=658374,2347953=72519} 2026-02-21T22:28:54.291 INFO:tasks.rados.rados.0.trial178.stdout:3116: writing trial17818531-14 from 420472 to 848208 tid 1 2026-02-21T22:28:54.295 INFO:tasks.rados.rados.0.trial178.stdout:3116: writing trial17818531-14 from 1268500 to 1926874 tid 2 2026-02-21T22:28:54.297 INFO:tasks.rados.rados.0.trial178.stdout:3116: writing trial17818531-14 from 2347953 to 2420472 tid 3 2026-02-21T22:28:54.297 INFO:tasks.rados.rados.0.trial178.stdout:3117: write oid 36 current snap is 313 2026-02-21T22:28:54.298 INFO:tasks.rados.rados.0.trial178.stdout:3117: seq_num 1097 ranges {437279=502985,1610629=518385} 2026-02-21T22:28:54.301 INFO:tasks.rados.rados.0.trial178.stdout:3117: writing trial17818531-36 from 437279 to 940264 tid 1 2026-02-21T22:28:54.306 INFO:tasks.rados.rados.0.trial178.stdout:3117: writing trial17818531-36 from 1610629 to 2129014 tid 2 2026-02-21T22:28:54.306 INFO:tasks.rados.rados.0.trial178.stdout:3118: snap_remove snap 308 2026-02-21T22:28:55.219 INFO:tasks.rados.rados.0.trial178.stdout:3118: done (10 left) 2026-02-21T22:28:55.219 INFO:tasks.rados.rados.0.trial178.stdout:3119: write oid 40 current snap is 313 2026-02-21T22:28:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3119: seq_num 1098 ranges {454086=578234,1669111=662043} 2026-02-21T22:28:55.224 INFO:tasks.rados.rados.0.trial178.stdout:3119: writing trial17818531-40 from 454086 to 1032320 tid 1 2026-02-21T22:28:55.230 INFO:tasks.rados.rados.0.trial178.stdout:3119: writing trial17818531-40 from 1669111 to 2331154 tid 2 2026-02-21T22:28:55.230 INFO:tasks.rados.rados.0.trial178.stdout:3120: write (excl) oid 50 current snap is 313 2026-02-21T22:28:55.230 INFO:tasks.rados.rados.0.trial178.stdout:3120: seq_num 1099 ranges {470893=653483,1727593=522054} 2026-02-21T22:28:55.234 INFO:tasks.rados.rados.0.trial178.stdout:3120: writing trial17818531-50 from 470893 to 1124376 tid 1 2026-02-21T22:28:55.238 INFO:tasks.rados.rados.0.trial178.stdout:3120: writing trial17818531-50 from 1727593 to 2249647 tid 2 2026-02-21T22:28:55.238 INFO:tasks.rados.rados.0.trial178.stdout:3121: copy_from oid 11 from oid 30 current snap is 313 2026-02-21T22:28:55.239 INFO:tasks.rados.rados.0.trial178.stdout:3111: expect (ObjNum 1087 snap 313 seq_num 1087) 2026-02-21T22:28:55.239 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 68850 2026-02-21T22:28:55.287 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 120401 2026-02-21T22:28:55.347 INFO:tasks.rados.rados.0.trial178.stdout:3108: finishing write tid 2 to trial17818531-20 2026-02-21T22:28:55.347 INFO:tasks.rados.rados.0.trial178.stdout:3109: expect (ObjNum 1022 snap 283 seq_num 1022) 2026-02-21T22:28:55.347 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 406923 2026-02-21T22:28:55.378 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: finishing write tid 3 to trial17818531-20 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: finishing write tid 4 to trial17818531-20 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: oid 20 updating version 0 to 1235 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: oid 20 version 1235 is already newer than 1233 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: oid 20 version 1235 is already newer than 1234 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1235 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3108: left oid 20 (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3110: finishing rollback tid 0 to trial17818531-13 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3110: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3110: finishing rollback tid 2 to trial17818531-13 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1469 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:28:55.438 INFO:tasks.rados.rados.0.trial178.stdout:3112: expect (ObjNum 1090 snap 313 seq_num 1090) 2026-02-21T22:28:55.492 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: finishing write tid 1 to trial17818531-26 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: finishing write tid 2 to trial17818531-26 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: finishing write tid 3 to trial17818531-26 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3108: done (12 left) 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3109: done (11 left) 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3110: done (10 left) 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3111: done (9 left) 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3112: done (8 left) 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3122: copy_from oid 15 from oid 9 current snap is 313 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: finishing write tid 4 to trial17818531-26 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: oid 26 updating version 0 to 1198 2026-02-21T22:28:55.520 INFO:tasks.rados.rados.0.trial178.stdout:3113: oid 26 updating version 1198 to 1199 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3113: oid 26 updating version 1199 to 1200 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1200 (ObjNum 1094 snap 313 seq_num 1094) dirty exists 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3113: left oid 26 (ObjNum 1094 snap 313 seq_num 1094) 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: finishing write tid 1 to trial17818531-42 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: finishing write tid 2 to trial17818531-42 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: finishing write tid 3 to trial17818531-42 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: finishing write tid 4 to trial17818531-42 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: oid 42 updating version 0 to 1708 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: oid 42 updating version 1708 to 1709 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: oid 42 version 1709 is already newer than 1707 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1709 (ObjNum 1095 snap 313 seq_num 1095) dirty exists 2026-02-21T22:28:55.521 INFO:tasks.rados.rados.0.trial178.stdout:3114: left oid 42 (ObjNum 1095 snap 313 seq_num 1095) 2026-02-21T22:28:55.522 INFO:tasks.rados.rados.0.trial178.stdout:3116: finishing write tid 1 to trial17818531-14 2026-02-21T22:28:55.522 INFO:tasks.rados.rados.0.trial178.stdout:3115: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:55.522 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:55.548 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 507358 2026-02-21T22:28:55.574 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: finishing write tid 2 to trial17818531-14 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: finishing write tid 3 to trial17818531-14 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: finishing write tid 4 to trial17818531-14 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: finishing write tid 5 to trial17818531-14 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: oid 14 updating version 0 to 1460 2026-02-21T22:28:55.603 INFO:tasks.rados.rados.0.trial178.stdout:3116: oid 14 version 1460 is already newer than 1458 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3116: oid 14 updating version 1460 to 1461 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3116: oid 14 version 1461 is already newer than 1459 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1461 (ObjNum 1096 snap 313 seq_num 1096) dirty exists 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3116: left oid 14 (ObjNum 1096 snap 313 seq_num 1096) 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: finishing write tid 1 to trial17818531-36 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: finishing write tid 2 to trial17818531-36 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: finishing write tid 3 to trial17818531-36 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: finishing write tid 4 to trial17818531-36 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: oid 36 updating version 0 to 1104 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: oid 36 updating version 1104 to 1105 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: oid 36 updating version 1105 to 1106 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1106 (ObjNum 1097 snap 313 seq_num 1097) dirty exists 2026-02-21T22:28:55.604 INFO:tasks.rados.rados.0.trial178.stdout:3117: left oid 36 (ObjNum 1097 snap 313 seq_num 1097) 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: finishing write tid 1 to trial17818531-40 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: finishing write tid 2 to trial17818531-40 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: finishing write tid 3 to trial17818531-40 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: finishing write tid 4 to trial17818531-40 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: oid 40 updating version 0 to 1597 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: oid 40 version 1597 is already newer than 1596 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: oid 40 version 1597 is already newer than 1595 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1597 (ObjNum 1098 snap 313 seq_num 1098) dirty exists 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3119: left oid 40 (ObjNum 1098 snap 313 seq_num 1098) 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3120: finishing write tid 1 to trial17818531-50 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3120: finishing write tid 2 to trial17818531-50 2026-02-21T22:28:55.605 INFO:tasks.rados.rados.0.trial178.stdout:3120: finishing write tid 3 to trial17818531-50 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3120: finishing write tid 4 to trial17818531-50 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3120: oid 50 updating version 0 to 1599 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3120: oid 50 version 1599 is already newer than 1598 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3120: oid 50 updating version 1599 to 1600 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1600 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3120: left oid 50 (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3121: finishing copy_from racing read to trial17818531-11 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3113: done (8 left) 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3114: done (7 left) 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3115: done (6 left) 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3116: done (5 left) 2026-02-21T22:28:55.606 INFO:tasks.rados.rados.0.trial178.stdout:3117: done (4 left) 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3119: done (3 left) 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3120: done (2 left) 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3123: read oid 38 snap -1 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3123: expect deleted 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3124: snap_create 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3121: finishing copy_from to trial17818531-11 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1180 (ObjNum 1019 snap 283 seq_num 1019) dirty exists 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3122: finishing copy_from to trial17818531-15 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3122: got expected ENOENT (src dne) 2026-02-21T22:28:55.607 INFO:tasks.rados.rados.0.trial178.stdout:3122: finishing copy_from racing read to trial17818531-15 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3121: done (3 left) 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3122: done (2 left) 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3123: done (1 left) 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3124: done (0 left) 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3125: copy_from oid 29 from oid 14 current snap is 314 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3126: write (excl) oid 25 current snap is 314 2026-02-21T22:28:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3126: seq_num 1100 ranges {487700=728732,1786075=665712} 2026-02-21T22:28:56.234 INFO:tasks.rados.rados.0.trial178.stdout:3126: writing trial17818531-25 from 487700 to 1216432 tid 1 2026-02-21T22:28:56.242 INFO:tasks.rados.rados.0.trial178.stdout:3126: writing trial17818531-25 from 1786075 to 2451787 tid 2 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3127: read oid 12 snap -1 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3127: expect (ObjNum 1082 snap 313 seq_num 1082) 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3128: read oid 24 snap -1 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3128: expect deleted 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3129: read oid 4 snap -1 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3129: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3130: read oid 5 snap -1 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3130: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3131: write (excl) oid 3 current snap is 314 2026-02-21T22:28:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3131: seq_num 1101 ranges {504507=403981,1328204=409370,2252362=252145} 2026-02-21T22:28:56.248 INFO:tasks.rados.rados.0.trial178.stdout:3131: writing trial17818531-3 from 504507 to 908488 tid 1 2026-02-21T22:28:56.254 INFO:tasks.rados.rados.0.trial178.stdout:3131: writing trial17818531-3 from 1328204 to 1737574 tid 2 2026-02-21T22:28:56.259 INFO:tasks.rados.rados.0.trial178.stdout:3131: writing trial17818531-3 from 2252362 to 2504507 tid 3 2026-02-21T22:28:56.259 INFO:tasks.rados.rados.0.trial178.stdout:3132: write oid 38 current snap is 314 2026-02-21T22:28:56.259 INFO:tasks.rados.rados.0.trial178.stdout:3132: seq_num 1102 ranges {521314=479230,1786686=669381} 2026-02-21T22:28:56.266 INFO:tasks.rados.rados.0.trial178.stdout:3132: writing trial17818531-38 from 521314 to 1000544 tid 1 2026-02-21T22:28:56.273 INFO:tasks.rados.rados.0.trial178.stdout:3132: writing trial17818531-38 from 1786686 to 2456067 tid 2 2026-02-21T22:28:56.273 INFO:tasks.rados.rados.0.trial178.stdout:3133: rollback oid 2 current snap is 314 2026-02-21T22:28:56.273 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 305 2026-02-21T22:28:56.273 INFO:tasks.rados.rados.0.trial178.stdout:3125: finishing copy_from racing read to trial17818531-29 2026-02-21T22:28:56.274 INFO:tasks.rados.rados.0.trial178.stdout:3134: write oid 23 current snap is 314 2026-02-21T22:28:56.274 INFO:tasks.rados.rados.0.trial178.stdout:3134: seq_num 1103 ranges {538121=670832,1961521=413039} 2026-02-21T22:28:56.278 INFO:tasks.rados.rados.0.trial178.stdout:3134: writing trial17818531-23 from 538121 to 1208953 tid 1 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:3134: writing trial17818531-23 from 1961521 to 2374560 tid 2 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:3135: rollback oid 31 current snap is 314 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 303 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:3125: finishing copy_from to trial17818531-29 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1461 (ObjNum 1096 snap 313 seq_num 1096) dirty exists 2026-02-21T22:28:56.283 INFO:tasks.rados.rados.0.trial178.stdout:3126: finishing write tid 1 to trial17818531-25 2026-02-21T22:28:56.284 INFO:tasks.rados.rados.0.trial178.stdout:3125: done (10 left) 2026-02-21T22:28:56.284 INFO:tasks.rados.rados.0.trial178.stdout:3136: write (excl) oid 40 current snap is 314 2026-02-21T22:28:56.284 INFO:tasks.rados.rados.0.trial178.stdout:3136: seq_num 1104 ranges {554928=746081,2020003=534925} 2026-02-21T22:28:56.288 INFO:tasks.rados.rados.0.trial178.stdout:3136: writing trial17818531-40 from 554928 to 1301009 tid 1 2026-02-21T22:28:56.293 INFO:tasks.rados.rados.0.trial178.stdout:3136: writing trial17818531-40 from 2020003 to 2554928 tid 2 2026-02-21T22:28:56.293 INFO:tasks.rados.rados.0.trial178.stdout:3137: copy_from oid 36 from oid 37 current snap is 314 2026-02-21T22:28:56.293 INFO:tasks.rados.rados.0.trial178.stdout:3127: expect (ObjNum 1082 snap 313 seq_num 1082) 2026-02-21T22:28:56.325 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1486655 2026-02-21T22:28:56.354 INFO:tasks.rados.rados.0.trial178.stdout:3126: finishing write tid 2 to trial17818531-25 2026-02-21T22:28:56.354 INFO:tasks.rados.rados.0.trial178.stdout:3126: finishing write tid 3 to trial17818531-25 2026-02-21T22:28:56.354 INFO:tasks.rados.rados.0.trial178.stdout:3126: finishing write tid 4 to trial17818531-25 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:3126: oid 25 updating version 0 to 1461 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:3126: oid 25 version 1461 is already newer than 1460 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:3126: oid 25 version 1461 is already newer than 1459 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1461 (ObjNum 1100 snap 314 seq_num 1100) dirty exists 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:3126: left oid 25 (ObjNum 1100 snap 314 seq_num 1100) 2026-02-21T22:28:56.355 INFO:tasks.rados.rados.0.trial178.stdout:3129: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:28:56.387 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 435067 2026-02-21T22:28:56.423 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 648974 2026-02-21T22:28:56.454 INFO:tasks.rados.rados.0.trial178.stdout:3130: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:28:56.476 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: finishing write tid 1 to trial17818531-3 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: finishing write tid 2 to trial17818531-3 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: finishing write tid 3 to trial17818531-3 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: finishing write tid 4 to trial17818531-3 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: finishing write tid 5 to trial17818531-3 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: oid 3 updating version 0 to 1473 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: oid 3 version 1473 is already newer than 1472 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: oid 3 version 1473 is already newer than 1471 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:3131: oid 3 version 1473 is already newer than 1470 2026-02-21T22:28:56.530 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1473 (ObjNum 1101 snap 314 seq_num 1101) dirty exists 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3131: left oid 3 (ObjNum 1101 snap 314 seq_num 1101) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: finishing write tid 1 to trial17818531-38 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3133: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3133: finishing rollback tid 2 to trial17818531-2 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1202 (ObjNum 996 snap 282 seq_num 996) dirty exists 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: finishing write tid 2 to trial17818531-38 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: finishing write tid 3 to trial17818531-38 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3126: done (11 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3127: done (10 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3128: done (9 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3129: done (8 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3130: done (7 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3131: done (6 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3133: done (5 left) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3138: read oid 39 snap -1 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3138: expect (ObjNum 1034 snap 289 seq_num 1034) 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3139: snap_create 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: finishing write tid 4 to trial17818531-38 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: oid 38 updating version 0 to 1190 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: oid 38 updating version 1190 to 1191 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: oid 38 version 1191 is already newer than 1189 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1191 (ObjNum 1102 snap 314 seq_num 1102) dirty exists 2026-02-21T22:28:56.531 INFO:tasks.rados.rados.0.trial178.stdout:3132: left oid 38 (ObjNum 1102 snap 314 seq_num 1102) 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: finishing write tid 1 to trial17818531-23 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: finishing write tid 2 to trial17818531-23 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: finishing write tid 3 to trial17818531-23 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3135: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3135: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: finishing write tid 4 to trial17818531-23 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: oid 23 updating version 0 to 1391 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: oid 23 version 1391 is already newer than 1390 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: oid 23 version 1391 is already newer than 1389 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1391 (ObjNum 1103 snap 314 seq_num 1103) dirty exists 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3134: left oid 23 (ObjNum 1103 snap 314 seq_num 1103) 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3135: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1464 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: finishing write tid 1 to trial17818531-40 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3137: finishing copy_from to trial17818531-36 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3137: got expected ENOENT (src dne) 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3137: finishing copy_from racing read to trial17818531-36 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: finishing write tid 2 to trial17818531-40 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: finishing write tid 3 to trial17818531-40 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: finishing write tid 4 to trial17818531-40 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: oid 40 updating version 0 to 1601 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: oid 40 updating version 1601 to 1602 2026-02-21T22:28:56.532 INFO:tasks.rados.rados.0.trial178.stdout:3136: oid 40 updating version 1602 to 1603 2026-02-21T22:28:56.533 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1603 (ObjNum 1104 snap 314 seq_num 1104) dirty exists 2026-02-21T22:28:56.533 INFO:tasks.rados.rados.0.trial178.stdout:3136: left oid 40 (ObjNum 1104 snap 314 seq_num 1104) 2026-02-21T22:28:56.538 INFO:tasks.rados.rados.0.trial178.stdout:3138: expect (ObjNum 1034 snap 289 seq_num 1034) 2026-02-21T22:28:56.538 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:56.580 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:28:56.643 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2285612 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3132: done (6 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3134: done (5 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3135: done (4 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3136: done (3 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3137: done (2 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3138: done (1 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3139: done (0 left) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3140: read oid 27 snap -1 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3140: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:28:57.192 INFO:tasks.rados.rados.0.trial178.stdout:3141: snap_create 2026-02-21T22:28:57.205 INFO:tasks.rados.rados.0.trial178.stdout:3140: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:28:57.283 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1018950 2026-02-21T22:28:58.197 INFO:tasks.rados.rados.0.trial178.stdout:3140: done (1 left) 2026-02-21T22:28:58.198 INFO:tasks.rados.rados.0.trial178.stdout:3141: done (0 left) 2026-02-21T22:28:58.198 INFO:tasks.rados.rados.0.trial178.stdout:3142: delete oid 36 current snap is 316 2026-02-21T22:28:58.201 INFO:tasks.rados.rados.0.trial178.stdout:3142: done (0 left) 2026-02-21T22:28:58.201 INFO:tasks.rados.rados.0.trial178.stdout:3143: snap_create 2026-02-21T22:28:58.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:28:58.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:28:59.209 INFO:tasks.rados.rados.0.trial178.stdout:3143: done (0 left) 2026-02-21T22:28:59.209 INFO:tasks.rados.rados.0.trial178.stdout:3144: copy_from oid 5 from oid 35 current snap is 317 2026-02-21T22:28:59.209 INFO:tasks.rados.rados.0.trial178.stdout:3145: snap_create 2026-02-21T22:28:59.222 INFO:tasks.rados.rados.0.trial178.stdout:3144: finishing copy_from racing read to trial17818531-5 2026-02-21T22:28:59.223 INFO:tasks.rados.rados.0.trial178.stdout:3144: finishing copy_from to trial17818531-5 2026-02-21T22:28:59.223 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1224 (ObjNum 1022 snap 283 seq_num 1022) dirty exists 2026-02-21T22:29:00.210 INFO:tasks.rados.rados.0.trial178.stdout:3144: done (1 left) 2026-02-21T22:29:00.210 INFO:tasks.rados.rados.0.trial178.stdout:3145: done (0 left) 2026-02-21T22:29:00.210 INFO:tasks.rados.rados.0.trial178.stdout:3146: write oid 26 current snap is 318 2026-02-21T22:29:00.210 INFO:tasks.rados.rados.0.trial178.stdout:3146: seq_num 1106 ranges {588542=496579,1620614=676719} 2026-02-21T22:29:00.217 INFO:tasks.rados.rados.0.trial178.stdout:3146: writing trial17818531-26 from 588542 to 1085121 tid 1 2026-02-21T22:29:00.226 INFO:tasks.rados.rados.0.trial178.stdout:3146: writing trial17818531-26 from 1620614 to 2297333 tid 2 2026-02-21T22:29:00.226 INFO:tasks.rados.rados.0.trial178.stdout:3147: copy_from oid 29 from oid 5 current snap is 318 2026-02-21T22:29:00.226 INFO:tasks.rados.rados.0.trial178.stdout:3146: finishing write tid 1 to trial17818531-26 2026-02-21T22:29:00.226 INFO:tasks.rados.rados.0.trial178.stdout:3148: delete oid 46 current snap is 318 2026-02-21T22:29:00.227 INFO:tasks.rados.rados.0.trial178.stdout:3148: done (2 left) 2026-02-21T22:29:00.227 INFO:tasks.rados.rados.0.trial178.stdout:3149: rollback oid 44 current snap is 318 2026-02-21T22:29:00.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 317 2026-02-21T22:29:00.227 INFO:tasks.rados.rados.0.trial178.stdout:3150: write oid 17 current snap is 318 2026-02-21T22:29:00.227 INFO:tasks.rados.rados.0.trial178.stdout:3150: seq_num 1108 ranges {622156=647077,1737578=680388} 2026-02-21T22:29:00.236 INFO:tasks.rados.rados.0.trial178.stdout:3150: writing trial17818531-17 from 622156 to 1269233 tid 1 2026-02-21T22:29:00.244 INFO:tasks.rados.rados.0.trial178.stdout:3150: writing trial17818531-17 from 1737578 to 2417966 tid 2 2026-02-21T22:29:00.244 INFO:tasks.rados.rados.0.trial178.stdout:3151: write (excl) oid 32 current snap is 318 2026-02-21T22:29:00.244 INFO:tasks.rados.rados.0.trial178.stdout:3151: seq_num 1109 ranges {638963=722326,2079707=424046} 2026-02-21T22:29:00.253 INFO:tasks.rados.rados.0.trial178.stdout:3151: writing trial17818531-32 from 638963 to 1361289 tid 1 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3151: writing trial17818531-32 from 2079707 to 2503753 tid 2 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3152: rollback oid 38 current snap is 318 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 315 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3149: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1458 (ObjNum 177 snap 0 seq_num 101014976) dirty dne 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3149: done (5 left) 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3153: write (excl) oid 40 current snap is 318 2026-02-21T22:29:00.260 INFO:tasks.rados.rados.0.trial178.stdout:3153: seq_num 1110 ranges {655770=513928,1854542=684057} 2026-02-21T22:29:00.264 INFO:tasks.rados.rados.0.trial178.stdout:3153: writing trial17818531-40 from 655770 to 1169698 tid 1 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3153: writing trial17818531-40 from 1854542 to 2538599 tid 2 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3154: delete oid 47 current snap is 318 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: finishing write tid 2 to trial17818531-26 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: finishing write tid 3 to trial17818531-26 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: finishing write tid 4 to trial17818531-26 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: oid 26 updating version 0 to 1205 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: oid 26 version 1205 is already newer than 1204 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: oid 26 version 1205 is already newer than 1203 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1205 (ObjNum 1106 snap 318 seq_num 1106) dirty exists 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3146: left oid 26 (ObjNum 1106 snap 318 seq_num 1106) 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3147: finishing copy_from racing read to trial17818531-29 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: finishing write tid 1 to trial17818531-17 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3147: finishing copy_from to trial17818531-29 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1224 (ObjNum 1022 snap 283 seq_num 1022) dirty exists 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: finishing write tid 2 to trial17818531-17 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: finishing write tid 3 to trial17818531-17 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: finishing write tid 4 to trial17818531-17 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: oid 17 updating version 0 to 1237 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: oid 17 version 1237 is already newer than 1236 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: oid 17 updating version 1237 to 1238 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1238 (ObjNum 1108 snap 318 seq_num 1108) dirty exists 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3150: left oid 17 (ObjNum 1108 snap 318 seq_num 1108) 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: finishing write tid 1 to trial17818531-32 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3152: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: finishing write tid 2 to trial17818531-32 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3152: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3152: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1194 (ObjNum 1102 snap 314 seq_num 1102) dirty exists 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: finishing write tid 3 to trial17818531-32 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: finishing write tid 4 to trial17818531-32 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: oid 32 updating version 0 to 1475 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: oid 32 version 1475 is already newer than 1474 2026-02-21T22:29:00.270 INFO:tasks.rados.rados.0.trial178.stdout:3151: oid 32 updating version 1475 to 1476 2026-02-21T22:29:00.271 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1476 (ObjNum 1109 snap 318 seq_num 1109) dirty exists 2026-02-21T22:29:00.271 INFO:tasks.rados.rados.0.trial178.stdout:3151: left oid 32 (ObjNum 1109 snap 318 seq_num 1109) 2026-02-21T22:29:00.271 INFO:tasks.rados.rados.0.trial178.stdout:3153: finishing write tid 1 to trial17818531-40 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: finishing write tid 2 to trial17818531-40 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: finishing write tid 3 to trial17818531-40 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: finishing write tid 4 to trial17818531-40 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: oid 40 updating version 0 to 1606 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: oid 40 version 1606 is already newer than 1605 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: oid 40 version 1606 is already newer than 1604 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1606 (ObjNum 1110 snap 318 seq_num 1110) dirty exists 2026-02-21T22:29:00.272 INFO:tasks.rados.rados.0.trial178.stdout:3153: left oid 40 (ObjNum 1110 snap 318 seq_num 1110) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3146: done (6 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3147: done (5 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3150: done (4 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3151: done (3 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3152: done (2 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3153: done (1 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3154: done (0 left) 2026-02-21T22:29:00.273 INFO:tasks.rados.rados.0.trial178.stdout:3155: snap_remove snap 316 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:3155: done (0 left) 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:3156: read oid 12 snap -1 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:3156: expect (ObjNum 1082 snap 313 seq_num 1082) 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:3157: rollback oid 13 current snap is 318 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 313 2026-02-21T22:29:01.215 INFO:tasks.rados.rados.0.trial178.stdout:3158: snap_remove snap 314 2026-02-21T22:29:02.191 INFO:tasks.rados.rados.0.trial178.stdout:3158: done (2 left) 2026-02-21T22:29:02.192 INFO:tasks.rados.rados.0.trial178.stdout:3159: read oid 34 snap 315 2026-02-21T22:29:02.192 INFO:tasks.rados.rados.0.trial178.stdout:3159: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:29:02.192 INFO:tasks.rados.rados.0.trial178.stdout:3160: copy_from oid 45 from oid 32 current snap is 318 2026-02-21T22:29:02.192 INFO:tasks.rados.rados.0.trial178.stdout:3156: expect (ObjNum 1082 snap 313 seq_num 1082) 2026-02-21T22:29:02.254 INFO:tasks.rados.rados.0.trial178.stdout:3157: finishing rollback tid 0 to trial17818531-13 2026-02-21T22:29:02.255 INFO:tasks.rados.rados.0.trial178.stdout:3157: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:29:02.255 INFO:tasks.rados.rados.0.trial178.stdout:3157: finishing rollback tid 2 to trial17818531-13 2026-02-21T22:29:02.255 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1480 (ObjNum 769 snap 217 seq_num 769) dirty exists 2026-02-21T22:29:02.255 INFO:tasks.rados.rados.0.trial178.stdout:3159: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:29:02.310 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 200028 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3160: finishing copy_from racing read to trial17818531-45 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3160: finishing copy_from to trial17818531-45 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1476 (ObjNum 1109 snap 318 seq_num 1109) dirty exists 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3156: done (3 left) 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3157: done (2 left) 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3159: done (1 left) 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3160: done (0 left) 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3161: read oid 43 snap -1 2026-02-21T22:29:02.337 INFO:tasks.rados.rados.0.trial178.stdout:3161: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:29:02.338 INFO:tasks.rados.rados.0.trial178.stdout:3162: delete oid 23 current snap is 318 2026-02-21T22:29:02.343 INFO:tasks.rados.rados.0.trial178.stdout:3161: expect (ObjNum 998 snap 283 seq_num 998) 2026-02-21T22:29:02.370 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2189211 2026-02-21T22:29:02.415 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1482412 2026-02-21T22:29:02.431 INFO:tasks.rados.rados.0.trial178.stdout:3161: done (1 left) 2026-02-21T22:29:02.431 INFO:tasks.rados.rados.0.trial178.stdout:3162: done (0 left) 2026-02-21T22:29:02.432 INFO:tasks.rados.rados.0.trial178.stdout:3163: read oid 13 snap -1 2026-02-21T22:29:02.432 INFO:tasks.rados.rados.0.trial178.stdout:3163: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:29:02.432 INFO:tasks.rados.rados.0.trial178.stdout:3164: snap_remove snap 315 2026-02-21T22:29:03.224 INFO:tasks.rados.rados.0.trial178.stdout:3164: done (1 left) 2026-02-21T22:29:03.224 INFO:tasks.rados.rados.0.trial178.stdout:3165: copy_from oid 35 from oid 20 current snap is 318 2026-02-21T22:29:03.224 INFO:tasks.rados.rados.0.trial178.stdout:3163: expect (ObjNum 769 snap 217 seq_num 769) 2026-02-21T22:29:03.275 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1994780 2026-02-21T22:29:03.288 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3165: finishing copy_from racing read to trial17818531-35 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3165: finishing copy_from to trial17818531-35 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1235 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3163: done (1 left) 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3165: done (0 left) 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3166: copy_from oid 8 from oid 3 current snap is 318 2026-02-21T22:29:03.323 INFO:tasks.rados.rados.0.trial178.stdout:3167: delete oid 4 current snap is 318 2026-02-21T22:29:03.325 INFO:tasks.rados.rados.0.trial178.stdout:3167: done (1 left) 2026-02-21T22:29:03.325 INFO:tasks.rados.rados.0.trial178.stdout:3168: copy_from oid 33 from oid 23 current snap is 318 2026-02-21T22:29:03.325 INFO:tasks.rados.rados.0.trial178.stdout:3169: snap_remove snap 317 2026-02-21T22:29:03.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:03.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:04.228 INFO:tasks.rados.rados.0.trial178.stdout:3169: done (2 left) 2026-02-21T22:29:04.228 INFO:tasks.rados.rados.0.trial178.stdout:3170: write (excl) oid 47 current snap is 318 2026-02-21T22:29:04.228 INFO:tasks.rados.rados.0.trial178.stdout:3170: seq_num 1114 ranges {722998=414924,1572117=575042} 2026-02-21T22:29:04.236 INFO:tasks.rados.rados.0.trial178.stdout:3170: writing trial17818531-47 from 722998 to 1137922 tid 1 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3170: writing trial17818531-47 from 1572117 to 2147159 tid 2 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3171: copy_from oid 12 from oid 50 current snap is 318 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3168: finishing copy_from to trial17818531-33 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3168: got expected ENOENT (src dne) 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3168: finishing copy_from racing read to trial17818531-33 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3166: finishing copy_from racing read to trial17818531-8 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3166: finishing copy_from to trial17818531-8 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1473 (ObjNum 1101 snap 314 seq_num 1101) dirty exists 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3166: done (3 left) 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3168: done (2 left) 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3172: copy_from oid 36 from oid 10 current snap is 318 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3173: rollback oid 11 current snap is 318 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 309 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3174: write oid 39 current snap is 318 2026-02-21T22:29:04.243 INFO:tasks.rados.rados.0.trial178.stdout:3174: seq_num 1115 ranges {739805=490173,1630599=435053,2636284=103521} 2026-02-21T22:29:04.251 INFO:tasks.rados.rados.0.trial178.stdout:3174: writing trial17818531-39 from 739805 to 1229978 tid 1 2026-02-21T22:29:04.256 INFO:tasks.rados.rados.0.trial178.stdout:3174: writing trial17818531-39 from 1630599 to 2065652 tid 2 2026-02-21T22:29:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3174: writing trial17818531-39 from 2636284 to 2739805 tid 3 2026-02-21T22:29:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3175: write (excl) oid 28 current snap is 318 2026-02-21T22:29:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3175: seq_num 1116 ranges {756612=565422,2089081=578711} 2026-02-21T22:29:04.270 INFO:tasks.rados.rados.0.trial178.stdout:3175: writing trial17818531-28 from 756612 to 1322034 tid 1 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3175: writing trial17818531-28 from 2089081 to 2667792 tid 2 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3176: copy_from oid 3 from oid 49 current snap is 318 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3172: finishing copy_from to trial17818531-36 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3172: got expected ENOENT (src dne) 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3172: finishing copy_from racing read to trial17818531-36 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3173: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3173: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3173: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1502 (ObjNum 1015 snap 283 seq_num 1015) dirty exists 2026-02-21T22:29:04.279 INFO:tasks.rados.rados.0.trial178.stdout:3170: finishing write tid 1 to trial17818531-47 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3172: done (6 left) 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3173: done (5 left) 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3177: read oid 2 snap -1 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3177: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3178: read oid 4 snap 309 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3178: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:29:04.280 INFO:tasks.rados.rados.0.trial178.stdout:3179: snap_remove snap 313 2026-02-21T22:29:05.230 INFO:tasks.rados.rados.0.trial178.stdout:3179: done (7 left) 2026-02-21T22:29:05.230 INFO:tasks.rados.rados.0.trial178.stdout:3180: snap_create 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: finishing write tid 2 to trial17818531-47 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: finishing write tid 3 to trial17818531-47 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: finishing write tid 4 to trial17818531-47 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: oid 47 updating version 0 to 1730 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: oid 47 version 1730 is already newer than 1728 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: oid 47 version 1730 is already newer than 1729 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1730 (ObjNum 1114 snap 318 seq_num 1114) dirty exists 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3170: left oid 47 (ObjNum 1114 snap 318 seq_num 1114) 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3174: finishing write tid 1 to trial17818531-39 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3171: finishing copy_from racing read to trial17818531-12 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3171: finishing copy_from to trial17818531-12 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1600 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3174: finishing write tid 2 to trial17818531-39 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3174: finishing write tid 3 to trial17818531-39 2026-02-21T22:29:05.231 INFO:tasks.rados.rados.0.trial178.stdout:3174: finishing write tid 4 to trial17818531-39 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: finishing write tid 5 to trial17818531-39 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: oid 39 updating version 0 to 1462 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: oid 39 updating version 1462 to 1465 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: oid 39 version 1465 is already newer than 1464 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: oid 39 version 1465 is already newer than 1463 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1465 (ObjNum 1115 snap 318 seq_num 1115) dirty exists 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3174: left oid 39 (ObjNum 1115 snap 318 seq_num 1115) 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3175: finishing write tid 1 to trial17818531-28 2026-02-21T22:29:05.232 INFO:tasks.rados.rados.0.trial178.stdout:3178: expect (ObjNum 981 snap 281 seq_num 981) 2026-02-21T22:29:05.348 INFO:tasks.rados.rados.0.trial178.stdout:3175: finishing write tid 2 to trial17818531-28 2026-02-21T22:29:05.348 INFO:tasks.rados.rados.0.trial178.stdout:3175: finishing write tid 3 to trial17818531-28 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3175: finishing write tid 4 to trial17818531-28 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3175: oid 28 updating version 0 to 1197 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3175: oid 28 version 1197 is already newer than 1195 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3175: oid 28 version 1197 is already newer than 1196 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1197 (ObjNum 1116 snap 318 seq_num 1116) dirty exists 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3175: left oid 28 (ObjNum 1116 snap 318 seq_num 1116) 2026-02-21T22:29:05.349 INFO:tasks.rados.rados.0.trial178.stdout:3177: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:29:05.391 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:05.439 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:05.487 INFO:tasks.rados.rados.0.trial178.stdout:3176: finishing copy_from racing read to trial17818531-3 2026-02-21T22:29:05.487 INFO:tasks.rados.rados.0.trial178.stdout:3176: finishing copy_from to trial17818531-3 2026-02-21T22:29:05.488 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1452 (ObjNum 1080 snap 313 seq_num 1080) dirty exists 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3170: done (7 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3171: done (6 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3174: done (5 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3175: done (4 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3176: done (3 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3177: done (2 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3178: done (1 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3180: done (0 left) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3181: copy_from oid 15 from oid 35 current snap is 319 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3182: read oid 32 snap -1 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3182: expect (ObjNum 1109 snap 318 seq_num 1109) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3183: read oid 27 snap -1 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3183: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:29:06.245 INFO:tasks.rados.rados.0.trial178.stdout:3184: delete oid 11 current snap is 319 2026-02-21T22:29:06.255 INFO:tasks.rados.rados.0.trial178.stdout:3183: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:29:06.435 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2145022 2026-02-21T22:29:06.464 INFO:tasks.rados.rados.0.trial178.stdout:3182: expect (ObjNum 1109 snap 318 seq_num 1109) 2026-02-21T22:29:06.464 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:06.558 INFO:tasks.rados.rados.0.trial178.stdout:3181: finishing copy_from racing read to trial17818531-15 2026-02-21T22:29:06.558 INFO:tasks.rados.rados.0.trial178.stdout:3181: finishing copy_from to trial17818531-15 2026-02-21T22:29:06.558 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1235 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:29:06.558 INFO:tasks.rados.rados.0.trial178.stdout:3181: done (3 left) 2026-02-21T22:29:06.559 INFO:tasks.rados.rados.0.trial178.stdout:3182: done (2 left) 2026-02-21T22:29:06.559 INFO:tasks.rados.rados.0.trial178.stdout:3183: done (1 left) 2026-02-21T22:29:06.559 INFO:tasks.rados.rados.0.trial178.stdout:3184: done (0 left) 2026-02-21T22:29:06.559 INFO:tasks.rados.rados.0.trial178.stdout:3185: snap_create 2026-02-21T22:29:07.193 INFO:tasks.rados.rados.0.trial178.stdout:3185: done (0 left) 2026-02-21T22:29:07.193 INFO:tasks.rados.rados.0.trial178.stdout:3186: read oid 19 snap 309 2026-02-21T22:29:07.193 INFO:tasks.rados.rados.0.trial178.stdout:3186: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:29:07.194 INFO:tasks.rados.rados.0.trial178.stdout:3187: write oid 29 current snap is 320 2026-02-21T22:29:07.194 INFO:tasks.rados.rados.0.trial178.stdout:3187: seq_num 1118 ranges {790226=432273,1806045=582380} 2026-02-21T22:29:07.201 INFO:tasks.rados.rados.0.trial178.stdout:3187: writing trial17818531-29 from 790226 to 1222499 tid 1 2026-02-21T22:29:07.209 INFO:tasks.rados.rados.0.trial178.stdout:3187: writing trial17818531-29 from 1806045 to 2388425 tid 2 2026-02-21T22:29:07.209 INFO:tasks.rados.rados.0.trial178.stdout:3188: read oid 49 snap 305 2026-02-21T22:29:07.209 INFO:tasks.rados.rados.0.trial178.stdout:3188: expect deleted 2026-02-21T22:29:07.209 INFO:tasks.rados.rados.0.trial178.stdout:3189: snap_remove snap 318 2026-02-21T22:29:08.197 INFO:tasks.rados.rados.0.trial178.stdout:3189: done (3 left) 2026-02-21T22:29:08.197 INFO:tasks.rados.rados.0.trial178.stdout:3190: read oid 31 snap -1 2026-02-21T22:29:08.197 INFO:tasks.rados.rados.0.trial178.stdout:3190: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:29:08.197 INFO:tasks.rados.rados.0.trial178.stdout:3191: write (excl) oid 48 current snap is 320 2026-02-21T22:29:08.197 INFO:tasks.rados.rados.0.trial178.stdout:3191: seq_num 1119 ranges {407033=507522,1464527=442391,2345976=461057} 2026-02-21T22:29:08.203 INFO:tasks.rados.rados.0.trial178.stdout:3191: writing trial17818531-48 from 407033 to 914555 tid 1 2026-02-21T22:29:08.210 INFO:tasks.rados.rados.0.trial178.stdout:3191: writing trial17818531-48 from 1464527 to 1906918 tid 2 2026-02-21T22:29:08.216 INFO:tasks.rados.rados.0.trial178.stdout:3191: writing trial17818531-48 from 2345976 to 2807033 tid 3 2026-02-21T22:29:08.216 INFO:tasks.rados.rados.0.trial178.stdout:3192: write oid 6 current snap is 320 2026-02-21T22:29:08.216 INFO:tasks.rados.rados.0.trial178.stdout:3192: seq_num 1120 ranges {423840=582771,1523009=586049,2657046=166794} 2026-02-21T22:29:08.223 INFO:tasks.rados.rados.0.trial178.stdout:3192: writing trial17818531-6 from 423840 to 1006611 tid 1 2026-02-21T22:29:08.231 INFO:tasks.rados.rados.0.trial178.stdout:3192: writing trial17818531-6 from 1523009 to 2109058 tid 2 2026-02-21T22:29:08.236 INFO:tasks.rados.rados.0.trial178.stdout:3192: writing trial17818531-6 from 2657046 to 2823840 tid 3 2026-02-21T22:29:08.236 INFO:tasks.rados.rados.0.trial178.stdout:3193: snap_create 2026-02-21T22:29:08.236 INFO:tasks.rados.rados.0.trial178.stdout:3186: expect (ObjNum 1010 snap 283 seq_num 1010) 2026-02-21T22:29:08.304 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:08.398 INFO:tasks.rados.rados.0.trial178.stdout:3187: finishing write tid 1 to trial17818531-29 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: finishing write tid 2 to trial17818531-29 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: finishing write tid 3 to trial17818531-29 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: finishing write tid 4 to trial17818531-29 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: oid 29 updating version 0 to 1463 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: oid 29 version 1463 is already newer than 1462 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: oid 29 updating version 1463 to 1464 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1464 (ObjNum 1118 snap 320 seq_num 1118) dirty exists 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3187: left oid 29 (ObjNum 1118 snap 320 seq_num 1118) 2026-02-21T22:29:08.399 INFO:tasks.rados.rados.0.trial178.stdout:3190: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:29:08.444 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: finishing write tid 1 to trial17818531-48 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: finishing write tid 2 to trial17818531-48 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: finishing write tid 3 to trial17818531-48 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: finishing write tid 4 to trial17818531-48 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: finishing write tid 5 to trial17818531-48 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: oid 48 updating version 0 to 1479 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: oid 48 version 1479 is already newer than 1477 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: oid 48 version 1479 is already newer than 1478 2026-02-21T22:29:08.472 INFO:tasks.rados.rados.0.trial178.stdout:3191: oid 48 updating version 1479 to 1480 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1480 (ObjNum 1119 snap 320 seq_num 1119) dirty exists 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3191: left oid 48 (ObjNum 1119 snap 320 seq_num 1119) 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: finishing write tid 1 to trial17818531-6 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: finishing write tid 2 to trial17818531-6 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: finishing write tid 3 to trial17818531-6 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: finishing write tid 4 to trial17818531-6 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: finishing write tid 5 to trial17818531-6 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: oid 6 updating version 0 to 1209 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: oid 6 version 1209 is already newer than 1208 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: oid 6 version 1209 is already newer than 1207 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:3192: oid 6 version 1209 is already newer than 1206 2026-02-21T22:29:08.473 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1209 (ObjNum 1120 snap 320 seq_num 1120) dirty exists 2026-02-21T22:29:08.474 INFO:tasks.rados.rados.0.trial178.stdout:3192: left oid 6 (ObjNum 1120 snap 320 seq_num 1120) 2026-02-21T22:29:08.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:08.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3186: done (6 left) 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3187: done (5 left) 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3188: done (4 left) 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3190: done (3 left) 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3191: done (2 left) 2026-02-21T22:29:09.239 INFO:tasks.rados.rados.0.trial178.stdout:3192: done (1 left) 2026-02-21T22:29:09.240 INFO:tasks.rados.rados.0.trial178.stdout:3193: done (0 left) 2026-02-21T22:29:09.240 INFO:tasks.rados.rados.0.trial178.stdout:3194: snap_remove snap 303 2026-02-21T22:29:10.242 INFO:tasks.rados.rados.0.trial178.stdout:3194: done (0 left) 2026-02-21T22:29:10.242 INFO:tasks.rados.rados.0.trial178.stdout:3195: write oid 25 current snap is 321 2026-02-21T22:29:10.242 INFO:tasks.rados.rados.0.trial178.stdout:3195: seq_num 1121 ranges {440647=658020,1581491=446060,2800822=39825} 2026-02-21T22:29:10.249 INFO:tasks.rados.rados.0.trial178.stdout:3195: writing trial17818531-25 from 440647 to 1098667 tid 1 2026-02-21T22:29:10.255 INFO:tasks.rados.rados.0.trial178.stdout:3195: writing trial17818531-25 from 1581491 to 2027551 tid 2 2026-02-21T22:29:10.261 INFO:tasks.rados.rados.0.trial178.stdout:3195: writing trial17818531-25 from 2800822 to 2840647 tid 3 2026-02-21T22:29:10.261 INFO:tasks.rados.rados.0.trial178.stdout:3196: read oid 20 snap -1 2026-02-21T22:29:10.261 INFO:tasks.rados.rados.0.trial178.stdout:3196: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:10.261 INFO:tasks.rados.rados.0.trial178.stdout:3197: snap_remove snap 309 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3197: done (2 left) 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3198: delete oid 40 current snap is 321 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: finishing write tid 1 to trial17818531-25 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: finishing write tid 2 to trial17818531-25 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: finishing write tid 3 to trial17818531-25 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: finishing write tid 4 to trial17818531-25 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: finishing write tid 5 to trial17818531-25 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: oid 25 updating version 0 to 1468 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: oid 25 version 1468 is already newer than 1466 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: oid 25 version 1468 is already newer than 1467 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: oid 25 version 1468 is already newer than 1465 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1468 (ObjNum 1121 snap 321 seq_num 1121) dirty exists 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3195: left oid 25 (ObjNum 1121 snap 321 seq_num 1121) 2026-02-21T22:29:11.243 INFO:tasks.rados.rados.0.trial178.stdout:3196: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:11.290 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:11.346 INFO:tasks.rados.rados.0.trial178.stdout:3195: done (2 left) 2026-02-21T22:29:11.346 INFO:tasks.rados.rados.0.trial178.stdout:3196: done (1 left) 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3198: done (0 left) 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3199: read oid 48 snap -1 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3199: expect (ObjNum 1119 snap 320 seq_num 1119) 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3200: rollback oid 4 current snap is 321 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 320 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3201: write oid 11 current snap is 321 2026-02-21T22:29:11.347 INFO:tasks.rados.rados.0.trial178.stdout:3201: seq_num 1123 ranges {474261=408518,1582102=449729,2739315=134946} 2026-02-21T22:29:11.352 INFO:tasks.rados.rados.0.trial178.stdout:3201: writing trial17818531-11 from 474261 to 882779 tid 1 2026-02-21T22:29:11.360 INFO:tasks.rados.rados.0.trial178.stdout:3201: writing trial17818531-11 from 1582102 to 2031831 tid 2 2026-02-21T22:29:11.366 INFO:tasks.rados.rados.0.trial178.stdout:3201: writing trial17818531-11 from 2739315 to 2874261 tid 3 2026-02-21T22:29:11.366 INFO:tasks.rados.rados.0.trial178.stdout:3202: write oid 13 current snap is 321 2026-02-21T22:29:11.367 INFO:tasks.rados.rados.0.trial178.stdout:3202: seq_num 1124 ranges {491068=483767,1640584=593387,2650385=240683} 2026-02-21T22:29:11.373 INFO:tasks.rados.rados.0.trial178.stdout:3202: writing trial17818531-13 from 491068 to 974835 tid 1 2026-02-21T22:29:11.382 INFO:tasks.rados.rados.0.trial178.stdout:3202: writing trial17818531-13 from 1640584 to 2233971 tid 2 2026-02-21T22:29:11.387 INFO:tasks.rados.rados.0.trial178.stdout:3202: writing trial17818531-13 from 2650385 to 2891068 tid 3 2026-02-21T22:29:11.387 INFO:tasks.rados.rados.0.trial178.stdout:3203: snap_create 2026-02-21T22:29:11.387 INFO:tasks.rados.rados.0.trial178.stdout:3199: expect (ObjNum 1119 snap 320 seq_num 1119) 2026-02-21T22:29:11.418 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 974314 2026-02-21T22:29:11.445 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1598196 2026-02-21T22:29:11.457 INFO:tasks.rados.rados.0.trial178.stdout:3200: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1462 (ObjNum 81 snap 0 seq_num 0) dirty dne 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: finishing write tid 1 to trial17818531-11 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: finishing write tid 2 to trial17818531-11 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: finishing write tid 3 to trial17818531-11 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: finishing write tid 4 to trial17818531-11 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: finishing write tid 5 to trial17818531-11 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: oid 11 updating version 0 to 1541 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: oid 11 version 1541 is already newer than 1538 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: oid 11 version 1541 is already newer than 1539 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: oid 11 version 1541 is already newer than 1540 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1541 (ObjNum 1123 snap 321 seq_num 1123) dirty exists 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3201: left oid 11 (ObjNum 1123 snap 321 seq_num 1123) 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: finishing write tid 1 to trial17818531-13 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: finishing write tid 2 to trial17818531-13 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: finishing write tid 3 to trial17818531-13 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: finishing write tid 4 to trial17818531-13 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: finishing write tid 5 to trial17818531-13 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: oid 13 updating version 0 to 1544 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: oid 13 version 1544 is already newer than 1543 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: oid 13 updating version 1544 to 1545 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: oid 13 updating version 1545 to 1546 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1546 (ObjNum 1124 snap 321 seq_num 1124) dirty exists 2026-02-21T22:29:11.458 INFO:tasks.rados.rados.0.trial178.stdout:3202: left oid 13 (ObjNum 1124 snap 321 seq_num 1124) 2026-02-21T22:29:12.195 INFO:tasks.rados.rados.0.trial178.stdout:3199: done (4 left) 2026-02-21T22:29:12.195 INFO:tasks.rados.rados.0.trial178.stdout:3200: done (3 left) 2026-02-21T22:29:12.195 INFO:tasks.rados.rados.0.trial178.stdout:3201: done (2 left) 2026-02-21T22:29:12.195 INFO:tasks.rados.rados.0.trial178.stdout:3202: done (1 left) 2026-02-21T22:29:12.195 INFO:tasks.rados.rados.0.trial178.stdout:3203: done (0 left) 2026-02-21T22:29:12.196 INFO:tasks.rados.rados.0.trial178.stdout:3204: read oid 2 snap -1 2026-02-21T22:29:12.196 INFO:tasks.rados.rados.0.trial178.stdout:3204: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:29:12.196 INFO:tasks.rados.rados.0.trial178.stdout:3205: delete oid 6 current snap is 322 2026-02-21T22:29:12.200 INFO:tasks.rados.rados.0.trial178.stdout:3204: expect (ObjNum 996 snap 282 seq_num 996) 2026-02-21T22:29:12.272 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1941563 2026-02-21T22:29:12.329 INFO:tasks.rados.rados.0.trial178.stdout:3204: done (1 left) 2026-02-21T22:29:12.329 INFO:tasks.rados.rados.0.trial178.stdout:3205: done (0 left) 2026-02-21T22:29:12.329 INFO:tasks.rados.rados.0.trial178.stdout:3206: delete oid 22 current snap is 322 2026-02-21T22:29:12.330 INFO:tasks.rados.rados.0.trial178.stdout:3206: done (0 left) 2026-02-21T22:29:12.330 INFO:tasks.rados.rados.0.trial178.stdout:3207: snap_remove snap 320 2026-02-21T22:29:13.250 INFO:tasks.rados.rados.0.trial178.stdout:3207: done (0 left) 2026-02-21T22:29:13.250 INFO:tasks.rados.rados.0.trial178.stdout:3208: snap_create 2026-02-21T22:29:13.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:13.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:14.253 INFO:tasks.rados.rados.0.trial178.stdout:3208: done (0 left) 2026-02-21T22:29:14.254 INFO:tasks.rados.rados.0.trial178.stdout:3209: rollback oid 16 current snap is 323 2026-02-21T22:29:14.254 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 319 2026-02-21T22:29:14.254 INFO:tasks.rados.rados.0.trial178.stdout:3210: copy_from oid 21 from oid 48 current snap is 323 2026-02-21T22:29:14.254 INFO:tasks.rados.rados.0.trial178.stdout:3211: write (excl) oid 22 current snap is 323 2026-02-21T22:29:14.254 INFO:tasks.rados.rados.0.trial178.stdout:3211: seq_num 1127 ranges {541489=425867,1416030=740714,2732654=208835} 2026-02-21T22:29:14.260 INFO:tasks.rados.rados.0.trial178.stdout:3211: writing trial17818531-22 from 541489 to 967356 tid 1 2026-02-21T22:29:14.268 INFO:tasks.rados.rados.0.trial178.stdout:3211: writing trial17818531-22 from 1416030 to 2156744 tid 2 2026-02-21T22:29:14.273 INFO:tasks.rados.rados.0.trial178.stdout:3211: writing trial17818531-22 from 2732654 to 2941489 tid 3 2026-02-21T22:29:14.274 INFO:tasks.rados.rados.0.trial178.stdout:3212: write (excl) oid 26 current snap is 323 2026-02-21T22:29:14.274 INFO:tasks.rados.rados.0.trial178.stdout:3212: seq_num 1128 ranges {558296=501116,1474512=600725,2760077=198219} 2026-02-21T22:29:14.281 INFO:tasks.rados.rados.0.trial178.stdout:3212: writing trial17818531-26 from 558296 to 1059412 tid 1 2026-02-21T22:29:14.287 INFO:tasks.rados.rados.0.trial178.stdout:3212: writing trial17818531-26 from 1474512 to 2075237 tid 2 2026-02-21T22:29:14.294 INFO:tasks.rados.rados.0.trial178.stdout:3212: writing trial17818531-26 from 2760077 to 2958296 tid 3 2026-02-21T22:29:14.294 INFO:tasks.rados.rados.0.trial178.stdout:3213: copy_from oid 43 from oid 25 current snap is 323 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3209: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3209: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3209: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1603 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3211: finishing write tid 1 to trial17818531-22 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3209: done (4 left) 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3214: delete oid 44 current snap is 323 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3210: finishing copy_from racing read to trial17818531-21 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:3210: finishing copy_from to trial17818531-21 2026-02-21T22:29:14.295 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1480 (ObjNum 1119 snap 320 seq_num 1119) dirty exists 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: finishing write tid 2 to trial17818531-22 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: finishing write tid 3 to trial17818531-22 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: finishing write tid 4 to trial17818531-22 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: finishing write tid 5 to trial17818531-22 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: oid 22 updating version 0 to 1608 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: oid 22 updating version 1608 to 1609 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: oid 22 updating version 1609 to 1611 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: oid 22 version 1611 is already newer than 1610 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1611 (ObjNum 1127 snap 323 seq_num 1127) dirty exists 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3211: left oid 22 (ObjNum 1127 snap 323 seq_num 1127) 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3212: finishing write tid 1 to trial17818531-26 2026-02-21T22:29:14.296 INFO:tasks.rados.rados.0.trial178.stdout:3212: finishing write tid 2 to trial17818531-26 2026-02-21T22:29:14.297 INFO:tasks.rados.rados.0.trial178.stdout:3210: done (4 left) 2026-02-21T22:29:14.297 INFO:tasks.rados.rados.0.trial178.stdout:3211: done (3 left) 2026-02-21T22:29:14.297 INFO:tasks.rados.rados.0.trial178.stdout:3214: done (2 left) 2026-02-21T22:29:14.297 INFO:tasks.rados.rados.0.trial178.stdout:3215: rollback oid 10 current snap is 323 2026-02-21T22:29:14.298 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 319 2026-02-21T22:29:14.298 INFO:tasks.rados.rados.0.trial178.stdout:3216: snap_remove snap 305 2026-02-21T22:29:15.255 INFO:tasks.rados.rados.0.trial178.stdout:3216: done (3 left) 2026-02-21T22:29:15.255 INFO:tasks.rados.rados.0.trial178.stdout:3217: write oid 2 current snap is 323 2026-02-21T22:29:15.255 INFO:tasks.rados.rados.0.trial178.stdout:3217: seq_num 1130 ranges {591910=651614,1875123=604394} 2026-02-21T22:29:15.263 INFO:tasks.rados.rados.0.trial178.stdout:3217: writing trial17818531-2 from 591910 to 1243524 tid 1 2026-02-21T22:29:15.271 INFO:tasks.rados.rados.0.trial178.stdout:3217: writing trial17818531-2 from 1875123 to 2479517 tid 2 2026-02-21T22:29:15.271 INFO:tasks.rados.rados.0.trial178.stdout:3218: delete oid 44 current snap is 323 2026-02-21T22:29:15.271 INFO:tasks.rados.rados.0.trial178.stdout:3212: finishing write tid 3 to trial17818531-26 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: finishing write tid 4 to trial17818531-26 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: finishing write tid 5 to trial17818531-26 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: oid 26 updating version 0 to 1212 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: oid 26 updating version 1212 to 1214 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: oid 26 version 1214 is already newer than 1211 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: oid 26 version 1214 is already newer than 1213 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1214 (ObjNum 1128 snap 323 seq_num 1128) dirty exists 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3212: left oid 26 (ObjNum 1128 snap 323 seq_num 1128) 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3215: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 0 (ObjNum 97 snap 0 seq_num 100480256) dirty dne 2026-02-21T22:29:15.272 INFO:tasks.rados.rados.0.trial178.stdout:3213: finishing copy_from racing read to trial17818531-43 2026-02-21T22:29:15.273 INFO:tasks.rados.rados.0.trial178.stdout:3213: finishing copy_from to trial17818531-43 2026-02-21T22:29:15.273 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1468 (ObjNum 1121 snap 321 seq_num 1121) dirty exists 2026-02-21T22:29:15.273 INFO:tasks.rados.rados.0.trial178.stdout:3217: finishing write tid 1 to trial17818531-2 2026-02-21T22:29:15.273 INFO:tasks.rados.rados.0.trial178.stdout:3212: done (4 left) 2026-02-21T22:29:15.273 INFO:tasks.rados.rados.0.trial178.stdout:3213: done (3 left) 2026-02-21T22:29:15.274 INFO:tasks.rados.rados.0.trial178.stdout:3215: done (2 left) 2026-02-21T22:29:15.274 INFO:tasks.rados.rados.0.trial178.stdout:3218: done (1 left) 2026-02-21T22:29:15.274 INFO:tasks.rados.rados.0.trial178.stdout:3219: copy_from oid 35 from oid 36 current snap is 323 2026-02-21T22:29:15.274 INFO:tasks.rados.rados.0.trial178.stdout:3220: snap_remove snap 319 2026-02-21T22:29:16.274 INFO:tasks.rados.rados.0.trial178.stdout:3220: done (2 left) 2026-02-21T22:29:16.274 INFO:tasks.rados.rados.0.trial178.stdout:3221: snap_remove snap 321 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3221: done (2 left) 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3222: rollback oid 12 current snap is 323 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 322 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3219: finishing copy_from to trial17818531-35 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3219: got expected ENOENT (src dne) 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3219: finishing copy_from racing read to trial17818531-35 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3217: finishing write tid 2 to trial17818531-2 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3217: finishing write tid 3 to trial17818531-2 2026-02-21T22:29:17.195 INFO:tasks.rados.rados.0.trial178.stdout:3217: finishing write tid 4 to trial17818531-2 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3217: oid 2 updating version 0 to 1216 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3217: oid 2 version 1216 is already newer than 1215 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3217: oid 2 updating version 1216 to 1217 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1217 (ObjNum 1130 snap 323 seq_num 1130) dirty exists 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3217: left oid 2 (ObjNum 1130 snap 323 seq_num 1130) 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3217: done (2 left) 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3219: done (1 left) 2026-02-21T22:29:17.196 INFO:tasks.rados.rados.0.trial178.stdout:3223: delete oid 27 current snap is 323 2026-02-21T22:29:17.205 INFO:tasks.rados.rados.0.trial178.stdout:3223: done (1 left) 2026-02-21T22:29:17.205 INFO:tasks.rados.rados.0.trial178.stdout:3224: read oid 2 snap -1 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3224: expect (ObjNum 1130 snap 323 seq_num 1130) 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3225: read oid 16 snap -1 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3225: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3226: snap_create 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3222: finishing rollback tid 0 to trial17818531-12 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3222: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:3222: finishing rollback tid 2 to trial17818531-12 2026-02-21T22:29:17.206 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1606 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:17.211 INFO:tasks.rados.rados.0.trial178.stdout:3225: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:29:17.252 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1989474 2026-02-21T22:29:17.267 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:17.294 INFO:tasks.rados.rados.0.trial178.stdout:3224: expect (ObjNum 1130 snap 323 seq_num 1130) 2026-02-21T22:29:17.329 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2149692 2026-02-21T22:29:17.402 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3222: done (3 left) 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3224: done (2 left) 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3225: done (1 left) 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3226: done (0 left) 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3227: write (excl) oid 5 current snap is 324 2026-02-21T22:29:18.197 INFO:tasks.rados.rados.0.trial178.stdout:3227: seq_num 1133 ranges {642331=593714,1766922=751721} 2026-02-21T22:29:18.206 INFO:tasks.rados.rados.0.trial178.stdout:3227: writing trial17818531-5 from 642331 to 1236045 tid 1 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3227: writing trial17818531-5 from 1766922 to 2518643 tid 2 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3228: read oid 37 snap -1 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3228: expect deleted 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3229: read oid 20 snap 323 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3229: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3230: snap_create 2026-02-21T22:29:18.215 INFO:tasks.rados.rados.0.trial178.stdout:3227: finishing write tid 1 to trial17818531-5 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: finishing write tid 2 to trial17818531-5 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: finishing write tid 3 to trial17818531-5 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: finishing write tid 4 to trial17818531-5 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: oid 5 updating version 0 to 1590 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: oid 5 updating version 1590 to 1591 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: oid 5 version 1591 is already newer than 1589 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1591 (ObjNum 1133 snap 324 seq_num 1133) dirty exists 2026-02-21T22:29:18.218 INFO:tasks.rados.rados.0.trial178.stdout:3227: left oid 5 (ObjNum 1133 snap 324 seq_num 1133) 2026-02-21T22:29:18.219 INFO:tasks.rados.rados.0.trial178.stdout:3229: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:18.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:18.584 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3227: done (3 left) 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3228: done (2 left) 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3229: done (1 left) 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3230: done (0 left) 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3231: read oid 27 snap -1 2026-02-21T22:29:19.923 INFO:tasks.rados.rados.0.trial178.stdout:3231: expect deleted 2026-02-21T22:29:19.924 INFO:tasks.rados.rados.0.trial178.stdout:3232: write oid 40 current snap is 325 2026-02-21T22:29:19.924 INFO:tasks.rados.rados.0.trial178.stdout:3232: seq_num 1134 ranges {659138=668963,2109051=611732} 2026-02-21T22:29:19.933 INFO:tasks.rados.rados.0.trial178.stdout:3232: writing trial17818531-40 from 659138 to 1328101 tid 1 2026-02-21T22:29:19.943 INFO:tasks.rados.rados.0.trial178.stdout:3232: writing trial17818531-40 from 2109051 to 2720783 tid 2 2026-02-21T22:29:19.943 INFO:tasks.rados.rados.0.trial178.stdout:3233: snap_remove snap 323 2026-02-21T22:29:20.205 INFO:tasks.rados.rados.0.trial178.stdout:3233: done (2 left) 2026-02-21T22:29:20.205 INFO:tasks.rados.rados.0.trial178.stdout:3234: write oid 4 current snap is 325 2026-02-21T22:29:20.206 INFO:tasks.rados.rados.0.trial178.stdout:3234: seq_num 1135 ranges {675945=744212,2167533=755390} 2026-02-21T22:29:20.216 INFO:tasks.rados.rados.0.trial178.stdout:3234: writing trial17818531-4 from 675945 to 1420157 tid 1 2026-02-21T22:29:20.226 INFO:tasks.rados.rados.0.trial178.stdout:3234: writing trial17818531-4 from 2167533 to 2922923 tid 2 2026-02-21T22:29:20.226 INFO:tasks.rados.rados.0.trial178.stdout:3235: write oid 42 current snap is 325 2026-02-21T22:29:20.226 INFO:tasks.rados.rados.0.trial178.stdout:3235: seq_num 1136 ranges {692752=419461,1826015=615401,2979461=113291} 2026-02-21T22:29:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3235: writing trial17818531-42 from 692752 to 1112213 tid 1 2026-02-21T22:29:20.242 INFO:tasks.rados.rados.0.trial178.stdout:3235: writing trial17818531-42 from 1826015 to 2441416 tid 2 2026-02-21T22:29:20.247 INFO:tasks.rados.rados.0.trial178.stdout:3235: writing trial17818531-42 from 2979461 to 3092752 tid 3 2026-02-21T22:29:20.247 INFO:tasks.rados.rados.0.trial178.stdout:3236: write (excl) oid 21 current snap is 325 2026-02-21T22:29:20.248 INFO:tasks.rados.rados.0.trial178.stdout:3236: seq_num 1137 ranges {709559=494710,1884497=759059} 2026-02-21T22:29:20.253 INFO:tasks.rados.rados.0.trial178.stdout:3236: writing trial17818531-21 from 709559 to 1204269 tid 1 2026-02-21T22:29:20.262 INFO:tasks.rados.rados.0.trial178.stdout:3236: writing trial17818531-21 from 1884497 to 2643556 tid 2 2026-02-21T22:29:20.262 INFO:tasks.rados.rados.0.trial178.stdout:3237: copy_from oid 12 from oid 8 current snap is 325 2026-02-21T22:29:20.262 INFO:tasks.rados.rados.0.trial178.stdout:3232: finishing write tid 1 to trial17818531-40 2026-02-21T22:29:20.262 INFO:tasks.rados.rados.0.trial178.stdout:3232: finishing write tid 2 to trial17818531-40 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: finishing write tid 3 to trial17818531-40 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: finishing write tid 4 to trial17818531-40 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: oid 40 updating version 0 to 1614 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: oid 40 version 1614 is already newer than 1613 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: oid 40 version 1614 is already newer than 1612 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1614 (ObjNum 1134 snap 325 seq_num 1134) dirty exists 2026-02-21T22:29:20.263 INFO:tasks.rados.rados.0.trial178.stdout:3232: left oid 40 (ObjNum 1134 snap 325 seq_num 1134) 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:3234: finishing write tid 1 to trial17818531-4 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:3231: done (5 left) 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:3232: done (4 left) 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:3238: rollback oid 38 current snap is 325 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 324 2026-02-21T22:29:20.264 INFO:tasks.rados.rados.0.trial178.stdout:3234: finishing write tid 2 to trial17818531-4 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: finishing write tid 3 to trial17818531-4 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: finishing write tid 4 to trial17818531-4 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: oid 4 updating version 0 to 1609 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: oid 4 version 1609 is already newer than 1608 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: oid 4 version 1609 is already newer than 1607 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1609 (ObjNum 1135 snap 325 seq_num 1135) dirty exists 2026-02-21T22:29:20.265 INFO:tasks.rados.rados.0.trial178.stdout:3234: left oid 4 (ObjNum 1135 snap 325 seq_num 1135) 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3235: finishing write tid 1 to trial17818531-42 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3234: done (4 left) 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3239: copy_from oid 3 from oid 13 current snap is 325 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3235: finishing write tid 2 to trial17818531-42 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3235: finishing write tid 3 to trial17818531-42 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3235: finishing write tid 4 to trial17818531-42 2026-02-21T22:29:20.266 INFO:tasks.rados.rados.0.trial178.stdout:3235: finishing write tid 5 to trial17818531-42 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3235: oid 42 updating version 0 to 1766 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3235: oid 42 updating version 1766 to 1768 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3235: oid 42 updating version 1768 to 1769 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3235: oid 42 version 1769 is already newer than 1767 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1769 (ObjNum 1136 snap 325 seq_num 1136) dirty exists 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3235: left oid 42 (ObjNum 1136 snap 325 seq_num 1136) 2026-02-21T22:29:20.267 INFO:tasks.rados.rados.0.trial178.stdout:3236: finishing write tid 1 to trial17818531-21 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3235: done (4 left) 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3240: delete oid 18 current snap is 325 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3240: done (4 left) 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3241: copy_from oid 11 from oid 15 current snap is 325 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3242: write oid 18 current snap is 325 2026-02-21T22:29:20.268 INFO:tasks.rados.rados.0.trial178.stdout:3242: seq_num 1139 ranges {743173=645208,1885108=762728} 2026-02-21T22:29:20.269 INFO:tasks.rados.rados.0.trial178.stdout:3242: writing trial17818531-18 from 743173 to 1388381 tid 1 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:3242: writing trial17818531-18 from 1885108 to 2647836 tid 2 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:3243: rollback oid 16 current snap is 325 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 322 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:3238: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:3244: read oid 50 snap -1 2026-02-21T22:29:20.273 INFO:tasks.rados.rados.0.trial178.stdout:3244: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3245: read oid 25 snap -1 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3245: expect (ObjNum 1121 snap 321 seq_num 1121) 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3246: read oid 27 snap 322 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3246: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3247: read oid 10 snap 322 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3247: expect deleted 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3248: snap_create 2026-02-21T22:29:20.274 INFO:tasks.rados.rados.0.trial178.stdout:3238: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3238: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1200 (ObjNum 1102 snap 314 seq_num 1102) dirty exists 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: finishing write tid 2 to trial17818531-21 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: finishing write tid 3 to trial17818531-21 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: finishing write tid 4 to trial17818531-21 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: oid 21 updating version 0 to 1773 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: oid 21 version 1773 is already newer than 1772 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:3236: oid 21 version 1773 is already newer than 1771 2026-02-21T22:29:20.275 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1773 (ObjNum 1137 snap 325 seq_num 1137) dirty exists 2026-02-21T22:29:20.276 INFO:tasks.rados.rados.0.trial178.stdout:3236: left oid 21 (ObjNum 1137 snap 325 seq_num 1137) 2026-02-21T22:29:20.276 INFO:tasks.rados.rados.0.trial178.stdout:3242: finishing write tid 1 to trial17818531-18 2026-02-21T22:29:20.277 INFO:tasks.rados.rados.0.trial178.stdout:3237: finishing copy_from racing read to trial17818531-12 2026-02-21T22:29:20.277 INFO:tasks.rados.rados.0.trial178.stdout:3241: finishing copy_from racing read to trial17818531-11 2026-02-21T22:29:20.277 INFO:tasks.rados.rados.0.trial178.stdout:3239: finishing copy_from racing read to trial17818531-3 2026-02-21T22:29:20.277 INFO:tasks.rados.rados.0.trial178.stdout:3237: finishing copy_from to trial17818531-12 2026-02-21T22:29:20.278 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1473 (ObjNum 1101 snap 314 seq_num 1101) dirty exists 2026-02-21T22:29:20.278 INFO:tasks.rados.rados.0.trial178.stdout:3243: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:29:20.278 INFO:tasks.rados.rados.0.trial178.stdout:3243: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:29:20.278 INFO:tasks.rados.rados.0.trial178.stdout:3243: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:29:20.278 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1612 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:29:20.279 INFO:tasks.rados.rados.0.trial178.stdout:3239: finishing copy_from to trial17818531-3 2026-02-21T22:29:20.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1546 (ObjNum 1124 snap 321 seq_num 1124) dirty exists 2026-02-21T22:29:20.279 INFO:tasks.rados.rados.0.trial178.stdout:3242: finishing write tid 2 to trial17818531-18 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: finishing write tid 3 to trial17818531-18 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: finishing write tid 4 to trial17818531-18 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: oid 18 updating version 0 to 1594 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: oid 18 version 1594 is already newer than 1593 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: oid 18 updating version 1594 to 1595 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1595 (ObjNum 1139 snap 325 seq_num 1139) dirty exists 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3242: left oid 18 (ObjNum 1139 snap 325 seq_num 1139) 2026-02-21T22:29:20.280 INFO:tasks.rados.rados.0.trial178.stdout:3244: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:29:20.283 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 709838 2026-02-21T22:29:20.318 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:20.367 INFO:tasks.rados.rados.0.trial178.stdout:3241: finishing copy_from to trial17818531-11 2026-02-21T22:29:20.367 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1235 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:29:20.367 INFO:tasks.rados.rados.0.trial178.stdout:3245: expect (ObjNum 1121 snap 321 seq_num 1121) 2026-02-21T22:29:20.481 INFO:tasks.rados.rados.0.trial178.stdout:3246: expect (ObjNum 1068 snap 310 seq_num 1068) 2026-02-21T22:29:20.489 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1183507 2026-02-21T22:29:20.547 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:20.587 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:29:20.587 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:29:20.587 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:29:20.588 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:29:20.590 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:29:20.590 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:29:20.590 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:29:20.618 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: get_monmap_and_config 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: build_initial_monmap 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: monmap: 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:20.682076+0000 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:20.682076+0000 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:20.681 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 1 Processor -- start 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 1 -- start start 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: init 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:20.682 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-a con 0x7f3e7c08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-b con 0x7f3e7c08b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-c con 0x7f3e7c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c086a10 con 0x7f3e7c08ce00 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c08b130 con 0x7f3e7c08d8e0 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c08bd60 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e83f67640 10 monclient(hunting): authenticate will time out at 1969.918171s 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e81cdc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08ce00 0x7f3e7c08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50220/0 (socket says 10.20.193.102:50220) 2026-02-21T22:29:20.683 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.677+0000 7f3e81cdc640 1 -- 10.20.193.102:0/1916992131 learned_addr learned my addr 10.20.193.102:0/1916992131 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:20.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.681+0000 7f3e81cdc640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.681+0000 7f3e814db640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.684 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.681+0000 7f3e824dd640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e7c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787497983 0 0) 0x7f3e7c086a10 con 0x7f3e7c08ce00 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient(hunting): my global_id is 34665 2026-02-21T22:29:20.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e64003620 con 0x7f3e7c08ce00 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3750478139 0 0) 0x7f3e64003620 con 0x7f3e7c08ce00 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e7c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e7c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: _finish_hunting 0 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 monclient: found mon.noname-a 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c08b370 con 0x7f3e7c08ce00 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: _finish_auth 0 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: _check_auth_tickets 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:20.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3e6c003110 con 0x7f3e7c08ce00 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: dump: 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 5 monclient: authenticate success, global_id 34665 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e6c0033e0 con 0x7f3e7c08ce00 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 1 -- 10.20.193.102:0/1916992131 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3e6c004b50 con 0x7f3e7c08ce00 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e80cda640 10 monclient: dump: 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 4 set_mon_vals no callback set 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 10 monclient: shutdown 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 1 -- 10.20.193.102:0/1916992131 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 1 -- 10.20.193.102:0/1916992131 shutdown_connections 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 1 -- 10.20.193.102:0/1916992131 >> 10.20.193.102:0/1916992131 conn(0x7f3e7c091a50 msgr2=0x7f3e7c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:20.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 1 -- 10.20.193.102:0/1916992131 shutdown_connections 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 1 -- 10.20.193.102:0/1916992131 wait complete. 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 10 monclient: build_initial_monmap 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.709+0000 7f3e83f67640 10 monclient: monmap: 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 1 Processor -- start 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 1 -- start start 2026-02-21T22:29:20.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient: init 2026-02-21T22:29:20.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-a con 0x7f3e7c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-c con 0x7f3e7c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient(hunting): picked mon.noname-b con 0x7f3e7c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c07ee10 con 0x7f3e7c08d8e0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c081170 con 0x7f3e7c08ce00 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e83f67640 10 monclient(hunting): authenticate will time out at 1969.952681s 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e81cdc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 0x7f3e7c0846c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50224/0 (socket says 10.20.193.102:50224) 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e81cdc640 1 -- 10.20.193.102:0/2697167335 learned_addr learned my addr 10.20.193.102:0/2697167335 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e81cdc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e7c0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e814db640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920759537 0 0) 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): my global_id is 34668 2026-02-21T22:29:20.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e4c003620 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e824dd640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e7c193a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e81cdc640 1 --1- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 0x7f3e7c0846c0 :-1 s=OPENED pgs=532 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f3e6c0048e0 mon_map magic: 0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3532810673 0 0) 0x7f3e4c003620 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3e6c0048e0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): dump: 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.713+0000 7f3e6a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:20.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691039774 0 0) 0x7f3e7c081170 con 0x7f3e7c08ce00 2026-02-21T22:29:20.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.730 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.731 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 10 monclient(hunting): my global_id is 94175 2026-02-21T22:29:20.731 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.725+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e4c003620 con 0x7f3e7c08ce00 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756041673 0 0) 0x7f3e7c07ee10 con 0x7f3e7c08d8e0 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): my global_id is 104098 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e7c081170 con 0x7f3e7c08d8e0 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2662709367 0 0) 0x7f3e4c003620 con 0x7f3e7c08ce00 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e7c07ee10 con 0x7f3e7c08ce00 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3e70003270 con 0x7f3e7c08ce00 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): dump: 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.733 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1784756432 0 0) 0x7f3e7c07ee10 con 0x7f3e7c08ce00 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e7c193a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e7c0846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 monclient: found mon.c 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c0808d0 con 0x7f3e7c08ce00 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 512448903 0 0) 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:29:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 5 monclient: authenticate success, global_id 94175 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 monclient: _renew_subs 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c084dd0 con 0x7f3e7c08ce00 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 monclient: _renew_subs 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f3e7c07fb40 con 0x7f3e7c08ce00 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e814db640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e814db640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c1922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e814db640 1 --1- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 0x7f3e7c1922e0 :-1 s=OPENED pgs=433 gs=2 cs=1 l=1).handle_message read tag failed 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e814db640 1 --1- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 0x7f3e7c1922e0 :-1 s=OPENED pgs=433 gs=2 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6790/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e7c08ce00 legacy=0x7f3e7c1922e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): picked mon.a con 0x7f3e7c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): picked mon.c con 0x7f3e4c026a80 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): picked mon.b con 0x7f3e7c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f3e7c07ee10 con 0x7f3e7c08d8e0 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f3e4c003620 con 0x7f3e4c026a80 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e83f67640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e81cdc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e4c0047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e814db640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e4c026a80 legacy=0x7f3e4c046e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460457156 0 0) 0x7f3e7c05e200 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f3e4c06c410 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054031480 0 0) 0x7f3e4c003620 con 0x7f3e4c026a80 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f3e7c05e200 con 0x7f3e4c026a80 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e824dd640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e4c0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 3607962899 0 0) 0x7f3e4c06c410 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6790/0 conn(0x7f3e4c026a80 legacy=0x7f3e4c046e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.178:6789/0 conn(0x7f3e7c08d8e0 legacy=0x7f3e4c0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 monclient: found mon.a 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7f3e4c06d660 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c0008e0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c003620 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f3e6c005170 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.729+0000 7f3e6a7fc640 10 monclient: dump: 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.738 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e6c005310 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f3e6c0119b0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6bfff640 4 set_mon_vals no callback set 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 6 ==== osd_map(673..673 src has 395..673) ==== 3975+0+0 (unknown 2800644831 0 0) 0x7f3e6c04c7c0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e814db640 10 monclient: get_auth_request con 0x7f3e7c08ce00 auth_method 0 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f3e6c017850 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.739 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.733+0000 7f3e6a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:20.741 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.737+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 8 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f3e6c017130 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.741 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.737+0000 7f3e6a7fc640 10 monclient: handle_mon_command_ack 1 not found 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3e7c062ce0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e6a7fc640 1 -- 10.20.193.102:0/2697167335 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 1021644238) 0x7f3e6c01c9f0 con 0x7f3e7c08b7f0 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e6a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e6a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:20.852 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.178:6800/2549949439 conn(0x7f3e7c08ce00 legacy=0x7f3e4c0c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 10 monclient: shutdown 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 >> v1:10.20.193.102:6789/0 conn(0x7f3e7c08b7f0 legacy=0x7f3e4c0047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 shutdown_connections 2026-02-21T22:29:20.853 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 >> 10.20.193.102:0/2697167335 conn(0x7f3e7c091a50 msgr2=0x7f3e7c0575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:20.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 shutdown_connections 2026-02-21T22:29:20.854 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.845+0000 7f3e83f67640 1 -- 10.20.193.102:0/2697167335 wait complete. 2026-02-21T22:29:20.860 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:29:20.860 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:29:20.860 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: get_monmap_and_config 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: build_initial_monmap 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: monmap: 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.932 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:20.931975+0000 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:20.931975+0000 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 1 Processor -- start 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 1 -- start start 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: init 2026-02-21T22:29:20.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-c con 0x7fcb3408c4e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-b con 0x7fcb3408d410 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-a con 0x7fcb3408d890 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3408bd50 con 0x7fcb3408d890 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3408c8b0 con 0x7fcb3408c4e0 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb34083560 con 0x7fcb3408d410 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.925+0000 7fcb3a909640 10 monclient(hunting): authenticate will time out at 1970.167907s 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb38e7f640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 0x7fcb341aa1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50254/0 (socket says 10.20.193.102:50254) 2026-02-21T22:29:20.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb38e7f640 1 -- 10.20.193.102:0/4171097287 learned_addr learned my addr 10.20.193.102:0/4171097287 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:20.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb337fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fcb3408d410 0x7fcb34188a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59354/0 (socket says 10.20.193.102:59354) 2026-02-21T22:29:20.942 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb38e7f640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 legacy=0x7fcb341aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb337fe640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.102:6790/0 conn(0x7fcb3408d410 legacy=0x7fcb34188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb33fff640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.178:6789/0 conn(0x7fcb3408c4e0 legacy=0x7fcb3408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775976764 0 0) 0x7fcb3408bd50 con 0x7fcb3408d890 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): my global_id is 34671 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb18003620 con 0x7fcb3408d890 2026-02-21T22:29:20.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323874103 0 0) 0x7fcb24002b90 con 0x7fcb3408d410 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient(hunting): my global_id is 94178 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb3408bd50 con 0x7fcb3408d410 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1134969815 0 0) 0x7fcb18003620 con 0x7fcb3408d890 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.102:6790/0 conn(0x7fcb3408d410 legacy=0x7fcb34188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.178:6789/0 conn(0x7fcb3408c4e0 legacy=0x7fcb3408cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 monclient: found mon.noname-a 2026-02-21T22:29:20.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb34084bc0 con 0x7fcb3408d890 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb28004620 con 0x7fcb3408d890 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: dump: 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.945 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 5 monclient: authenticate success, global_id 34671 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcb280048f0 con 0x7fcb3408d890 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:20.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 1 -- 10.20.193.102:0/4171097287 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb28006060 con 0x7fcb3408d890 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb32ffd640 10 monclient: dump: 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.947 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 4 set_mon_vals no callback set 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 10 monclient: shutdown 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 1 -- 10.20.193.102:0/4171097287 >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 legacy=0x7fcb341aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 1 -- 10.20.193.102:0/4171097287 shutdown_connections 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 1 -- 10.20.193.102:0/4171097287 >> 10.20.193.102:0/4171097287 conn(0x7fcb34091a50 msgr2=0x7fcb34091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 1 -- 10.20.193.102:0/4171097287 shutdown_connections 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 1 -- 10.20.193.102:0/4171097287 wait complete. 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 10 monclient: build_initial_monmap 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.929+0000 7fcb3a909640 10 monclient: monmap: 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:20.948 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 Processor -- start 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- start start 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: init 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:20.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-c con 0x7fcb3408c4e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-b con 0x7fcb3408d410 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient(hunting): picked mon.noname-a con 0x7fcb3408d890 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3405c780 con 0x7fcb3408d890 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3405d860 con 0x7fcb3408d410 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb3405abe0 con 0x7fcb3408c4e0 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient(hunting): authenticate will time out at 1970.172573s 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb38e7f640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 0x7fcb34240fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50264/0 (socket says 10.20.193.102:50264) 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb38e7f640 1 -- 10.20.193.102:0/3676372455 learned_addr learned my addr 10.20.193.102:0/3676372455 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb38e7f640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 legacy=0x7fcb34240fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb33fff640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6790/0 conn(0x7fcb3408c4e0 legacy=0x7fcb34059dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651489497 0 0) 0x7fcb3405c780 con 0x7fcb3408d890 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): my global_id is 34674 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb04003620 con 0x7fcb3408d890 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721252893 0 0) 0x7fcb3405abe0 con 0x7fcb3408c4e0 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): my global_id is 94181 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb3405c780 con 0x7fcb3408c4e0 2026-02-21T22:29:20.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb337fe640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.178:6789/0 conn(0x7fcb3408d410 legacy=0x7fcb3405a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3174867132 0 0) 0x7fcb04003620 con 0x7fcb3408d890 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb3405abe0 con 0x7fcb3408d890 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb28004080 con 0x7fcb3408d890 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.952 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3434335755 0 0) 0x7fcb3405c780 con 0x7fcb3408c4e0 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb04003620 con 0x7fcb3408c4e0 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb20003380 con 0x7fcb3408c4e0 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.953 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 570237552 0 0) 0x7fcb3405abe0 con 0x7fcb3408d890 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6790/0 conn(0x7fcb3408c4e0 legacy=0x7fcb34059dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.178:6789/0 conn(0x7fcb3408d410 legacy=0x7fcb3405a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:20.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 monclient: found mon.a 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3405db40 con 0x7fcb3408d890 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 5 monclient: authenticate success, global_id 34674 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _renew_subs 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3405b7e0 con 0x7fcb3408d890 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _renew_subs 2026-02-21T22:29:20.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fcb3405e2c0 con 0x7fcb3408d890 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcb28006070 con 0x7fcb3408d890 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb28006830 con 0x7fcb3408d890 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb327fc640 4 set_mon_vals no callback set 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 10 monclient: dump: 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.956 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fcb3405cf10 con 0x7fcb341b1ac0 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fcb28012ec0 con 0x7fcb3408d890 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.933+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(673..673 src has 395..673) ==== 3975+0+0 (unknown 2800644831 0 0) 0x7fcb2804daf0 con 0x7fcb3408d890 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb337fe640 10 monclient: get_auth_request con 0x7fcb0403d6f0 auth_method 0 2026-02-21T22:29:20.957 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb33fff640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b1ac0 legacy=0x7fcb341b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299231723 0 0) 0x7fcb3405cf10 con 0x7fcb341b1ac0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient(hunting): my global_id is 34677 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb3405abe0 con 0x7fcb341b1ac0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1919242738 0 0) 0x7fcb3405abe0 con 0x7fcb341b1ac0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb3405cf10 con 0x7fcb341b1ac0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb20003b30 con 0x7fcb341b1ac0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient: dump: 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.958 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 741344770 0 0) 0x7fcb3405cf10 con 0x7fcb341b1ac0 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb34086010 con 0x7fcb341b1ac0 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fcb200058c0 con 0x7fcb341b1ac0 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:20.959 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:20.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b1ac0 legacy=0x7fcb341b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.960 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.937+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b1ac0 legacy=0x7fcb341b1e90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:20.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.941+0000 7fcb3a909640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:20.961 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.941+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fcb34062ce0 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb33fff640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b25a0 legacy=0x7fcb34263700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934449785 0 0) 0x7fcb34062ce0 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient(hunting): my global_id is 34680 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb3405cf10 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2774067724 0 0) 0x7fcb3405cf10 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb34062ce0 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fcb200040b0 con 0x7fcb341b25a0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:20.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient: dump: 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1747677295 0 0) 0x7fcb34062ce0 con 0x7fcb341b25a0 2026-02-21T22:29:20.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcb341b2970 con 0x7fcb341b25a0 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3589136460) 0x7fcb200044a0 con 0x7fcb341b25a0 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b25a0 legacy=0x7fcb34263700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb30ff9640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb341b25a0 legacy=0x7fcb34263700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:20.964 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 709814, 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:20.965 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:20.966 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.967 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:20.968 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:20.969 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:20.970 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.971 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:20.972 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:20.973 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.974 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:20.975 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:20.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.178:6800/2549949439 conn(0x7fcb0403d6f0 legacy=0x7fcb0405dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 10 monclient: shutdown 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 >> v1:10.20.193.102:6789/0 conn(0x7fcb3408d890 legacy=0x7fcb34240fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb38e7f640 1 -- 10.20.193.102:0/3676372455 reap_dead start 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 shutdown_connections 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 >> 10.20.193.102:0/3676372455 conn(0x7fcb34091a50 msgr2=0x7fcb34088180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 shutdown_connections 2026-02-21T22:29:20.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:20.945+0000 7fcb3a909640 1 -- 10.20.193.102:0/3676372455 wait complete. 2026-02-21T22:29:20.977 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:20.977 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:29:20.978 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:29:21.047 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: get_monmap_and_config 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: monmap: 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:21.047392+0000 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:21.047392+0000 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.048 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 1 Processor -- start 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 1 -- start start 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: init 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-b con 0x7fe37408e6c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.049 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-c con 0x7fe374059100 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-a con 0x7fe37408f1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3740904a0 con 0x7fe37408f1a0 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe374090260 con 0x7fe374059100 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe374058820 con 0x7fe37408e6c0 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe37a67b640 10 monclient(hunting): authenticate will time out at 1970.283291s 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe378bf1640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fe37408f1a0 0x7fe3741a5870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50292/0 (socket says 10.20.193.102:50292) 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe373fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fe37408e6c0 0x7fe37408ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59376/0 (socket says 10.20.193.102:59376) 2026-02-21T22:29:21.050 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.041+0000 7fe378bf1640 1 -- 10.20.193.102:0/3816013162 learned_addr learned my addr 10.20.193.102:0/3816013162 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe373fff640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.102:6790/0 conn(0x7fe37408e6c0 legacy=0x7fe37408ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe378bf1640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.102:6789/0 conn(0x7fe37408f1a0 legacy=0x7fe3741a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603659921 0 0) 0x7fe3740904a0 con 0x7fe37408f1a0 2026-02-21T22:29:21.057 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): my global_id is 34683 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe358003620 con 0x7fe37408f1a0 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650123103 0 0) 0x7fe374058820 con 0x7fe37408e6c0 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient(hunting): my global_id is 94184 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe3737fe640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.178:6789/0 conn(0x7fe374059100 legacy=0x7fe374184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3740904a0 con 0x7fe37408e6c0 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 569155518 0 0) 0x7fe358003620 con 0x7fe37408f1a0 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.102:6790/0 conn(0x7fe37408e6c0 legacy=0x7fe37408ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.178:6789/0 conn(0x7fe374059100 legacy=0x7fe374184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 monclient: found mon.noname-a 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe374057f80 con 0x7fe37408f1a0 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.060 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe368004550 con 0x7fe37408f1a0 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: dump: 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.061 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 5 monclient: authenticate success, global_id 34683 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:21.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe368004820 con 0x7fe37408f1a0 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 1 -- 10.20.193.102:0/3816013162 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe368005eb0 con 0x7fe37408f1a0 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe372ffd640 10 monclient: dump: 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.063 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.064 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 4 set_mon_vals no callback set 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: shutdown 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- 10.20.193.102:0/3816013162 >> v1:10.20.193.102:6789/0 conn(0x7fe37408f1a0 legacy=0x7fe3741a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- 10.20.193.102:0/3816013162 shutdown_connections 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- 10.20.193.102:0/3816013162 >> 10.20.193.102:0/3816013162 conn(0x7fe374056b30 msgr2=0x7fe374093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- 10.20.193.102:0/3816013162 shutdown_connections 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- 10.20.193.102:0/3816013162 wait complete. 2026-02-21T22:29:21.065 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: monmap: 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:21.066 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 Processor -- start 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- start start 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: init 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.067 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-b con 0x7fe374059100 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-a con 0x7fe37408e6c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient(hunting): picked mon.noname-c con 0x7fe37408f1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37412e000 con 0x7fe37408e6c0 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37405ea30 con 0x7fe374059100 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe374143430 con 0x7fe37408f1a0 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.068 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.045+0000 7fe37a67b640 10 monclient(hunting): authenticate will time out at 1970.287953s 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe378bf1640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fe37408f1a0 0x7fe374195e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59382/0 (socket says 10.20.193.102:59382) 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe378bf1640 1 -- 10.20.193.102:0/1393534048 learned_addr learned my addr 10.20.193.102:0/1393534048 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe378bf1640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe37408f1a0 legacy=0x7fe374195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe3737fe640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6789/0 conn(0x7fe37408e6c0 legacy=0x7fe374194770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934910013 0 0) 0x7fe374143430 con 0x7fe37408f1a0 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.069 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): my global_id is 94187 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe344003620 con 0x7fe37408f1a0 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3089328459 0 0) 0x7fe37412e000 con 0x7fe37408e6c0 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): my global_id is 34686 2026-02-21T22:29:21.070 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe374143430 con 0x7fe37408e6c0 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe373fff640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.178:6789/0 conn(0x7fe374059100 legacy=0x7fe37413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 741387601 0 0) 0x7fe344003620 con 0x7fe37408f1a0 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe37412e000 con 0x7fe37408f1a0 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe368003fb0 con 0x7fe37408f1a0 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:21.071 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.072 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 683847042 0 0) 0x7fe374143430 con 0x7fe37408e6c0 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe344003620 con 0x7fe37408e6c0 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe364003350 con 0x7fe37408e6c0 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.073 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.074 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101843590 0 0) 0x7fe37405ea30 con 0x7fe374059100 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient(hunting): my global_id is 104110 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe374143430 con 0x7fe374059100 2026-02-21T22:29:21.075 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4002033013 0 0) 0x7fe37412e000 con 0x7fe37408f1a0 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.178:6789/0 conn(0x7fe374059100 legacy=0x7fe37413c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6789/0 conn(0x7fe37408e6c0 legacy=0x7fe374194770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 monclient: found mon.c 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741400d0 con 0x7fe37408f1a0 2026-02-21T22:29:21.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 5 monclient: authenticate success, global_id 94187 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 10 monclient: _renew_subs 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe37412dd20 con 0x7fe37408f1a0 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 10 monclient: _renew_subs 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:21.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe374153670 con 0x7fe37408f1a0 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe3680047c0 con 0x7fe37408f1a0 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe368006680 con 0x7fe37408f1a0 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 10 monclient: dump: 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.078 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe3727fc640 4 set_mon_vals no callback set 2026-02-21T22:29:21.079 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe374153960 con 0x7fe374092c10 2026-02-21T22:29:21.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.049+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fe368012d10 con 0x7fe37408f1a0 2026-02-21T22:29:21.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(673..673 src has 395..673) ==== 3975+0+0 (unknown 2800644831 0 0) 0x7fe36804da90 con 0x7fe37408f1a0 2026-02-21T22:29:21.080 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe3737fe640 10 monclient: get_auth_request con 0x7fe34403d850 auth_method 0 2026-02-21T22:29:21.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe373fff640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe374092c10 legacy=0x7fe374089f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950050943 0 0) 0x7fe374153960 con 0x7fe374092c10 2026-02-21T22:29:21.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient(hunting): my global_id is 94190 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe37412e000 con 0x7fe374092c10 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2152683117 0 0) 0x7fe37412e000 con 0x7fe374092c10 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe374153960 con 0x7fe374092c10 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe360003270 con 0x7fe374092c10 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient: dump: 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.084 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3995872027 0 0) 0x7fe374153960 con 0x7fe374092c10 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3740008e0 con 0x7fe374092c10 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fe3600048e0 con 0x7fe374092c10 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe374092c10 legacy=0x7fe374089f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.053+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe374092c10 legacy=0x7fe374089f60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.057+0000 7fe37a67b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.057+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe37404d990 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe373fff640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe37413cd10 legacy=0x7fe37404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614979919 0 0) 0x7fe37404d990 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient(hunting): my global_id is 94193 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe374153960 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2683648845 0 0) 0x7fe374153960 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe37404d990 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fe360004ee0 con 0x7fe37413cd10 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:21.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient: dump: 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4149751375 0 0) 0x7fe37404d990 con 0x7fe37413cd10 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe3740949d0 con 0x7fe37413cd10 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 2727079362) 0x7fe3600091b0 con 0x7fe37413cd10 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:21.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe37413cd10 legacy=0x7fe37404d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe370ff9640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe37413cd10 legacy=0x7fe37404d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 107336, 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:21.088 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.089 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.090 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:21.091 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.092 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:21.093 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.094 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.095 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:21.096 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:21.097 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.178:6800/2549949439 conn(0x7fe34403d850 legacy=0x7fe34405dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 10 monclient: shutdown 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 >> v1:10.20.193.102:6790/0 conn(0x7fe37408f1a0 legacy=0x7fe374195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe378bf1640 1 -- 10.20.193.102:0/1393534048 reap_dead start 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe378bf1640 1 -- 10.20.193.102:0/1393534048 reap_dead start 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 shutdown_connections 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 >> 10.20.193.102:0/1393534048 conn(0x7fe374056b30 msgr2=0x7fe374141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 shutdown_connections 2026-02-21T22:29:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.061+0000 7fe37a67b640 1 -- 10.20.193.102:0/1393534048 wait complete. 2026-02-21T22:29:21.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:21.099 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:29:21.099 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: get_monmap_and_config 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: monmap: 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:21.158624+0000 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:21.158624+0000 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.159 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 1 Processor -- start 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 1 -- start start 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: init 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-c con 0x7f9d4808c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-a con 0x7f9d4808cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-b con 0x7f9d48183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d4808d780 con 0x7f9d4808cc80 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d48085da0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d48085420 con 0x7f9d48183480 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4df79640 10 monclient(hunting): authenticate will time out at 1970.394541s 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d3ffff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9d4808cc80 0x7f9d48181d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50306/0 (socket says 10.20.193.102:50306) 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d3ffff640 1 -- 10.20.193.102:0/408083568 learned_addr learned my addr 10.20.193.102:0/408083568 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.153+0000 7f9d4d778640 1 --1- 10.20.193.102:0/408083568 >> v1:10.20.193.102:6790/0 conn(0x7f9d48183480 0x7f9d481a3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59404/0 (socket says 10.20.193.102:59404) 2026-02-21T22:29:21.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4d778640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.102:6790/0 conn(0x7f9d48183480 legacy=0x7f9d481a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3ffff640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.102:6789/0 conn(0x7f9d4808cc80 legacy=0x7f9d48181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4cf77640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.178:6789/0 conn(0x7f9d4808c1a0 legacy=0x7f9d4808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044262458 0 0) 0x7f9d48085420 con 0x7f9d48183480 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): my global_id is 94196 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d34003620 con 0x7f9d48183480 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325696448 0 0) 0x7f9d30002c60 con 0x7f9d4808cc80 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient(hunting): my global_id is 34689 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d48085420 con 0x7f9d4808cc80 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2605517653 0 0) 0x7f9d48085420 con 0x7f9d4808cc80 2026-02-21T22:29:21.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.102:6790/0 conn(0x7f9d48183480 legacy=0x7f9d481a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.178:6789/0 conn(0x7f9d4808c1a0 legacy=0x7f9d4808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 monclient: found mon.noname-a 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d48086640 con 0x7f9d4808cc80 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d300032e0 con 0x7f9d4808cc80 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 5 monclient: authenticate success, global_id 34689 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:21.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: dump: 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9d300049a0 con 0x7f9d4808cc80 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 1 -- 10.20.193.102:0/408083568 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d30004d30 con 0x7f9d4808cc80 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d3f7fe640 10 monclient: dump: 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 4 set_mon_vals no callback set 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: shutdown 2026-02-21T22:29:21.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- 10.20.193.102:0/408083568 >> v1:10.20.193.102:6789/0 conn(0x7f9d4808cc80 legacy=0x7f9d48181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- 10.20.193.102:0/408083568 shutdown_connections 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- 10.20.193.102:0/408083568 >> 10.20.193.102:0/408083568 conn(0x7f9d4808f6a0 msgr2=0x7f9d4808faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- 10.20.193.102:0/408083568 shutdown_connections 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- 10.20.193.102:0/408083568 wait complete. 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: monmap: 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:21.169 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 Processor -- start 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- start start 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: init 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-a con 0x7f9d4808c1a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-c con 0x7f9d4808cc80 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient(hunting): picked mon.noname-b con 0x7f9d48183480 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d48088990 con 0x7f9d4808c1a0 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480830d0 con 0x7f9d48183480 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480833b0 con 0x7f9d4808cc80 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.170 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4df79640 10 monclient(hunting): authenticate will time out at 1970.399161s 2026-02-21T22:29:21.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4cf77640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9d4808c1a0 0x7f9d48152fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50314/0 (socket says 10.20.193.102:50314) 2026-02-21T22:29:21.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.157+0000 7f9d4cf77640 1 -- 10.20.193.102:0/2893748863 learned_addr learned my addr 10.20.193.102:0/2893748863 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4cf77640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.102:6789/0 conn(0x7f9d4808c1a0 legacy=0x7f9d48152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3ffff640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.102:6790/0 conn(0x7f9d4808cc80 legacy=0x7f9d480848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335001855 0 0) 0x7f9d38002e00 con 0x7f9d4808c1a0 2026-02-21T22:29:21.174 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): my global_id is 34692 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d18003620 con 0x7f9d4808c1a0 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789292751 0 0) 0x7f9d480833b0 con 0x7f9d4808cc80 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): my global_id is 94199 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d38002e00 con 0x7f9d4808cc80 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4d778640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d48183480 legacy=0x7f9d48175340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2604083736 0 0) 0x7f9d18003620 con 0x7f9d4808c1a0 2026-02-21T22:29:21.175 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d480833b0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d38003710 con 0x7f9d4808c1a0 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): dump: 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.176 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2880834448 0 0) 0x7f9d38002e00 con 0x7f9d4808cc80 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d18003620 con 0x7f9d4808cc80 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d30002b30 con 0x7f9d4808cc80 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient(hunting): dump: 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.177 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2413067662 0 0) 0x7f9d480833b0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.102:6790/0 conn(0x7f9d4808cc80 legacy=0x7f9d480848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d48183480 legacy=0x7f9d48175340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 monclient: found mon.a 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4805deb0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 5 monclient: authenticate success, global_id 34692 2026-02-21T22:29:21.178 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 10 monclient: _renew_subs 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9d48081bb0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 10 monclient: _renew_subs 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9d480851a0 con 0x7f9d4808c1a0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9d38003d30 con 0x7f9d4808c1a0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d38004090 con 0x7f9d4808c1a0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 10 monclient: dump: 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.179 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3effd640 4 set_mon_vals no callback set 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9d480008e0 con 0x7f9d4805b630 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9d38004230 con 0x7f9d4808c1a0 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(673..673 src has 395..673) ==== 3975+0+0 (unknown 2800644831 0 0) 0x7f9d3804c960 con 0x7f9d4808c1a0 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.161+0000 7f9d4d778640 10 monclient: get_auth_request con 0x7f9d1803d610 auth_method 0 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3ffff640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805b630 legacy=0x7f9d4805ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158462672 0 0) 0x7f9d480008e0 con 0x7f9d4805b630 2026-02-21T22:29:21.180 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient(hunting): my global_id is 104119 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d480833b0 con 0x7f9d4805b630 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 915587743 0 0) 0x7f9d480833b0 con 0x7f9d4805b630 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d480008e0 con 0x7f9d4805b630 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d300031f0 con 0x7f9d4805b630 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient: dump: 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.181 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3010011138 0 0) 0x7f9d480008e0 con 0x7f9d4805b630 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d480857a0 con 0x7f9d4805b630 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f9d300050b0 con 0x7f9d4805b630 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805b630 legacy=0x7f9d4805ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.165+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805b630 legacy=0x7f9d4805ba00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.169+0000 7f9d4df79640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:21.182 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.169+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9d4805c510 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3ffff640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805c110 legacy=0x7f9d4817f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287589651 0 0) 0x7f9d4805c510 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient(hunting): my global_id is 104122 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d480008e0 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1131711840 0 0) 0x7f9d480008e0 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d4805c510 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d30005780 con 0x7f9d4805c110 2026-02-21T22:29:21.184 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient: dump: 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 926849823 0 0) 0x7f9d4805c510 con 0x7f9d4805c110 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d48054480 con 0x7f9d4805c110 2026-02-21T22:29:21.185 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4344 (unknown 0 0 2284458700) 0x7f9d30005a50 con 0x7f9d4805c110 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805c110 legacy=0x7f9d4817f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d3d7fa640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6789/0 conn(0x7f9d4805c110 legacy=0x7f9d4817f0c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:21.186 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 547, 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.187 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:21.188 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:21.189 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:21.190 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.191 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.192 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.193 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:21.194 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:21.195 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:21.196 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:21.196 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:21.196 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:21.196 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:21.196 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9d1803d610 legacy=0x7f9d1805dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 10 monclient: shutdown 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 >> v1:10.20.193.102:6789/0 conn(0x7f9d4808c1a0 legacy=0x7f9d48152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4d778640 1 -- 10.20.193.102:0/2893748863 reap_dead start 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4d778640 1 -- 10.20.193.102:0/2893748863 reap_dead start 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 shutdown_connections 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 >> 10.20.193.102:0/2893748863 conn(0x7f9d4808f6a0 msgr2=0x7f9d480574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 shutdown_connections 2026-02-21T22:29:21.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.173+0000 7f9d4df79640 1 -- 10.20.193.102:0/2893748863 wait complete. 2026-02-21T22:29:21.198 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:21.198 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:29:21.198 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3236: done (11 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3237: done (10 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3238: done (9 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3239: done (8 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3241: done (7 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3242: done (6 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3243: done (5 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3244: done (4 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3245: done (3 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3246: done (2 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3247: done (1 left) 2026-02-21T22:29:21.207 INFO:tasks.rados.rados.0.trial178.stdout:3248: done (0 left) 2026-02-21T22:29:21.208 INFO:tasks.rados.rados.0.trial178.stdout:3249: copy_from oid 30 from oid 27 current snap is 326 2026-02-21T22:29:21.208 INFO:tasks.rados.rados.0.trial178.stdout:3250: delete oid 13 current snap is 326 2026-02-21T22:29:21.211 INFO:tasks.rados.rados.0.trial178.stdout:3249: finishing copy_from to trial17818531-30 2026-02-21T22:29:21.211 INFO:tasks.rados.rados.0.trial178.stdout:3249: got expected ENOENT (src dne) 2026-02-21T22:29:21.211 INFO:tasks.rados.rados.0.trial178.stdout:3249: finishing copy_from racing read to trial17818531-30 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3249: done (1 left) 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3250: done (0 left) 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3251: read oid 36 snap -1 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3251: expect deleted 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3252: rollback oid 48 current snap is 326 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 325 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3253: write oid 25 current snap is 326 2026-02-21T22:29:21.213 INFO:tasks.rados.rados.0.trial178.stdout:3253: seq_num 1141 ranges {776787=512059,1718425=766397,3000223=176564} 2026-02-21T22:29:21.221 INFO:tasks.rados.rados.0.trial178.stdout:3253: writing trial17818531-25 from 776787 to 1288846 tid 1 2026-02-21T22:29:21.230 INFO:tasks.rados.rados.0.trial178.stdout:3253: writing trial17818531-25 from 1718425 to 2484822 tid 2 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3253: writing trial17818531-25 from 3000223 to 3176787 tid 3 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3254: delete oid 2 current snap is 326 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3252: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3252: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3252: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1483 (ObjNum 1119 snap 320 seq_num 1119) dirty exists 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3253: finishing write tid 1 to trial17818531-25 2026-02-21T22:29:21.235 INFO:tasks.rados.rados.0.trial178.stdout:3253: finishing write tid 2 to trial17818531-25 2026-02-21T22:29:21.238 INFO:tasks.rados.rados.0.trial178.stdout:3253: finishing write tid 3 to trial17818531-25 2026-02-21T22:29:21.238 INFO:tasks.rados.rados.0.trial178.stdout:3253: finishing write tid 4 to trial17818531-25 2026-02-21T22:29:21.238 INFO:tasks.rados.rados.0.trial178.stdout:3253: finishing write tid 5 to trial17818531-25 2026-02-21T22:29:21.238 INFO:tasks.rados.rados.0.trial178.stdout:3253: oid 25 updating version 0 to 1469 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3253: oid 25 updating version 1469 to 1472 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3253: oid 25 version 1472 is already newer than 1470 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3253: oid 25 version 1472 is already newer than 1471 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1472 (ObjNum 1141 snap 326 seq_num 1141) dirty exists 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3253: left oid 25 (ObjNum 1141 snap 326 seq_num 1141) 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3251: done (3 left) 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3252: done (2 left) 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3253: done (1 left) 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3254: done (0 left) 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3255: write oid 26 current snap is 326 2026-02-21T22:29:21.239 INFO:tasks.rados.rados.0.trial178.stdout:3255: seq_num 1143 ranges {410401=662557,1719036=770066,2938716=271685} 2026-02-21T22:29:21.245 INFO:tasks.rados.rados.0.trial178.stdout:3255: writing trial17818531-26 from 410401 to 1072958 tid 1 2026-02-21T22:29:21.254 INFO:tasks.rados.rados.0.trial178.stdout:3255: writing trial17818531-26 from 1719036 to 2489102 tid 2 2026-02-21T22:29:21.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: get_monmap_and_config 2026-02-21T22:29:21.258 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: monmap: 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:21.259689+0000 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:21.259689+0000 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 1 Processor -- start 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 1 -- start start 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: init 2026-02-21T22:29:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-a con 0x7f019413f750 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-c con 0x7f0194058250 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-b con 0x7f01941852c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0194140230 con 0x7f019413f750 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01940565a0 con 0x7f0194058250 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0194058700 con 0x7f01941852c0 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.253+0000 7f0199c05640 10 monclient(hunting): authenticate will time out at 1970.495991s 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0198c03640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f019413f750 0x7f019413fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:50324/0 (socket says 10.20.193.102:50324) 2026-02-21T22:29:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0198c03640 1 -- 10.20.193.102:0/2058809204 learned_addr learned my addr 10.20.193.102:0/2058809204 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.260 INFO:tasks.rados.rados.0.trial178.stdout:3255: writing trial17818531-26 from 2938716 to 3210401 tid 3 2026-02-21T22:29:21.260 INFO:tasks.rados.rados.0.trial178.stdout:3256: delete oid 28 current snap is 326 2026-02-21T22:29:21.260 INFO:tasks.rados.rados.0.trial178.stdout:3255: finishing write tid 1 to trial17818531-26 2026-02-21T22:29:21.260 INFO:tasks.rados.rados.0.trial178.stdout:3255: finishing write tid 2 to trial17818531-26 2026-02-21T22:29:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0198c03640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.102:6789/0 conn(0x7f019413f750 legacy=0x7f019413fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199404640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.102:6790/0 conn(0x7f01941852c0 legacy=0x7f01941a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018bfff640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.178:6789/0 conn(0x7f0194058250 legacy=0x7f0194183ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388767632 0 0) 0x7f0184002c60 con 0x7f019413f750 2026-02-21T22:29:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): my global_id is 34695 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0180003620 con 0x7f019413f750 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1845563096 0 0) 0x7f0194058700 con 0x7f01941852c0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): my global_id is 94202 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0184002c60 con 0x7f01941852c0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962433838 0 0) 0x7f01940565a0 con 0x7f0194058250 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient(hunting): my global_id is 104125 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0194058700 con 0x7f0194058250 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4112364517 0 0) 0x7f0180003620 con 0x7f019413f750 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.102:6790/0 conn(0x7f01941852c0 legacy=0x7f01941a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.178:6789/0 conn(0x7f0194058250 legacy=0x7f0194183ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 monclient: found mon.noname-a 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0194056fd0 con 0x7f019413f750 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f01840032e0 con 0x7f019413f750 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: dump: 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.263 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 5 monclient: authenticate success, global_id 34695 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f01840049a0 con 0x7f019413f750 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 1 -- 10.20.193.102:0/2058809204 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0184004d30 con 0x7f019413f750 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f018b7fe640 10 monclient: dump: 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:21.264 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 4 set_mon_vals no callback set 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: finishing write tid 3 to trial17818531-26 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: finishing write tid 4 to trial17818531-26 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: finishing write tid 5 to trial17818531-26 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: oid 26 updating version 0 to 1220 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: oid 26 updating version 1220 to 1221 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: oid 26 version 1221 is already newer than 1219 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: oid 26 updating version 1221 to 1222 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1222 (ObjNum 1143 snap 326 seq_num 1143) dirty exists 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: left oid 26 (ObjNum 1143 snap 326 seq_num 1143) 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3255: done (1 left) 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3256: done (0 left) 2026-02-21T22:29:21.265 INFO:tasks.rados.rados.0.trial178.stdout:3257: snap_remove snap 325 2026-02-21T22:29:21.265 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 10 monclient: shutdown 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 -- 10.20.193.102:0/2058809204 >> v1:10.20.193.102:6789/0 conn(0x7f019413f750 legacy=0x7f019413fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 -- 10.20.193.102:0/2058809204 shutdown_connections 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 -- 10.20.193.102:0/2058809204 >> 10.20.193.102:0/2058809204 conn(0x7f019405e210 msgr2=0x7f0194095c70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 -- 10.20.193.102:0/2058809204 shutdown_connections 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 -- 10.20.193.102:0/2058809204 wait complete. 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 10 monclient: build_initial_monmap 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 10 monclient: monmap: 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.257+0000 7f0199c05640 1 Processor -- start 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- start start 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: init 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-b con 0x7f0194058250 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-c con 0x7f019413f750 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient(hunting): picked mon.noname-a con 0x7f01941852c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0194112020 con 0x7f01941852c0 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01941111b0 con 0x7f0194058250 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f019410fcc0 con 0x7f019413f750 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient(hunting): authenticate will time out at 1970.500280s 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f018bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f019413f750 0x7f0194180430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59420/0 (socket says 10.20.193.102:59420) 2026-02-21T22:29:21.266 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f018bfff640 1 -- 10.20.193.102:0/1368554053 learned_addr learned my addr 10.20.193.102:0/1368554053 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:21.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f018bfff640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.102:6790/0 conn(0x7f019413f750 legacy=0x7f0194180430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.267 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199404640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.102:6789/0 conn(0x7f01941852c0 legacy=0x7f0194181b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0198c03640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.178:6789/0 conn(0x7f0194058250 legacy=0x7f019413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 331813132 0 0) 0x7f019410fcc0 con 0x7f019413f750 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): my global_id is 94205 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0164003620 con 0x7f019413f750 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997343885 0 0) 0x7f0194112020 con 0x7f01941852c0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): my global_id is 34698 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f019410fcc0 con 0x7f01941852c0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366365446 0 0) 0x7f01941111b0 con 0x7f0194058250 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _negotiate 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): my global_id is 104128 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0194112020 con 0x7f0194058250 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1664046468 0 0) 0x7f019410fcc0 con 0x7f01941852c0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01941111b0 con 0x7f01941852c0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0190004820 con 0x7f01941852c0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): dump: 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.268 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 802982191 0 0) 0x7f0164003620 con 0x7f019413f750 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f019410fcc0 con 0x7f019413f750 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f017c003490 con 0x7f019413f750 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient(hunting): dump: 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2601023302 0 0) 0x7f01941111b0 con 0x7f01941852c0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.102:6790/0 conn(0x7f019413f750 legacy=0x7f0194180430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.178:6789/0 conn(0x7f0194058250 legacy=0x7f019413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: _finish_hunting 0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 monclient: found mon.a 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0194111780 con 0x7f01941852c0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: _finish_auth 0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: _check_auth_tickets 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 5 monclient: authenticate success, global_id 34698 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _renew_subs 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f019410f670 con 0x7f01941852c0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _renew_subs 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0194110820 con 0x7f01941852c0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0190004ed0 con 0x7f01941852c0 2026-02-21T22:29:21.269 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0190005230 con 0x7f01941852c0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 10 monclient: dump: 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f018affd640 4 set_mon_vals no callback set 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01941a6d40 con 0x7f01941852c0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.261+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0190005490 con 0x7f01941852c0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.265+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(674..674 src has 395..674) ==== 3947+0+0 (unknown 1286230997 0 0) 0x7f019004da30 con 0x7f01941852c0 2026-02-21T22:29:21.270 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.265+0000 7f0198c03640 10 monclient: get_auth_request con 0x7f016403d800 auth_method 0 2026-02-21T22:29:21.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.265+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0190018a20 con 0x7f01941852c0 2026-02-21T22:29:21.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.265+0000 7f01897fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:21.271 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.265+0000 7f01897fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:21.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:29:21.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:29:21.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:21.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0194110b60 con 0x7f01941852c0 2026-02-21T22:29:21.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f01897fa640 1 -- 10.20.193.102:0/1368554053 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f0190018300 con 0x7f01941852c0 2026-02-21T22:29:21.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f01897fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:29:21.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f01897fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:21.395 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.395 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=13,kv=12,logm=53} crc {auth=1647625846,config=1618278320,health=3811746669,kv=514132712,logm=3090151617}) 2026-02-21T22:29:21.395 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=12,kv=9,logm=57} crc {auth=3865182841,config=1618278320,health=4205230687,kv=3791546590,logm=3019174380}) 2026-02-21T22:29:21.395 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.396 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=13,kv=12,logm=53} crc {auth=1647625846,config=1618278320,health=3811746669,kv=514132712,logm=3090151617}) 2026-02-21T22:29:21.396 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=19,config=2,health=13,kv=12,logm=54} crc {auth=2854477124,config=1618278320,health=3811746669,kv=514132712,logm=3065256046}) 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.178:6800/2549949439 conn(0x7f016403d800 legacy=0x7f016405dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 10 monclient: shutdown 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 >> v1:10.20.193.102:6789/0 conn(0x7f01941852c0 legacy=0x7f0194181b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 shutdown_connections 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 >> 10.20.193.102:0/1368554053 conn(0x7f019405e210 msgr2=0x7f01941a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 shutdown_connections 2026-02-21T22:29:21.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:21.389+0000 7f0199c05640 1 -- 10.20.193.102:0/1368554053 wait complete. 2026-02-21T22:29:21.399 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2527720044}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=869850309}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2527720044}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.389+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2648759881}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3820885523}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=888900398}) 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.400 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3820885523}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3774979931}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743151654}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2693506221}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743151654}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1047942745}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159717229}) 2026-02-21T22:29:21.401 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=308104952}) 2026-02-21T22:29:21.402 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.402 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4159717229}) 2026-02-21T22:29:21.402 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.393+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2298361980}) 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968697881}) 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2982106399}) 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=968697881}) 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1245728899}) 2026-02-21T22:29:21.404 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142728404}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2593806403}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=142728404}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2440968472}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207620405}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1750557146}) 2026-02-21T22:29:21.405 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3207620405}) 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3252472650}) 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=173176288}) 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3331042437}) 2026-02-21T22:29:21.406 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.407 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=173176288}) 2026-02-21T22:29:21.407 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.397+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4165298391}) 2026-02-21T22:29:21.408 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.408 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2844253920}) 2026-02-21T22:29:21.408 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1237466843}) 2026-02-21T22:29:21.408 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2844253920}) 2026-02-21T22:29:21.409 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4230485651}) 2026-02-21T22:29:21.410 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33307805}) 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=365916747}) 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=33307805}) 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1331158893}) 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3170735383}) 2026-02-21T22:29:21.414 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3204327845}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3170735383}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.401+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4158108762}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2956901607}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3219934541}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2956901607}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=423810488}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2386533696}) 2026-02-21T22:29:21.415 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2386228612}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2386533696}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3124848216}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093619816}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2153162237}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2093619816}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3850404675}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3713891848}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=116169746}) 2026-02-21T22:29:21.416 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3713891848}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.405+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4045847509}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351245692}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2369697913}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2351245692}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=210236074}) 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.417 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3582300028}) 2026-02-21T22:29:21.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3246807439}) 2026-02-21T22:29:21.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3582300028}) 2026-02-21T22:29:21.418 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2397102845}) 2026-02-21T22:29:21.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.421 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3074810753}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1051274168}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3074810753}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2514267943}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3073576759}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4176355868}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3073576759}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.409+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1676720680}) 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.422 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3966993800}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=750903913}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3966993800}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4037341040}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2930752141}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=674625397}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2930752141}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1200002230}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619554091}) 2026-02-21T22:29:21.423 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2959165888}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1619554091}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.413+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2436314456}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=50258649}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1407359242}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=50258649}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1346335944}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420172428}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2831296662}) 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.424 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2420172428}) 2026-02-21T22:29:21.425 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1322206862}) 2026-02-21T22:29:21.426 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1529730300}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2508891748}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1529730300}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3680479737}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157969724}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1957686038}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=157969724}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.417+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=949713961}) 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.427 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358906412}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1821650991}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=358906412}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1201810798}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=919430316}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3339311854}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=919430316}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1134792566}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1502131620}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3607552554}) 2026-02-21T22:29:21.428 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1502131620}) 2026-02-21T22:29:21.429 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.421+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3349232463}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=22,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=31} crc {logm=1428925377,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1611611666,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3171781147,osd_pg_creating=47271684,osd_snap=2359264418}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=21,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=31} crc {logm=637117320,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1251687105,mon_config_key=794043746,monmap=918322475,nvmeofgw=3360707892,osd_metadata=2457828410,osd_pg_creating=47271684,osd_snap=3171367766}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=22,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=31} crc {logm=1428925377,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1611611666,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3171781147,osd_pg_creating=47271684,osd_snap=2359264418}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=22,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=10,monmap=3,nvmeofgw=3,osd_metadata=4,osd_pg_creating=1,osd_snap=32} crc {logm=1273858737,mdsmap=3074481156,mgr=3834574798,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=1251687105,mon_config_key=2063301084,monmap=918322475,nvmeofgw=3360707892,osd_metadata=721314461,osd_pg_creating=47271684,osd_snap=1602438597}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=95,osdmap=5} crc {osd_snap=3963556751,osdmap=1103223707}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=96,osdmap=4} crc {osd_snap=2311798381,osdmap=1097883525}) 2026-02-21T22:29:21.430 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.431 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=95,osdmap=5} crc {osd_snap=3963556751,osdmap=1103223707}) 2026-02-21T22:29:21.431 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=96,osdmap=4} crc {osd_snap=756801016,osdmap=1097883525}) 2026-02-21T22:29:21.431 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=524230598}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=719627925}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=524230598}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=943661581}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2457503712}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3413268438}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2457503712}) 2026-02-21T22:29:21.432 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.425+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2671632634}) 2026-02-21T22:29:21.433 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.433 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2037646027}) 2026-02-21T22:29:21.433 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2350351032}) 2026-02-21T22:29:21.434 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.434 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2037646027}) 2026-02-21T22:29:21.434 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=468598447}) 2026-02-21T22:29:21.434 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3062817030}) 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1636665922}) 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3062817030}) 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=115131033}) 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=14} crc {osdmap=2606964733}) 2026-02-21T22:29:21.435 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:29:21.429+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=13} crc {osdmap=2197805397}) 2026-02-21T22:29:22.197 INFO:tasks.rados.rados.0.trial178.stdout:3257: done (0 left) 2026-02-21T22:29:22.197 INFO:tasks.rados.rados.0.trial178.stdout:3258: rollback oid 50 current snap is 326 2026-02-21T22:29:22.197 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 50 to 322 2026-02-21T22:29:22.197 INFO:tasks.rados.rados.0.trial178.stdout:3259: write oid 1 current snap is 326 2026-02-21T22:29:22.197 INFO:tasks.rados.rados.0.trial178.stdout:3259: seq_num 1145 ranges {444015=413055,1436000=773735,2709915=534100} 2026-02-21T22:29:22.202 INFO:tasks.rados.rados.0.trial178.stdout:3259: writing trial17818531-1 from 444015 to 857070 tid 1 2026-02-21T22:29:22.212 INFO:tasks.rados.rados.0.trial178.stdout:3259: writing trial17818531-1 from 1436000 to 2209735 tid 2 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3259: writing trial17818531-1 from 2709915 to 3244015 tid 3 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3260: read oid 7 snap -1 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3260: expect (ObjNum 1062 snap 308 seq_num 1062) 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3261: snap_create 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3258: finishing rollback tid 0 to trial17818531-50 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3258: finishing rollback tid 1 to trial17818531-50 2026-02-21T22:29:22.220 INFO:tasks.rados.rados.0.trial178.stdout:3258: finishing rollback tid 2 to trial17818531-50 2026-02-21T22:29:22.221 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1617 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:22.221 INFO:tasks.rados.rados.0.trial178.stdout:3259: finishing write tid 1 to trial17818531-1 2026-02-21T22:29:22.221 INFO:tasks.rados.rados.0.trial178.stdout:3259: finishing write tid 2 to trial17818531-1 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: finishing write tid 3 to trial17818531-1 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: finishing write tid 4 to trial17818531-1 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: finishing write tid 5 to trial17818531-1 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: oid 1 updating version 0 to 1487 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: oid 1 version 1487 is already newer than 1486 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: oid 1 version 1487 is already newer than 1485 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:3259: oid 1 version 1487 is already newer than 1484 2026-02-21T22:29:22.222 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1487 (ObjNum 1145 snap 326 seq_num 1145) dirty exists 2026-02-21T22:29:22.223 INFO:tasks.rados.rados.0.trial178.stdout:3259: left oid 1 (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:29:22.225 INFO:tasks.rados.rados.0.trial178.stdout:3260: expect (ObjNum 1062 snap 308 seq_num 1062) 2026-02-21T22:29:22.271 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3697188 2026-02-21T22:29:22.411 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:29:22.412 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:29:22.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: get_monmap_and_config 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: monmap: 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:22.482747+0000 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:22.482747+0000 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 1 Processor -- start 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 1 -- start start 2026-02-21T22:29:22.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: init 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-a con 0x7fbd58088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-b con 0x7fbd5808a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-c con 0x7fbd58089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd58085920 con 0x7fbd58088d80 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5805e780 con 0x7fbd58089860 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd58086330 con 0x7fbd5808a1d0 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd5d1c4640 10 monclient(hunting): authenticate will time out at 1971.718969s 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd56d76640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbd58088d80 0x7fbd58089150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41118/0 (socket says 10.20.193.102:41118) 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd56d76640 1 -- 10.20.193.102:0/1010224502 learned_addr learned my addr 10.20.193.102:0/1010224502 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.477+0000 7fbd56575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fbd5808a1d0 0x7fbd58180c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42662/0 (socket says 10.20.193.102:42662) 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd56d76640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.102:6789/0 conn(0x7fbd58088d80 legacy=0x7fbd58089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd56575640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.102:6790/0 conn(0x7fbd5808a1d0 legacy=0x7fbd58180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720983989 0 0) 0x7fbd58085920 con 0x7fbd58088d80 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient(hunting): my global_id is 34701 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd44003620 con 0x7fbd58088d80 2026-02-21T22:29:22.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd57577640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.178:6789/0 conn(0x7fbd58089860 legacy=0x7fbd581a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 923708671 0 0) 0x7fbd44003620 con 0x7fbd58088d80 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.102:6790/0 conn(0x7fbd5808a1d0 legacy=0x7fbd58180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.178:6789/0 conn(0x7fbd58089860 legacy=0x7fbd581a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 monclient: found mon.noname-a 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5808c6b0 con 0x7fbd58088d80 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 5 monclient: authenticate success, global_id 34701 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbd4c003110 con 0x7fbd58088d80 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: dump: 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.486 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbd4c003450 con 0x7fbd58088d80 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 1 -- 10.20.193.102:0/1010224502 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbd4c004b50 con 0x7fbd58088d80 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd55d74640 10 monclient: dump: 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 4 set_mon_vals no callback set 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: shutdown 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1010224502 >> v1:10.20.193.102:6789/0 conn(0x7fbd58088d80 legacy=0x7fbd58089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1010224502 shutdown_connections 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1010224502 >> 10.20.193.102:0/1010224502 conn(0x7fbd58091a50 msgr2=0x7fbd58091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1010224502 shutdown_connections 2026-02-21T22:29:22.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1010224502 wait complete. 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: monmap: 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 Processor -- start 2026-02-21T22:29:22.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- start start 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: init 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-a con 0x7fbd58088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-c con 0x7fbd58089860 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient(hunting): picked mon.noname-b con 0x7fbd5808a1d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5813f680 con 0x7fbd58088d80 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd58153ee0 con 0x7fbd5808a1d0 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5813e160 con 0x7fbd58089860 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd5d1c4640 10 monclient(hunting): authenticate will time out at 1971.723546s 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd56575640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fbd58089860 0x7fbd581537d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42664/0 (socket says 10.20.193.102:42664) 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.481+0000 7fbd56575640 1 -- 10.20.193.102:0/1647063629 learned_addr learned my addr 10.20.193.102:0/1647063629 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd56575640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.102:6790/0 conn(0x7fbd58089860 legacy=0x7fbd581537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd56d76640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.102:6789/0 conn(0x7fbd58088d80 legacy=0x7fbd581530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2072583150 0 0) 0x7fbd5813f680 con 0x7fbd58088d80 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): my global_id is 34704 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd24003620 con 0x7fbd58088d80 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd57577640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.178:6789/0 conn(0x7fbd5808a1d0 legacy=0x7fbd58195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1453966689 0 0) 0x7fbd24003620 con 0x7fbd58088d80 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbd5813f680 con 0x7fbd58088d80 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbd4c004c40 con 0x7fbd58088d80 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): dump: 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1146572512 0 0) 0x7fbd5813f680 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.102:6790/0 conn(0x7fbd58089860 legacy=0x7fbd581537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.178:6789/0 conn(0x7fbd5808a1d0 legacy=0x7fbd58195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 monclient: found mon.a 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd580878e0 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 5 monclient: authenticate success, global_id 34704 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _renew_subs 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5813f960 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _renew_subs 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbd5813de70 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbd4c0031d0 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbd4c005130 con 0x7fbd58088d80 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 10 monclient: dump: 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd55573640 4 set_mon_vals no callback set 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd581401e0 con 0x7fbd58088d80 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fbd4c0117c0 con 0x7fbd58088d80 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(675..675 src has 395..675) ==== 4003+0+0 (unknown 2961038760 0 0) 0x7fbd4c04c6c0 con 0x7fbd58088d80 2026-02-21T22:29:22.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.485+0000 7fbd56575640 10 monclient: get_auth_request con 0x7fbd2403d480 auth_method 0 2026-02-21T22:29:22.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.489+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fbd4c017730 con 0x7fbd58088d80 2026-02-21T22:29:22.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.489+0000 7fbd377fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.489+0000 7fbd377fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:22.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd5d1c4640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:29:22.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd5d1c4640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd5d1c4640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbd58062ce0 con 0x7fbd58088d80 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd377fe640 1 -- 10.20.193.102:0/1647063629 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 726103470) 0x7fbd4c017010 con 0x7fbd58088d80 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd377fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.605+0000 7fbd377fe640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:22.612 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:29:22.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.178:6800/2549949439 conn(0x7fbd2403d480 legacy=0x7fbd2405d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 10 monclient: shutdown 2026-02-21T22:29:22.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 >> v1:10.20.193.102:6789/0 conn(0x7fbd58088d80 legacy=0x7fbd581530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 shutdown_connections 2026-02-21T22:29:22.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 >> 10.20.193.102:0/1647063629 conn(0x7fbd58091a50 msgr2=0x7fbd5808cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 shutdown_connections 2026-02-21T22:29:22.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.609+0000 7fbd5d1c4640 1 -- 10.20.193.102:0/1647063629 wait complete. 2026-02-21T22:29:22.623 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:29:22.624 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:29:22.624 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:29:22.624 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: get_monmap_and_config 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: monmap: 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.687 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:22.687848+0000 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:22.687848+0000 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:22.688 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 1 Processor -- start 2026-02-21T22:29:22.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 1 -- start start 2026-02-21T22:29:22.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: init 2026-02-21T22:29:22.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-b con 0x7fb968091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-c con 0x7fb9680909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-a con 0x7fb9681854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968057a20 con 0x7fb9681854b0 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968091d20 con 0x7fb9680909a0 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968057740 con 0x7fb968091240 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.681+0000 7fb96e517640 10 monclient(hunting): authenticate will time out at 1971.923902s 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96ca8d640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fb9681854b0 0x7fb9681a5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41134/0 (socket says 10.20.193.102:41134) 2026-02-21T22:29:22.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96ca8d640 1 -- 10.20.193.102:0/1253457505 learned_addr learned my addr 10.20.193.102:0/1253457505 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb967fff640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.102:6790/0 conn(0x7fb968091240 legacy=0x7fb968091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96ca8d640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.102:6789/0 conn(0x7fb9681854b0 legacy=0x7fb9681a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb9677fe640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.178:6789/0 conn(0x7fb9680909a0 legacy=0x7fb968183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2333090700 0 0) 0x7fb968057740 con 0x7fb968091240 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): my global_id is 94208 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb94c003620 con 0x7fb968091240 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765974864 0 0) 0x7fb968057a20 con 0x7fb9681854b0 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient(hunting): my global_id is 34707 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968057740 con 0x7fb9681854b0 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 521349730 0 0) 0x7fb94c003620 con 0x7fb968091240 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.178:6789/0 conn(0x7fb9680909a0 legacy=0x7fb968183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.102:6789/0 conn(0x7fb9681854b0 legacy=0x7fb9681a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 monclient: found mon.noname-b 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb968090dc0 con 0x7fb968091240 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb954003160 con 0x7fb968091240 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: dump: 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.695 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 5 monclient: authenticate success, global_id 94208 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb954003430 con 0x7fb968091240 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 1 -- 10.20.193.102:0/1253457505 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb9540038c0 con 0x7fb968091240 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb966ffd640 10 monclient: dump: 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 4 set_mon_vals no callback set 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 10 monclient: shutdown 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 1 -- 10.20.193.102:0/1253457505 >> v1:10.20.193.102:6790/0 conn(0x7fb968091240 legacy=0x7fb968091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 1 -- 10.20.193.102:0/1253457505 shutdown_connections 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 1 -- 10.20.193.102:0/1253457505 >> 10.20.193.102:0/1253457505 conn(0x7fb96805e660 msgr2=0x7fb968095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 1 -- 10.20.193.102:0/1253457505 shutdown_connections 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 1 -- 10.20.193.102:0/1253457505 wait complete. 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.685+0000 7fb96e517640 10 monclient: monmap: 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 Processor -- start 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- start start 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: init 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-b con 0x7fb9680909a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-c con 0x7fb968091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient(hunting): picked mon.noname-a con 0x7fb9681854b0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb96812bbf0 con 0x7fb9681854b0 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968112e40 con 0x7fb9680909a0 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968111fd0 con 0x7fb968091240 2026-02-21T22:29:22.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient(hunting): authenticate will time out at 1971.928593s 2026-02-21T22:29:22.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb9677fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fb968091240 0x7fb9681728a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42680/0 (socket says 10.20.193.102:42680) 2026-02-21T22:29:22.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb9677fe640 1 -- 10.20.193.102:0/215689578 learned_addr learned my addr 10.20.193.102:0/215689578 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb9677fe640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6790/0 conn(0x7fb968091240 legacy=0x7fb9681728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96ca8d640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb9681854b0 legacy=0x7fb968173fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241691999 0 0) 0x7fb968111fd0 con 0x7fb968091240 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): my global_id is 94211 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb938003620 con 0x7fb968091240 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100241360 0 0) 0x7fb96812bbf0 con 0x7fb9681854b0 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): my global_id is 34710 2026-02-21T22:29:22.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968111fd0 con 0x7fb9681854b0 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb967fff640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.178:6789/0 conn(0x7fb9680909a0 legacy=0x7fb96812b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1240532905 0 0) 0x7fb938003620 con 0x7fb968091240 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb96812bbf0 con 0x7fb968091240 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb958003400 con 0x7fb968091240 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.705 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2935880148 0 0) 0x7fb968111fd0 con 0x7fb9681854b0 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb938003620 con 0x7fb9681854b0 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb95c005df0 con 0x7fb9681854b0 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.706 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 49483580 0 0) 0x7fb96812bbf0 con 0x7fb968091240 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.178:6789/0 conn(0x7fb9680909a0 legacy=0x7fb96812b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb9681854b0 legacy=0x7fb968173fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 monclient: found mon.c 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb968110ae0 con 0x7fb968091240 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 5 monclient: authenticate success, global_id 94211 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _renew_subs 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb9681125a0 con 0x7fb968091240 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _renew_subs 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fb968111810 con 0x7fb968091240 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb9580039c0 con 0x7fb968091240 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb958003d50 con 0x7fb968091240 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb9667fc640 4 set_mon_vals no callback set 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb964ff9640 10 monclient: dump: 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.689+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb968110530 con 0x7fb96812e480 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fb958003fe0 con 0x7fb968091240 2026-02-21T22:29:22.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(675..675 src has 395..675) ==== 4003+0+0 (unknown 2961038760 0 0) 0x7fb95804c710 con 0x7fb968091240 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb96ca8d640 10 monclient: get_auth_request con 0x7fb93803d610 auth_method 0 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb967fff640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96812e480 legacy=0x7fb96808da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585073591 0 0) 0x7fb968110530 con 0x7fb96812e480 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient(hunting): my global_id is 34713 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb96812bbf0 con 0x7fb96812e480 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3620061229 0 0) 0x7fb96812bbf0 con 0x7fb96812e480 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb968110530 con 0x7fb96812e480 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb954003340 con 0x7fb96812e480 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient: dump: 2026-02-21T22:29:22.710 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2597148889 0 0) 0x7fb968110530 con 0x7fb96812e480 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9680008e0 con 0x7fb96812e480 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fb954002de0 con 0x7fb96812e480 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96812e480 legacy=0x7fb96808da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.693+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96812e480 legacy=0x7fb96808da20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.697+0000 7fb96e517640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:22.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.697+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb96812e850 con 0x7fb96804d280 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb967fff640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96804d280 legacy=0x7fb9681766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375018365 0 0) 0x7fb96812e850 con 0x7fb96804d280 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient(hunting): my global_id is 34716 2026-02-21T22:29:22.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968110530 con 0x7fb96804d280 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1778521785 0 0) 0x7fb968110530 con 0x7fb96804d280 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb96812e850 con 0x7fb96804d280 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb954003de0 con 0x7fb96804d280 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient: dump: 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.715 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 900261775 0 0) 0x7fb96812e850 con 0x7fb96804d280 2026-02-21T22:29:22.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb968095010 con 0x7fb96804d280 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3086323771) 0x7fb9540040b0 con 0x7fb96804d280 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96804d280 legacy=0x7fb9681766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb964ff9640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6789/0 conn(0x7fb96804d280 legacy=0x7fb9681766f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.717 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 711570, 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:22.718 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:22.719 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.720 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.721 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.722 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:22.723 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:22.724 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.725 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.726 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.727 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:22.728 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:22.729 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.730 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.731 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:22.732 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:22.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.734 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:22.735 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.178:6800/2549949439 conn(0x7fb93803d610 legacy=0x7fb93805dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 10 monclient: shutdown 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 >> v1:10.20.193.102:6790/0 conn(0x7fb968091240 legacy=0x7fb9681728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96ca8d640 1 -- 10.20.193.102:0/215689578 reap_dead start 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96ca8d640 1 -- 10.20.193.102:0/215689578 reap_dead start 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 shutdown_connections 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 >> 10.20.193.102:0/215689578 conn(0x7fb96805e660 msgr2=0x7fb968089c70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 shutdown_connections 2026-02-21T22:29:22.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.701+0000 7fb96e517640 1 -- 10.20.193.102:0/215689578 wait complete. 2026-02-21T22:29:22.738 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:22.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:29:22.738 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:29:22.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: get_monmap_and_config 2026-02-21T22:29:22.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: monmap: 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:22.810382+0000 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:22.810382+0000 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:22.810 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 1 Processor -- start 2026-02-21T22:29:22.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 1 -- start start 2026-02-21T22:29:22.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: init 2026-02-21T22:29:22.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-c con 0x7f92d408e3a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-b con 0x7f92d408ee80 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-a con 0x7f92d4058e30 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d408fba0 con 0x7f92d4058e30 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d4057bd0 con 0x7f92d408e3a0 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d4057eb0 con 0x7f92d408ee80 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d9228640 10 monclient(hunting): authenticate will time out at 1972.046286s 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d8a27640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f92d4058e30 0x7f92d41a5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41150/0 (socket says 10.20.193.102:41150) 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d8a27640 1 -- 10.20.193.102:0/1992612257 learned_addr learned my addr 10.20.193.102:0/1992612257 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.812 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.805+0000 7f92d37fe640 1 --1- 10.20.193.102:0/1992612257 >> v1:10.20.193.102:6790/0 conn(0x7f92d408ee80 0x7f92d41842e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42696/0 (socket says 10.20.193.102:42696) 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d8a27640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.102:6789/0 conn(0x7f92d4058e30 legacy=0x7f92d41a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d37fe640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.102:6790/0 conn(0x7f92d408ee80 legacy=0x7f92d41842e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220368140 0 0) 0x7f92c8004170 con 0x7f92d4058e30 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): my global_id is 34719 2026-02-21T22:29:22.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d3fff640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.178:6789/0 conn(0x7f92d408e3a0 legacy=0x7f92d408e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b8003620 con 0x7f92d4058e30 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679901321 0 0) 0x7f92d4057eb0 con 0x7f92d408ee80 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient(hunting): my global_id is 94214 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92c8004170 con 0x7f92d408ee80 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3799745556 0 0) 0x7f92b8003620 con 0x7f92d4058e30 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.102:6790/0 conn(0x7f92d408ee80 legacy=0x7f92d41842e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.178:6789/0 conn(0x7f92d408e3a0 legacy=0x7f92d408e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 monclient: found mon.noname-a 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d4059280 con 0x7f92d4058e30 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 5 monclient: authenticate success, global_id 34719 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c80047f0 con 0x7f92d4058e30 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: dump: 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92c8005eb0 con 0x7f92d4058e30 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 1 -- 10.20.193.102:0/1992612257 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c8006240 con 0x7f92d4058e30 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d2ffd640 10 monclient: dump: 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.817 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 4 set_mon_vals no callback set 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: shutdown 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- 10.20.193.102:0/1992612257 >> v1:10.20.193.102:6789/0 conn(0x7f92d4058e30 legacy=0x7f92d41a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- 10.20.193.102:0/1992612257 shutdown_connections 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- 10.20.193.102:0/1992612257 >> 10.20.193.102:0/1992612257 conn(0x7f92d4056cc0 msgr2=0x7f92d40939a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- 10.20.193.102:0/1992612257 shutdown_connections 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- 10.20.193.102:0/1992612257 wait complete. 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: monmap: 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.818 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:22.819 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:22.819 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 Processor -- start 2026-02-21T22:29:22.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- start start 2026-02-21T22:29:22.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: init 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-b con 0x7f92d4058e30 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-c con 0x7f92d408e3a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient(hunting): picked mon.noname-a con 0x7f92d408ee80 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d408a290 con 0x7f92d408ee80 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d408ab90 con 0x7f92d4058e30 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92d413f3e0 con 0x7f92d408e3a0 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d9228640 10 monclient(hunting): authenticate will time out at 1972.050754s 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d37fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f92d408e3a0 0x7f92d4194820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42708/0 (socket says 10.20.193.102:42708) 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.809+0000 7f92d37fe640 1 -- 10.20.193.102:0/1989912943 learned_addr learned my addr 10.20.193.102:0/1989912943 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d37fe640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d408e3a0 legacy=0x7f92d4194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.823 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d8a27640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6789/0 conn(0x7f92d408ee80 legacy=0x7f92d4195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d3fff640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.178:6789/0 conn(0x7f92d4058e30 legacy=0x7f92d4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613006079 0 0) 0x7f92d413f3e0 con 0x7f92d408e3a0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): my global_id is 94217 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92a4003620 con 0x7f92d408e3a0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757475303 0 0) 0x7f92d408a290 con 0x7f92d408ee80 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): my global_id is 34722 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d413f3e0 con 0x7f92d408ee80 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2035506684 0 0) 0x7f92a4003620 con 0x7f92d408e3a0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92d408a290 con 0x7f92d408e3a0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c4003440 con 0x7f92d408e3a0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.824 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3842851987 0 0) 0x7f92d413f3e0 con 0x7f92d408ee80 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92a4003620 con 0x7f92d408ee80 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c8004640 con 0x7f92d408ee80 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:22.825 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138598084 0 0) 0x7f92d408ab90 con 0x7f92d4058e30 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient(hunting): my global_id is 104146 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d413f3e0 con 0x7f92d4058e30 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2080347526 0 0) 0x7f92d408a290 con 0x7f92d408e3a0 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.178:6789/0 conn(0x7f92d4058e30 legacy=0x7f92d4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6789/0 conn(0x7f92d408ee80 legacy=0x7f92d4195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 monclient: found mon.c 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d4141460 con 0x7f92d408e3a0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 5 monclient: authenticate success, global_id 94217 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 10 monclient: _renew_subs 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f92d4141740 con 0x7f92d408e3a0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 10 monclient: _renew_subs 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f92d413ea70 con 0x7f92d408e3a0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92c4004d70 con 0x7f92d408e3a0 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c40050d0 con 0x7f92d408e3a0 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 10 monclient: dump: 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d27fc640 4 set_mon_vals no callback set 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f92d40008e0 con 0x7f92d4153d20 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f92c4011760 con 0x7f92d408e3a0 2026-02-21T22:29:22.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(675..675 src has 395..675) ==== 4003+0+0 (unknown 2961038760 0 0) 0x7f92c404c7e0 con 0x7f92d408e3a0 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.813+0000 7f92d8a27640 10 monclient: get_auth_request con 0x7f92a403d880 auth_method 0 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d3fff640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d4153d20 legacy=0x7f92d4139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937458737 0 0) 0x7f92d40008e0 con 0x7f92d4153d20 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient(hunting): my global_id is 94220 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d408a290 con 0x7f92d4153d20 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 603084855 0 0) 0x7f92d408a290 con 0x7f92d4153d20 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92d40008e0 con 0x7f92d4153d20 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c0003260 con 0x7f92d4153d20 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient: dump: 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.829 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3752039115 0 0) 0x7f92d40008e0 con 0x7f92d4153d20 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92d4051160 con 0x7f92d4153d20 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f92c00058c0 con 0x7f92d4153d20 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d4153d20 legacy=0x7f92d4139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.830 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.817+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d4153d20 legacy=0x7f92d4139bf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d9228640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f92d413ed60 con 0x7f92d404d280 2026-02-21T22:29:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d3fff640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d404d280 legacy=0x7f92d4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218656027 0 0) 0x7f92d413ed60 con 0x7f92d404d280 2026-02-21T22:29:22.831 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient(hunting): my global_id is 94223 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d40008e0 con 0x7f92d404d280 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3787482337 0 0) 0x7f92d40008e0 con 0x7f92d404d280 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92d413ed60 con 0x7f92d404d280 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f92c0003af0 con 0x7f92d404d280 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient: dump: 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.832 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 119502409 0 0) 0x7f92d413ed60 con 0x7f92d404d280 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92d41410e0 con 0x7f92d404d280 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 1367641652) 0x7f92c0005b10 con 0x7f92d404d280 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.821+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d404d280 legacy=0x7f92d4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d0ff9640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d404d280 legacy=0x7f92d4198670 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:29:22.833 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 109096, 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.834 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.835 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:22.836 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.837 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.838 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.839 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.840 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:22.841 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:22.842 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.178:6800/2549949439 conn(0x7f92a403d880 legacy=0x7f92a405dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 10 monclient: shutdown 2026-02-21T22:29:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 >> v1:10.20.193.102:6790/0 conn(0x7f92d408e3a0 legacy=0x7f92d4194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d8a27640 1 -- 10.20.193.102:0/1989912943 reap_dead start 2026-02-21T22:29:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d8a27640 1 -- 10.20.193.102:0/1989912943 reap_dead start 2026-02-21T22:29:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 shutdown_connections 2026-02-21T22:29:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 >> 10.20.193.102:0/1989912943 conn(0x7f92d4056cc0 msgr2=0x7f92d408c020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 shutdown_connections 2026-02-21T22:29:22.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.825+0000 7f92d9228640 1 -- 10.20.193.102:0/1989912943 wait complete. 2026-02-21T22:29:22.844 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:22.844 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:29:22.844 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:29:22.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 10 monclient: get_monmap_and_config 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 10 monclient: monmap: 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:22.912238+0000 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:22.912238+0000 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 1 Processor -- start 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 1 -- start start 2026-02-21T22:29:22.912 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.905+0000 7f0d6cae3640 10 monclient: init 2026-02-21T22:29:22.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:22.913 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-c con 0x7f0d68088d80 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-b con 0x7f0d6808a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-a con 0x7f0d68089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68085920 con 0x7f0d68089860 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d6805e780 con 0x7f0d68088d80 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68086330 con 0x7f0d6808a1d0 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient(hunting): authenticate will time out at 1972.148351s 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0d6808a1d0 0x7f0d68180c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:42740/0 (socket says 10.20.193.102:42740) 2026-02-21T22:29:22.914 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65d74640 1 -- 10.20.193.102:0/1777612017 learned_addr learned my addr 10.20.193.102:0/1777612017 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d66d76640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.102:6789/0 conn(0x7f0d68089860 legacy=0x7f0d681a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65d74640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.102:6790/0 conn(0x7f0d6808a1d0 legacy=0x7f0d68180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d66575640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.178:6789/0 conn(0x7f0d68088d80 legacy=0x7f0d68089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491340185 0 0) 0x7f0d68086330 con 0x7f0d6808a1d0 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.917 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): my global_id is 94226 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d54003620 con 0x7f0d6808a1d0 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4201071405 0 0) 0x7f0d68085920 con 0x7f0d68089860 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient(hunting): my global_id is 34725 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d68086330 con 0x7f0d68089860 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36267899 0 0) 0x7f0d68086330 con 0x7f0d68089860 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.102:6790/0 conn(0x7f0d6808a1d0 legacy=0x7f0d68180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.178:6789/0 conn(0x7f0d68088d80 legacy=0x7f0d68089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 monclient: found mon.noname-a 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6808c6b0 con 0x7f0d68089860 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 5 monclient: authenticate success, global_id 34725 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d580045f0 con 0x7f0d68089860 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: dump: 2026-02-21T22:29:22.919 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0d580048c0 con 0x7f0d68089860 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 1 -- 10.20.193.102:0/1777612017 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d58004dd0 con 0x7f0d68089860 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d65573640 10 monclient: dump: 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 4 set_mon_vals no callback set 2026-02-21T22:29:22.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: shutdown 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- 10.20.193.102:0/1777612017 >> v1:10.20.193.102:6789/0 conn(0x7f0d68089860 legacy=0x7f0d681a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- 10.20.193.102:0/1777612017 shutdown_connections 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- 10.20.193.102:0/1777612017 >> 10.20.193.102:0/1777612017 conn(0x7f0d68091a50 msgr2=0x7f0d68091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- 10.20.193.102:0/1777612017 shutdown_connections 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- 10.20.193.102:0/1777612017 wait complete. 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: build_initial_monmap 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: monmap: 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:22.922 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 Processor -- start 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 1 -- start start 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.909+0000 7f0d6cae3640 10 monclient: init 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-b con 0x7f0d68088d80 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-a con 0x7f0d68089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:22.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient(hunting): picked mon.noname-c con 0x7f0d6808a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68084f50 con 0x7f0d68089860 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68058610 con 0x7f0d68088d80 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68059610 con 0x7f0d6808a1d0 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient(hunting): authenticate will time out at 1972.152226s 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d65d74640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0d68089860 0x7f0d6821fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41170/0 (socket says 10.20.193.102:41170) 2026-02-21T22:29:22.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d65d74640 1 -- 10.20.193.102:0/3054214633 learned_addr learned my addr 10.20.193.102:0/3054214633 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d65d74640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.102:6789/0 conn(0x7f0d68089860 legacy=0x7f0d6821fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d66d76640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.102:6790/0 conn(0x7f0d6808a1d0 legacy=0x7f0d68241330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33751137 0 0) 0x7f0d68084f50 con 0x7f0d68089860 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): my global_id is 34728 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d34003620 con 0x7f0d68089860 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2101974134 0 0) 0x7f0d68059610 con 0x7f0d6808a1d0 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): my global_id is 94229 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d68084f50 con 0x7f0d6808a1d0 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d66575640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d68088d80 legacy=0x7f0d681dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2425929594 0 0) 0x7f0d68084f50 con 0x7f0d6808a1d0 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0d68059610 con 0x7f0d6808a1d0 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d58004c20 con 0x7f0d6808a1d0 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): dump: 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.929 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1530276828 0 0) 0x7f0d34003620 con 0x7f0d68089860 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0d68084f50 con 0x7f0d68089860 2026-02-21T22:29:22.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d500048e0 con 0x7f0d68089860 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient(hunting): dump: 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 679419481 0 0) 0x7f0d68059610 con 0x7f0d6808a1d0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d68088d80 legacy=0x7f0d681dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.102:6789/0 conn(0x7f0d68089860 legacy=0x7f0d6821fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 monclient: found mon.c 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6805de60 con 0x7f0d6808a1d0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: _finish_auth 0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 5 monclient: authenticate success, global_id 94229 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _renew_subs 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0d680598f0 con 0x7f0d6808a1d0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _renew_subs 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f0d68083dc0 con 0x7f0d6808a1d0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0d58004e40 con 0x7f0d6808a1d0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d580053b0 con 0x7f0d6808a1d0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 10 monclient: dump: 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.933 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d64d72640 4 set_mon_vals no callback set 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0d58005650 con 0x7f0d6808a1d0 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0d68084150 con 0x7f0d6808cf90 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(675..675 src has 395..675) ==== 4003+0+0 (unknown 2961038760 0 0) 0x7f0d5804d980 con 0x7f0d6808a1d0 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.913+0000 7f0d65d74640 10 monclient: get_auth_request con 0x7f0d3403d6a0 auth_method 0 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d66575640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d6808cf90 legacy=0x7f0d68137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2207545356 0 0) 0x7f0d68084150 con 0x7f0d6808cf90 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient(hunting): my global_id is 104155 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d68059610 con 0x7f0d6808cf90 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2068645979 0 0) 0x7f0d68059610 con 0x7f0d6808cf90 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0d68084150 con 0x7f0d6808cf90 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d5c0031d0 con 0x7f0d6808cf90 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient: dump: 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.935 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1985091927 0 0) 0x7f0d68084150 con 0x7f0d6808cf90 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d680008e0 con 0x7f0d6808cf90 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f0d5c0058c0 con 0x7f0d6808cf90 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d6808cf90 legacy=0x7f0d68137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.917+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d6808cf90 legacy=0x7f0d68137960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.921+0000 7f0d6cae3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:22.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.921+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0d681df700 con 0x7f0d681df330 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d66575640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d681df330 legacy=0x7f0d68138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482522034 0 0) 0x7f0d681df700 con 0x7f0d681df330 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient(hunting): my global_id is 104158 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d68084150 con 0x7f0d681df330 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 479688117 0 0) 0x7f0d68084150 con 0x7f0d681df330 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0d681df700 con 0x7f0d681df330 2026-02-21T22:29:22.938 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0d5c003550 con 0x7f0d681df330 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient: dump: 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:22.939 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2812682170 0 0) 0x7f0d681df700 con 0x7f0d681df330 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0d68138780 con 0x7f0d681df330 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4345 (unknown 0 0 3949013770) 0x7f0d5c005b60 con 0x7f0d681df330 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d681df330 legacy=0x7f0d68138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d4effd640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6789/0 conn(0x7f0d681df330 legacy=0x7f0d68138070 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:22.940 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 52, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 2303, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.941 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:22.942 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.943 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:22.944 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:22.945 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.946 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:22.947 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.948 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:22.949 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:22.950 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:22.951 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0d3403d6a0 legacy=0x7f0d3405db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 10 monclient: shutdown 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 >> v1:10.20.193.102:6790/0 conn(0x7f0d6808a1d0 legacy=0x7f0d68241330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d66d76640 1 -- 10.20.193.102:0/3054214633 reap_dead start 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d66d76640 1 -- 10.20.193.102:0/3054214633 reap_dead start 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 shutdown_connections 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 >> 10.20.193.102:0/3054214633 conn(0x7f0d68091a50 msgr2=0x7f0d6805c920 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 shutdown_connections 2026-02-21T22:29:22.952 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:22.925+0000 7f0d6cae3640 1 -- 10.20.193.102:0/3054214633 wait complete. 2026-02-21T22:29:22.953 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:29:22.953 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-21T22:29:22.953 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2026-02-21T22:29:22.953 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-21T22:29:22.953 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-21T22:29:22.953 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:29:22.954 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:29:22.966 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:29:22.967 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.c 2026-02-21T22:29:22.967 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-21T22:29:22.967 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-21T22:29:22.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:29:23] 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:29:23, mode upmap, max misplaced 0.050000 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:29:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'unique_pool_0', 'rbd'] 2026-02-21T22:29:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:29:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:29:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:23.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:23.569 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:29:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:29:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:29:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:29:23.571 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:29:23.572 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:29:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:29:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:29:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:29:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:29:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:29:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:29:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:29:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:29:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:29:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:29:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:29:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:29:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:29:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "675", 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:29:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:29:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:29:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 675, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 332, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:29:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:29:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:29:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:29:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:29:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:29:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:29:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:29:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.000267827069317853 of space, bias 1.0, pg target 0.0803481207953559 quantized to 32 (current 16) 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:28.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:28.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:33.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:33.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:37.968 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.c 2026-02-21T22:29:37.968 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-21T22:29:37.968 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-21T22:29:37.968 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:29:37.968 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: get_monmap_and_config 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: build_initial_monmap 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: monmap: 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:38.035867+0000 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:38.035867+0000 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:38.035 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 1 Processor -- start 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 1 -- start start 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: init 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient(hunting): picked mon.noname-a con 0x7f7238084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient(hunting): picked mon.noname-c con 0x7f7238086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 10 monclient(hunting): picked mon.noname-b con 0x7f7238183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238086a70 con 0x7f7238084860 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.029+0000 7f723e964640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723808b130 con 0x7f7238086e90 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f723e964640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238085340 con 0x7f7238183350 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f723e964640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f723e964640 10 monclient(hunting): authenticate will time out at 1987.272017s 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f72377fe640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f7238181c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f7237fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7238084860 0x7f7238084c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:49300/0 (socket says 10.20.193.102:49300) 2026-02-21T22:29:38.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f7237fff640 1 -- 10.20.193.102:0/821754450 learned_addr learned my addr 10.20.193.102:0/821754450 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:38.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f7237fff640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6789/0 conn(0x7f7238084860 legacy=0x7f7238084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:38.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.033+0000 7f723ceda640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6790/0 conn(0x7f7238183350 legacy=0x7f72381a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:38.241 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.236+0000 7f72377fe640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f7238181c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:38.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:38.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:38.641 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:38.640+0000 7f72377fe640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f7238181c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:39.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:39.444+0000 7f72377fe640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f7238181c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 10 monclient(hunting): tick 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 1 monclient(hunting): continuing hunt 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6790/0 conn(0x7f7238183350 legacy=0x7f72381a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.028+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f7238181c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6789/0 conn(0x7f7238084860 legacy=0x7f7238084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:41.036 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 10 monclient(hunting): picked mon.noname-b con 0x7f7238183350 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 10 monclient(hunting): picked mon.noname-a con 0x7f7238084860 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 10 monclient(hunting): picked mon.noname-c con 0x7f7238086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238086790 con 0x7f7238084860 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723808b130 con 0x7f7238086e90 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 1 -- 10.20.193.102:0/821754450 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721406a0f0 con 0x7f7238183350 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7235ffb640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:41.037 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f723ceda640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f72140668d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:41.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f7237fff640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6790/0 conn(0x7f7238183350 legacy=0x7f72140238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:41.038 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.032+0000 7f72377fe640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6789/0 conn(0x7f7238084860 legacy=0x7f72140450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:41.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.236+0000 7f723ceda640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f72140668d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:41.640 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:41.640+0000 7f723ceda640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f72140668d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:42.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.444+0000 7f723ceda640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f72140668d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:42.999 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2387984952 0 0) 0x7f721406a0f0 con 0x7f7238183350 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient(hunting): my global_id is 94232 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f721c003620 con 0x7f7238183350 2026-02-21T22:29:43.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 990686158 0 0) 0x7f721c003620 con 0x7f7238183350 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.178:6789/0 conn(0x7f7238086e90 legacy=0x7f72140668d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6789/0 conn(0x7f7238084860 legacy=0x7f72140450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 monclient: found mon.noname-b 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721406b450 con 0x7f7238183350 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f72240030d0 con 0x7f7238183350 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 5 monclient: authenticate success, global_id 94232 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: dump: 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.002 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.003 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7224003410 con 0x7f7238183350 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 1 -- 10.20.193.102:0/821754450 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7224004b50 con 0x7f7238183350 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7236ffd640 10 monclient: dump: 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.004 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 4 set_mon_vals no callback set 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: shutdown 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.005 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- 10.20.193.102:0/821754450 >> v1:10.20.193.102:6790/0 conn(0x7f7238183350 legacy=0x7f72140238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- 10.20.193.102:0/821754450 shutdown_connections 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- 10.20.193.102:0/821754450 >> 10.20.193.102:0/821754450 conn(0x7f7238091a50 msgr2=0x7f7238091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- 10.20.193.102:0/821754450 shutdown_connections 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- 10.20.193.102:0/821754450 wait complete. 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: build_initial_monmap 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: monmap: 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:43.006 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 Processor -- start 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- start start 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: init 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient(hunting): picked mon.noname-b con 0x7f7238084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:43.007 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient(hunting): picked mon.noname-c con 0x7f7238086e90 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient(hunting): picked mon.noname-a con 0x7f7238183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238081880 con 0x7f7238183350 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238084330 con 0x7f7238084860 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7238080390 con 0x7f7238086e90 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723e964640 10 monclient(hunting): authenticate will time out at 1992.235146s 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f7237fff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f7238084860 legacy=0x7f7238083c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723ceda640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7238183350 0x7f7238175cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60306/0 (socket says 10.20.193.102:60306) 2026-02-21T22:29:43.008 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.992+0000 7f723ceda640 1 -- 10.20.193.102:0/3486165889 learned_addr learned my addr 10.20.193.102:0/3486165889 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723ceda640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.102:6789/0 conn(0x7f7238183350 legacy=0x7f7238175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f72377fe640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.102:6790/0 conn(0x7f7238086e90 legacy=0x7f7238174590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723ceda640 1 --1- 10.20.193.102:0/3486165889 >> v1:10.20.193.102:6789/0 conn(0x7f7238183350 0x7f7238175cb0 :-1 s=OPENED pgs=559 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f7238081880 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114244535 0 0) 0x7f7238080390 con 0x7f7238086e90 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): my global_id is 94235 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7208003620 con 0x7f7238086e90 2026-02-21T22:29:43.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1888409504 0 0) 0x7f7208003620 con 0x7f7238086e90 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7238080390 con 0x7f7238086e90 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7228004670 con 0x7f7238086e90 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.013 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4179696762 0 0) 0x7f7238080390 con 0x7f7238086e90 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.178:6789/0 conn(0x7f7238084860 legacy=0x7f7238083c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.102:6789/0 conn(0x7f7238183350 legacy=0x7f7238175cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 monclient: found mon.c 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72380826f0 con 0x7f7238086e90 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 290928299 0 0) 0x7f7238081880 con 0x7f7238183350 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 5 monclient: authenticate success, global_id 94235 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _renew_subs 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7238081e50 con 0x7f7238086e90 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _renew_subs 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f72380810c0 con 0x7f7238086e90 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7228004b40 con 0x7f7238086e90 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7228004f10 con 0x7f7238086e90 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 10 monclient: dump: 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.016 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f72367fc640 4 set_mon_vals no callback set 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7238058b40 con 0x7f7238086e90 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f72280051e0 con 0x7f7238086e90 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(676..676 src has 395..676) ==== 3947+0+0 (unknown 656618030 0 0) 0x7f722804c710 con 0x7f7238086e90 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:42.996+0000 7f7237fff640 10 monclient: get_auth_request con 0x7f720803d320 auth_method 0 2026-02-21T22:29:43.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.000+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f7228017640 con 0x7f7238086e90 2026-02-21T22:29:43.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.000+0000 7f7234ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.000+0000 7f7234ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7238062ce0 con 0x7f7238086e90 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f7234ff9640 1 -- 10.20.193.102:0/3486165889 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 388911139) 0x7f722801c960 con 0x7f7238086e90 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f7234ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f7234ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:43.121 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":54,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.178:6800/2549949439 conn(0x7f720803d320 legacy=0x7f720805d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 10 monclient: shutdown 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 >> v1:10.20.193.102:6790/0 conn(0x7f7238086e90 legacy=0x7f7238174590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 shutdown_connections 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 >> 10.20.193.102:0/3486165889 conn(0x7f7238091a50 msgr2=0x7f72380873e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 shutdown_connections 2026-02-21T22:29:43.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.116+0000 7f723e964640 1 -- 10.20.193.102:0/3486165889 wait complete. 2026-02-21T22:29:43.131 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:29:43.132 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:29:43.132 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 10 monclient: get_monmap_and_config 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 10 monclient: build_initial_monmap 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 10 monclient: monmap: 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:43.191892+0000 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:43.191892+0000 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.191 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:43.192 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:43.192 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:43.192 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 1 Processor -- start 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 1 -- start start 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.184+0000 7fbdb87e4640 10 monclient: init 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-b con 0x7fbdb008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-a con 0x7fbdb008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-c con 0x7fbdb008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb0086a10 con 0x7fbdb008b7f0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb008b130 con 0x7fbdb008d8e0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb008bd60 con 0x7fbdb008ce00 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient(hunting): authenticate will time out at 1992.428122s 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb6d5a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fbdb008d8e0 legacy=0x7fbdb01a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5d58640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 0x7fbdb0181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60314/0 (socket says 10.20.193.102:60314) 2026-02-21T22:29:43.193 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5d58640 1 -- 10.20.193.102:0/2131881533 learned_addr learned my addr 10.20.193.102:0/2131881533 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5d58640 1 -- 10.20.193.102:0/2131881533 >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 legacy=0x7fbdb0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb6559640 1 -- 10.20.193.102:0/2131881533 >> v1:10.20.193.102:6790/0 conn(0x7fbdb008ce00 legacy=0x7fbdb008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701684185 0 0) 0x7fbdb0086a10 con 0x7fbdb008b7f0 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.195 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): my global_id is 34734 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd98003620 con 0x7fbdb008b7f0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067742024 0 0) 0x7fbdb008bd60 con 0x7fbdb008ce00 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient(hunting): my global_id is 94238 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb0086a10 con 0x7fbdb008ce00 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 514349210 0 0) 0x7fbd98003620 con 0x7fbdb008b7f0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 >> v1:10.20.193.102:6790/0 conn(0x7fbdb008ce00 legacy=0x7fbdb008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 >> v1:10.20.193.178:6789/0 conn(0x7fbdb008d8e0 legacy=0x7fbdb01a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 monclient: found mon.noname-a 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb008b370 con 0x7fbdb008b7f0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 5 monclient: authenticate success, global_id 34734 2026-02-21T22:29:43.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda4003110 con 0x7fbdb008b7f0 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: dump: 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbda40033e0 con 0x7fbdb008b7f0 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 1 -- 10.20.193.102:0/2131881533 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda4004b50 con 0x7fbdb008b7f0 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb5557640 10 monclient: dump: 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.197 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 4 set_mon_vals no callback set 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: shutdown 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2131881533 >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 legacy=0x7fbdb0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2131881533 shutdown_connections 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2131881533 >> 10.20.193.102:0/2131881533 conn(0x7fbdb0091a50 msgr2=0x7fbdb0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2131881533 shutdown_connections 2026-02-21T22:29:43.198 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2131881533 wait complete. 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: build_initial_monmap 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.188+0000 7fbdb87e4640 10 monclient: monmap: 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 Processor -- start 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- start start 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: init 2026-02-21T22:29:43.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-a con 0x7fbdb008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-c con 0x7fbdb008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient(hunting): picked mon.noname-b con 0x7fbdb008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb0080fe0 con 0x7fbdb008b7f0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb0058e30 con 0x7fbdb008d8e0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb0088a70 con 0x7fbdb008ce00 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient(hunting): authenticate will time out at 1992.432737s 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb6559640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 0x7fbdb0084870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60318/0 (socket says 10.20.193.102:60318) 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb6559640 1 -- 10.20.193.102:0/2293626662 learned_addr learned my addr 10.20.193.102:0/2293626662 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb6d5a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fbdb008d8e0 legacy=0x7fbdb0195bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb5d58640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6790/0 conn(0x7fbdb008ce00 legacy=0x7fbdb0194490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb6559640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 legacy=0x7fbdb0084870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144446630 0 0) 0x7fbdb0088a70 con 0x7fbdb008ce00 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): my global_id is 94241 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd80003620 con 0x7fbdb008ce00 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569011973 0 0) 0x7fbdb0080fe0 con 0x7fbdb008b7f0 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): my global_id is 34737 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb0088a70 con 0x7fbdb008b7f0 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1461605552 0 0) 0x7fbd80003620 con 0x7fbdb008ce00 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbdb0080fe0 con 0x7fbdb008ce00 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda4002b70 con 0x7fbdb008ce00 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): dump: 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.203 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 148383163 0 0) 0x7fbdb0088a70 con 0x7fbdb008b7f0 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbd80003620 con 0x7fbdb008b7f0 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda0003490 con 0x7fbdb008b7f0 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient(hunting): dump: 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1419025367 0 0) 0x7fbdb0080fe0 con 0x7fbdb008ce00 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.178:6789/0 conn(0x7fbdb008d8e0 legacy=0x7fbdb0195bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:29:43.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb008b7f0 legacy=0x7fbdb0084870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 monclient: found mon.c 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb0083340 con 0x7fbdb008ce00 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 5 monclient: authenticate success, global_id 94241 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _renew_subs 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb0081b40 con 0x7fbdb008ce00 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _renew_subs 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbdb0085150 con 0x7fbdb008ce00 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbda40033c0 con 0x7fbdb008ce00 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda4005190 con 0x7fbdb008ce00 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbd96ffd640 10 monclient: dump: 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.205 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb4d56640 4 set_mon_vals no callback set 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.192+0000 7fbdb87e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fbda4011820 con 0x7fbdb008ce00 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbdb00008e0 con 0x7fbdb0153390 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(676..676 src has 395..676) ==== 3947+0+0 (unknown 656618030 0 0) 0x7fbda404c580 con 0x7fbdb008ce00 2026-02-21T22:29:43.206 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbdb6d5a640 10 monclient: get_auth_request con 0x7fbd8003d520 auth_method 0 2026-02-21T22:29:43.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbdb6559640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb0153390 legacy=0x7fbdb005a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367559339 0 0) 0x7fbdb00008e0 con 0x7fbdb0153390 2026-02-21T22:29:43.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient(hunting): my global_id is 34740 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb0080fe0 con 0x7fbdb0153390 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1664047963 0 0) 0x7fbdb0080fe0 con 0x7fbdb0153390 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbdb00008e0 con 0x7fbdb0153390 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda0003f40 con 0x7fbdb0153390 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient: dump: 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 165183991 0 0) 0x7fbdb00008e0 con 0x7fbdb0153390 2026-02-21T22:29:43.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdb0154080 con 0x7fbdb0153390 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fbda0004240 con 0x7fbdb0153390 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb0153390 legacy=0x7fbdb005a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.196+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb0153390 legacy=0x7fbdb005a1e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.200+0000 7fbdb87e4640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:43.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.200+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbdb0153760 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb6559640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb005a8f0 legacy=0x7fbdb01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973659813 0 0) 0x7fbdb0153760 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient(hunting): my global_id is 34743 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb00008e0 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3368181366 0 0) 0x7fbdb00008e0 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbdb0153760 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fbda0005a70 con 0x7fbdb005a8f0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient: dump: 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.210 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2709768738 0 0) 0x7fbdb0153760 con 0x7fbdb005a8f0 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbdb0090810 con 0x7fbdb005a8f0 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 1074987136) 0x7fbda0006e60 con 0x7fbdb005a8f0 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb005a8f0 legacy=0x7fbdb01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbd96ffd640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6789/0 conn(0x7fbdb005a8f0 legacy=0x7fbdb01982e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:43.211 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 54, 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 732073, 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.212 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.213 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:43.214 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.215 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.216 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.217 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.218 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.219 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.220 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:43.220 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.178:6800/2549949439 conn(0x7fbd8003d520 legacy=0x7fbd8005d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 10 monclient: shutdown 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 >> v1:10.20.193.102:6790/0 conn(0x7fbdb008ce00 legacy=0x7fbdb0194490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb6d5a640 1 -- 10.20.193.102:0/2293626662 reap_dead start 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb6d5a640 1 -- 10.20.193.102:0/2293626662 reap_dead start 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 shutdown_connections 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 >> 10.20.193.102:0/2293626662 conn(0x7fbdb0091a50 msgr2=0x7fbdb0088d80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 shutdown_connections 2026-02-21T22:29:43.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.204+0000 7fbdb87e4640 1 -- 10.20.193.102:0/2293626662 wait complete. 2026-02-21T22:29:43.221 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:29:43.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:29:43.222 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 10 monclient: get_monmap_and_config 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 10 monclient: build_initial_monmap 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 10 monclient: monmap: 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:29:43.292082+0000 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:29:43.292082+0000 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:29:43.291 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 1 Processor -- start 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 1 -- start start 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.284+0000 7f991a956640 10 monclient: init 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient(hunting): picked mon.noname-c con 0x7f991408ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient(hunting): picked mon.noname-a con 0x7f991408b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient(hunting): picked mon.noname-b con 0x7f991408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9914086a10 con 0x7f991408b7f0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991408b130 con 0x7f991408ce00 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991408bd60 con 0x7f991408d8e0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient(hunting): authenticate will time out at 1992.528306s 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9913fff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f991408ce00 legacy=0x7f991408d1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f99137fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f991408b7f0 0x7f9914181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60334/0 (socket says 10.20.193.102:60334) 2026-02-21T22:29:43.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f99137fe640 1 -- 10.20.193.102:0/886622366 learned_addr learned my addr 10.20.193.102:0/886622366 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:43.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f99137fe640 1 -- 10.20.193.102:0/886622366 >> v1:10.20.193.102:6789/0 conn(0x7f991408b7f0 legacy=0x7f9914181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9918ecc640 1 -- 10.20.193.102:0/886622366 >> v1:10.20.193.102:6790/0 conn(0x7f991408d8e0 legacy=0x7f99141a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816222792 0 0) 0x7f9914086a10 con 0x7f991408b7f0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): my global_id is 34746 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98f8003620 con 0x7f991408b7f0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969066501 0 0) 0x7f991408bd60 con 0x7f991408d8e0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient(hunting): my global_id is 94244 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9914086a10 con 0x7f991408d8e0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2679357596 0 0) 0x7f98f8003620 con 0x7f991408b7f0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 >> v1:10.20.193.102:6790/0 conn(0x7f991408d8e0 legacy=0x7f99141a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 >> v1:10.20.193.178:6789/0 conn(0x7f991408ce00 legacy=0x7f991408d1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 monclient: found mon.noname-a 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991408b370 con 0x7f991408b7f0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9904003110 con 0x7f991408b7f0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 5 monclient: authenticate success, global_id 34746 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: dump: 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.295 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f99040033e0 con 0x7f991408b7f0 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 1 -- 10.20.193.102:0/886622366 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9904004b50 con 0x7f991408b7f0 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f9912ffd640 10 monclient: dump: 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient: get_monmap_and_config success 2026-02-21T22:29:43.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 4 set_mon_vals no callback set 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 10 monclient: shutdown 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- 10.20.193.102:0/886622366 >> v1:10.20.193.102:6789/0 conn(0x7f991408b7f0 legacy=0x7f9914181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- 10.20.193.102:0/886622366 shutdown_connections 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- 10.20.193.102:0/886622366 >> 10.20.193.102:0/886622366 conn(0x7f9914091a50 msgr2=0x7f9914091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- 10.20.193.102:0/886622366 shutdown_connections 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.288+0000 7f991a956640 1 -- 10.20.193.102:0/886622366 wait complete. 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: build_initial_monmap 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: monmap: 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 Processor -- start 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- start start 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: init 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _reopen_session rank -1 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient(hunting): picked mon.noname-b con 0x7f991408b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient(hunting): picked mon.noname-a con 0x7f991408ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient(hunting): picked mon.noname-c con 0x7f991408d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9914082f50 con 0x7f991408ce00 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9914083230 con 0x7f991408b7f0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991405da00 con 0x7f991408d8e0 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient(hunting): _renew_subs 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient(hunting): authenticate will time out at 1992.532907s 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f99137fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f991408ce00 0x7f9914084740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60348/0 (socket says 10.20.193.102:60348) 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f99137fe640 1 -- 10.20.193.102:0/3755360625 learned_addr learned my addr 10.20.193.102:0/3755360625 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:29:43.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9913fff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f991408b7f0 legacy=0x7f9914152fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:29:43.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f99137fe640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6789/0 conn(0x7f991408ce00 legacy=0x7f9914084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9918ecc640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991408d8e0 legacy=0x7f9914195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98380731 0 0) 0x7f9914082f50 con 0x7f991408ce00 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): my global_id is 34749 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e4003620 con 0x7f991408ce00 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 922369007 0 0) 0x7f991405da00 con 0x7f991408d8e0 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): my global_id is 94247 2026-02-21T22:29:43.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9914082f50 con 0x7f991408d8e0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1820330388 0 0) 0x7f9914082f50 con 0x7f991408d8e0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f991405da00 con 0x7f991408d8e0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9908004900 con 0x7f991408d8e0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2103706109 0 0) 0x7f98e4003620 con 0x7f991408ce00 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9914082f50 con 0x7f991408ce00 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9904002b70 con 0x7f991408ce00 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient(hunting): dump: 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1943892902 0 0) 0x7f991405da00 con 0x7f991408d8e0 2026-02-21T22:29:43.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.178:6789/0 conn(0x7f991408b7f0 legacy=0x7f9914152fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6789/0 conn(0x7f991408ce00 legacy=0x7f9914084740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 monclient: found mon.c 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9914081780 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: _finish_auth 0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 5 monclient: authenticate success, global_id 94247 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _renew_subs 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9914082c70 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _renew_subs 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9914085020 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9908004ed0 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9908005230 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f99127fc640 4 set_mon_vals no callback set 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.292+0000 7f9910ff9640 10 monclient: dump: 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f991a956640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f99140008e0 con 0x7f991407a370 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f99080054c0 con 0x7f991408d8e0 2026-02-21T22:29:43.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(676..676 src has 395..676) ==== 3947+0+0 (unknown 656618030 0 0) 0x7f990804db70 con 0x7f991408d8e0 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f99137fe640 10 monclient: get_auth_request con 0x7f98e403d610 auth_method 0 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9913fff640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407a370 legacy=0x7f991407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499442250 0 0) 0x7f99140008e0 con 0x7f991407a370 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient(hunting): my global_id is 94250 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f991405da00 con 0x7f991407a370 2026-02-21T22:29:43.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2378523051 0 0) 0x7f991405da00 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f99140008e0 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9900004360 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient: dump: 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 867515801 0 0) 0x7f99140008e0 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9914090810 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f9900006790 con 0x7f991407a370 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407a370 legacy=0x7f991407a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.296+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407a370 legacy=0x7f991407a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:43.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.300+0000 7f991a956640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:29:43.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.300+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f99140909c0 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9913fff640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407ae50 legacy=0x7f9914197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1496148326 0 0) 0x7f99140909c0 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient(hunting): my global_id is 94253 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99140008e0 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1258323929 0 0) 0x7f99140008e0 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f99140909c0 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9900004b60 con 0x7f991407ae50 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient: dump: 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:29:43.310 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3234968170 0 0) 0x7f99140909c0 con 0x7f991407ae50 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f991407b250 con 0x7f991407ae50 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4503 (unknown 0 0 3932048004) 0x7f9900006940 con 0x7f991407ae50 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407ae50 legacy=0x7f9914197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9910ff9640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991407ae50 legacy=0x7f9914197a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:29:43.311 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 54, 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 129580, 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.312 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.313 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:29:43.314 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:29:43.315 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.316 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:29:43.317 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.318 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:29:43.319 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.178:6800/2549949439 conn(0x7f98e403d610 legacy=0x7f98e405dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 10 monclient: shutdown 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 >> v1:10.20.193.102:6790/0 conn(0x7f991408d8e0 legacy=0x7f9914195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9918ecc640 1 -- 10.20.193.102:0/3755360625 reap_dead start 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f9918ecc640 1 -- 10.20.193.102:0/3755360625 reap_dead start 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 shutdown_connections 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 >> 10.20.193.102:0/3755360625 conn(0x7f9914091a50 msgr2=0x7f991405a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 shutdown_connections 2026-02-21T22:29:43.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:29:43.304+0000 7f991a956640 1 -- 10.20.193.102:0/3755360625 wait complete. 2026-02-21T22:29:43.321 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:29:43.321 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:29:43.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:43.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:45.283 INFO:tasks.rados.rados.0.trial178.stdout:3258: done (3 left) 2026-02-21T22:29:45.283 INFO:tasks.rados.rados.0.trial178.stdout:3259: done (2 left) 2026-02-21T22:29:45.283 INFO:tasks.rados.rados.0.trial178.stdout:3260: done (1 left) 2026-02-21T22:29:45.283 INFO:tasks.rados.rados.0.trial178.stdout:3261: done (0 left) 2026-02-21T22:29:45.284 INFO:tasks.rados.rados.0.trial178.stdout:3262: write (excl) oid 41 current snap is 327 2026-02-21T22:29:45.284 INFO:tasks.rados.rados.0.trial178.stdout:3262: seq_num 1146 ranges {460822=488304,1378129=517393,2504632=405962} 2026-02-21T22:29:45.290 INFO:tasks.rados.rados.0.trial178.stdout:3262: writing trial17818531-41 from 460822 to 949126 tid 1 2026-02-21T22:29:45.296 INFO:tasks.rados.rados.0.trial178.stdout:3262: writing trial17818531-41 from 1378129 to 1895522 tid 2 2026-02-21T22:29:45.303 INFO:tasks.rados.rados.0.trial178.stdout:3262: writing trial17818531-41 from 2504632 to 2910594 tid 3 2026-02-21T22:29:45.303 INFO:tasks.rados.rados.0.trial178.stdout:3263: write (excl) oid 28 current snap is 327 2026-02-21T22:29:45.303 INFO:tasks.rados.rados.0.trial178.stdout:3263: seq_num 1147 ranges {477629=563553,1836611=777404,3048408=229221} 2026-02-21T22:29:45.310 INFO:tasks.rados.rados.0.trial178.stdout:3263: writing trial17818531-28 from 477629 to 1041182 tid 1 2026-02-21T22:29:45.321 INFO:tasks.rados.rados.0.trial178.stdout:3263: writing trial17818531-28 from 1836611 to 2614015 tid 2 2026-02-21T22:29:45.326 INFO:tasks.rados.rados.0.trial178.stdout:3263: writing trial17818531-28 from 3048408 to 3277629 tid 3 2026-02-21T22:29:45.326 INFO:tasks.rados.rados.0.trial178.stdout:3264: snap_create 2026-02-21T22:29:45.326 INFO:tasks.rados.rados.0.trial178.stdout:3262: finishing write tid 1 to trial17818531-41 2026-02-21T22:29:45.326 INFO:tasks.rados.rados.0.trial178.stdout:3262: finishing write tid 2 to trial17818531-41 2026-02-21T22:29:45.326 INFO:tasks.rados.rados.0.trial178.stdout:3262: finishing write tid 3 to trial17818531-41 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: finishing write tid 4 to trial17818531-41 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: finishing write tid 5 to trial17818531-41 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: oid 41 updating version 0 to 1247 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: oid 41 version 1247 is already newer than 1246 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: oid 41 updating version 1247 to 1248 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: oid 41 updating version 1248 to 1249 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1249 (ObjNum 1146 snap 327 seq_num 1146) dirty exists 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3262: left oid 41 (ObjNum 1146 snap 327 seq_num 1146) 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3263: finishing write tid 1 to trial17818531-28 2026-02-21T22:29:45.327 INFO:tasks.rados.rados.0.trial178.stdout:3263: finishing write tid 2 to trial17818531-28 2026-02-21T22:29:45.328 INFO:tasks.rados.rados.0.trial178.stdout:3263: finishing write tid 3 to trial17818531-28 2026-02-21T22:29:45.328 INFO:tasks.rados.rados.0.trial178.stdout:3263: finishing write tid 4 to trial17818531-28 2026-02-21T22:29:45.328 INFO:tasks.rados.rados.0.trial178.stdout:3263: finishing write tid 5 to trial17818531-28 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:3263: oid 28 updating version 0 to 1204 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:3263: oid 28 version 1204 is already newer than 1203 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:3263: oid 28 version 1204 is already newer than 1202 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:3263: oid 28 updating version 1204 to 1205 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1205 (ObjNum 1147 snap 327 seq_num 1147) dirty exists 2026-02-21T22:29:45.329 INFO:tasks.rados.rados.0.trial178.stdout:3263: left oid 28 (ObjNum 1147 snap 327 seq_num 1147) 2026-02-21T22:29:46.287 INFO:tasks.rados.rados.0.trial178.stdout:3262: done (2 left) 2026-02-21T22:29:46.287 INFO:tasks.rados.rados.0.trial178.stdout:3263: done (1 left) 2026-02-21T22:29:46.287 INFO:tasks.rados.rados.0.trial178.stdout:3264: done (0 left) 2026-02-21T22:29:46.287 INFO:tasks.rados.rados.0.trial178.stdout:3265: write oid 3 current snap is 328 2026-02-21T22:29:46.287 INFO:tasks.rados.rados.0.trial178.stdout:3265: seq_num 1148 ranges {494436=755155,2011446=521062,3075831=218605} 2026-02-21T22:29:46.296 INFO:tasks.rados.rados.0.trial178.stdout:3265: writing trial17818531-3 from 494436 to 1249591 tid 1 2026-02-21T22:29:46.305 INFO:tasks.rados.rados.0.trial178.stdout:3265: writing trial17818531-3 from 2011446 to 2532508 tid 2 2026-02-21T22:29:46.310 INFO:tasks.rados.rados.0.trial178.stdout:3265: writing trial17818531-3 from 3075831 to 3294436 tid 3 2026-02-21T22:29:46.310 INFO:tasks.rados.rados.0.trial178.stdout:3266: rollback oid 10 current snap is 328 2026-02-21T22:29:46.310 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 326 2026-02-21T22:29:46.310 INFO:tasks.rados.rados.0.trial178.stdout:3265: finishing write tid 1 to trial17818531-3 2026-02-21T22:29:46.311 INFO:tasks.rados.rados.0.trial178.stdout:3265: finishing write tid 2 to trial17818531-3 2026-02-21T22:29:46.311 INFO:tasks.rados.rados.0.trial178.stdout:3267: rollback oid 11 current snap is 328 2026-02-21T22:29:46.311 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 327 2026-02-21T22:29:46.311 INFO:tasks.rados.rados.0.trial178.stdout:3268: delete oid 7 current snap is 328 2026-02-21T22:29:46.312 INFO:tasks.rados.rados.0.trial178.stdout:3265: finishing write tid 3 to trial17818531-3 2026-02-21T22:29:46.313 INFO:tasks.rados.rados.0.trial178.stdout:3266: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:29:46.313 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 0 (ObjNum 129 snap 0 seq_num 100571320) dirty dne 2026-02-21T22:29:46.315 INFO:tasks.rados.rados.0.trial178.stdout:3266: done (3 left) 2026-02-21T22:29:46.315 INFO:tasks.rados.rados.0.trial178.stdout:3268: done (2 left) 2026-02-21T22:29:46.315 INFO:tasks.rados.rados.0.trial178.stdout:3269: snap_remove snap 324 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3269: done (2 left) 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3270: rollback oid 25 current snap is 328 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 25 to 326 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: finishing write tid 4 to trial17818531-3 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: finishing write tid 5 to trial17818531-3 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: oid 3 updating version 0 to 1570 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: oid 3 updating version 1570 to 1571 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: oid 3 version 1571 is already newer than 1569 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: oid 3 version 1571 is already newer than 1568 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1571 (ObjNum 1148 snap 328 seq_num 1148) dirty exists 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: left oid 3 (ObjNum 1148 snap 328 seq_num 1148) 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3267: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3267: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3265: done (2 left) 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3271: read oid 15 snap -1 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3271: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3272: rollback oid 30 current snap is 328 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 30 to 326 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:3267: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:29:47.198 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1575 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:29:47.199 INFO:tasks.rados.rados.0.trial178.stdout:3267: done (3 left) 2026-02-21T22:29:47.199 INFO:tasks.rados.rados.0.trial178.stdout:3273: delete oid 24 current snap is 328 2026-02-21T22:29:47.203 INFO:tasks.rados.rados.0.trial178.stdout:3273: done (3 left) 2026-02-21T22:29:47.203 INFO:tasks.rados.rados.0.trial178.stdout:3274: read oid 23 snap -1 2026-02-21T22:29:47.204 INFO:tasks.rados.rados.0.trial178.stdout:3274: expect deleted 2026-02-21T22:29:47.204 INFO:tasks.rados.rados.0.trial178.stdout:3275: read oid 37 snap -1 2026-02-21T22:29:47.204 INFO:tasks.rados.rados.0.trial178.stdout:3275: expect deleted 2026-02-21T22:29:47.204 INFO:tasks.rados.rados.0.trial178.stdout:3276: write (excl) oid 47 current snap is 328 2026-02-21T22:29:47.204 INFO:tasks.rados.rados.0.trial178.stdout:3276: seq_num 1151 ranges {544857=580902,1670539=668389,3041747=303110} 2026-02-21T22:29:47.207 INFO:tasks.rados.rados.0.trial178.stdout:3276: writing trial17818531-47 from 544857 to 1125759 tid 1 2026-02-21T22:29:47.212 INFO:tasks.rados.rados.0.trial178.stdout:3276: writing trial17818531-47 from 1670539 to 2338928 tid 2 2026-02-21T22:29:47.216 INFO:tasks.rados.rados.0.trial178.stdout:3276: writing trial17818531-47 from 3041747 to 3344857 tid 3 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3277: copy_from oid 34 from oid 50 current snap is 328 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3270: finishing rollback tid 0 to trial17818531-25 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3272: finishing rollback tid 0 to trial17818531-30 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3272: finishing rollback tid 1 to trial17818531-30 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3272: finishing rollback tid 2 to trial17818531-30 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1598 (ObjNum 1019 snap 283 seq_num 1019) dirty exists 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3270: finishing rollback tid 1 to trial17818531-25 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3270: finishing rollback tid 2 to trial17818531-25 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1475 (ObjNum 1141 snap 326 seq_num 1141) dirty exists 2026-02-21T22:29:47.217 INFO:tasks.rados.rados.0.trial178.stdout:3271: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: finishing write tid 1 to trial17818531-47 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: finishing write tid 2 to trial17818531-47 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: finishing write tid 3 to trial17818531-47 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: finishing write tid 4 to trial17818531-47 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: finishing write tid 5 to trial17818531-47 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: oid 47 updating version 0 to 1787 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: oid 47 version 1787 is already newer than 1786 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: oid 47 version 1787 is already newer than 1784 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: oid 47 version 1787 is already newer than 1785 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1787 (ObjNum 1151 snap 328 seq_num 1151) dirty exists 2026-02-21T22:29:47.319 INFO:tasks.rados.rados.0.trial178.stdout:3276: left oid 47 (ObjNum 1151 snap 328 seq_num 1151) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3277: finishing copy_from racing read to trial17818531-34 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3277: finishing copy_from to trial17818531-34 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1617 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3270: done (6 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3271: done (5 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3272: done (4 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3274: done (3 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3275: done (2 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3276: done (1 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3277: done (0 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3278: delete oid 2 current snap is 328 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3278: done (0 left) 2026-02-21T22:29:47.320 INFO:tasks.rados.rados.0.trial178.stdout:3279: read oid 48 snap -1 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3279: expect (ObjNum 1119 snap 320 seq_num 1119) 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3280: copy_from oid 19 from oid 36 current snap is 328 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3281: read oid 39 snap -1 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3281: expect (ObjNum 1115 snap 318 seq_num 1115) 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3282: rollback oid 2 current snap is 328 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 327 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3283: write oid 26 current snap is 328 2026-02-21T22:29:47.321 INFO:tasks.rados.rados.0.trial178.stdout:3283: seq_num 1153 ranges {578471=731400,1787503=672058,3096593=281878} 2026-02-21T22:29:47.329 INFO:tasks.rados.rados.0.trial178.stdout:3283: writing trial17818531-26 from 578471 to 1309871 tid 1 2026-02-21T22:29:47.337 INFO:tasks.rados.rados.0.trial178.stdout:3283: writing trial17818531-26 from 1787503 to 2459561 tid 2 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3283: writing trial17818531-26 from 3096593 to 3378471 tid 3 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3284: snap_create 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3280: finishing copy_from to trial17818531-19 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3280: got expected ENOENT (src dne) 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3282: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1218 (ObjNum 289 snap 0 seq_num 1) dirty dne 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3280: finishing copy_from racing read to trial17818531-19 2026-02-21T22:29:47.344 INFO:tasks.rados.rados.0.trial178.stdout:3281: expect (ObjNum 1115 snap 318 seq_num 1115) 2026-02-21T22:29:47.409 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:47.464 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1173244 2026-02-21T22:29:47.497 INFO:tasks.rados.rados.0.trial178.stdout:3279: expect (ObjNum 1119 snap 320 seq_num 1119) 2026-02-21T22:29:47.522 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:47.549 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: finishing write tid 1 to trial17818531-26 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: finishing write tid 2 to trial17818531-26 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: finishing write tid 3 to trial17818531-26 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: finishing write tid 4 to trial17818531-26 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: finishing write tid 5 to trial17818531-26 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: oid 26 updating version 0 to 1223 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: oid 26 updating version 1223 to 1226 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: oid 26 version 1226 is already newer than 1224 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: oid 26 version 1226 is already newer than 1225 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1226 (ObjNum 1153 snap 328 seq_num 1153) dirty exists 2026-02-21T22:29:47.576 INFO:tasks.rados.rados.0.trial178.stdout:3283: left oid 26 (ObjNum 1153 snap 328 seq_num 1153) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3279: done (5 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3280: done (4 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3281: done (3 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3282: done (2 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3283: done (1 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3284: done (0 left) 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3285: rollback oid 12 current snap is 329 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 322 2026-02-21T22:29:48.293 INFO:tasks.rados.rados.0.trial178.stdout:3286: snap_remove snap 326 2026-02-21T22:29:48.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:48.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3286: done (1 left) 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3287: read oid 23 snap 327 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3287: expect deleted 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3288: read oid 11 snap -1 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3288: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:49.293 INFO:tasks.rados.rados.0.trial178.stdout:3289: write (excl) oid 25 current snap is 329 2026-02-21T22:29:49.294 INFO:tasks.rados.rados.0.trial178.stdout:3285: finishing rollback tid 0 to trial17818531-12 2026-02-21T22:29:49.294 INFO:tasks.rados.rados.0.trial178.stdout:3285: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:29:49.294 INFO:tasks.rados.rados.0.trial178.stdout:3289: seq_num 1154 ranges {595278=406649,1729632=532069,3007663=387615} 2026-02-21T22:29:49.300 INFO:tasks.rados.rados.0.trial178.stdout:3289: writing trial17818531-25 from 595278 to 1001927 tid 1 2026-02-21T22:29:49.309 INFO:tasks.rados.rados.0.trial178.stdout:3289: writing trial17818531-25 from 1729632 to 2261701 tid 2 2026-02-21T22:29:49.316 INFO:tasks.rados.rados.0.trial178.stdout:3289: writing trial17818531-25 from 3007663 to 3395278 tid 3 2026-02-21T22:29:49.316 INFO:tasks.rados.rados.0.trial178.stdout:3290: snap_create 2026-02-21T22:29:49.316 INFO:tasks.rados.rados.0.trial178.stdout:3285: finishing rollback tid 2 to trial17818531-12 2026-02-21T22:29:49.317 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1615 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:49.317 INFO:tasks.rados.rados.0.trial178.stdout:3288: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:49.343 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:49.372 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 448467 2026-02-21T22:29:49.399 INFO:tasks.rados.rados.0.trial178.stdout:3289: finishing write tid 1 to trial17818531-25 2026-02-21T22:29:49.399 INFO:tasks.rados.rados.0.trial178.stdout:3289: finishing write tid 2 to trial17818531-25 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: finishing write tid 3 to trial17818531-25 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: finishing write tid 4 to trial17818531-25 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: finishing write tid 5 to trial17818531-25 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: oid 25 updating version 0 to 1478 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: oid 25 version 1478 is already newer than 1477 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: oid 25 updating version 1478 to 1479 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: oid 25 version 1479 is already newer than 1476 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1479 (ObjNum 1154 snap 329 seq_num 1154) dirty exists 2026-02-21T22:29:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3289: left oid 25 (ObjNum 1154 snap 329 seq_num 1154) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3285: done (4 left) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3287: done (3 left) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3288: done (2 left) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3289: done (1 left) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3290: done (0 left) 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3291: write (excl) oid 19 current snap is 330 2026-02-21T22:29:50.299 INFO:tasks.rados.rados.0.trial178.stdout:3291: seq_num 1155 ranges {612085=481898,1788114=675727,3035086=376999} 2026-02-21T22:29:50.307 INFO:tasks.rados.rados.0.trial178.stdout:3291: writing trial17818531-19 from 612085 to 1093983 tid 1 2026-02-21T22:29:50.316 INFO:tasks.rados.rados.0.trial178.stdout:3291: writing trial17818531-19 from 1788114 to 2463841 tid 2 2026-02-21T22:29:50.323 INFO:tasks.rados.rados.0.trial178.stdout:3291: writing trial17818531-19 from 3035086 to 3412085 tid 3 2026-02-21T22:29:50.323 INFO:tasks.rados.rados.0.trial178.stdout:3292: snap_create 2026-02-21T22:29:50.323 INFO:tasks.rados.rados.0.trial178.stdout:3291: finishing write tid 1 to trial17818531-19 2026-02-21T22:29:50.323 INFO:tasks.rados.rados.0.trial178.stdout:3291: finishing write tid 2 to trial17818531-19 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: finishing write tid 3 to trial17818531-19 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: finishing write tid 4 to trial17818531-19 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: finishing write tid 5 to trial17818531-19 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: oid 19 updating version 0 to 1796 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: oid 19 version 1796 is already newer than 1795 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: oid 19 version 1796 is already newer than 1794 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: oid 19 version 1796 is already newer than 1793 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1796 (ObjNum 1155 snap 330 seq_num 1155) dirty exists 2026-02-21T22:29:50.325 INFO:tasks.rados.rados.0.trial178.stdout:3291: left oid 19 (ObjNum 1155 snap 330 seq_num 1155) 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3291: done (1 left) 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3292: done (0 left) 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3293: copy_from oid 43 from oid 31 current snap is 331 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3294: read oid 37 snap -1 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3294: expect deleted 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3295: write oid 24 current snap is 331 2026-02-21T22:29:51.300 INFO:tasks.rados.rados.0.trial178.stdout:3295: seq_num 1156 ranges {628892=673500,1962949=535738,3178862=250030} 2026-02-21T22:29:51.309 INFO:tasks.rados.rados.0.trial178.stdout:3295: writing trial17818531-24 from 628892 to 1302392 tid 1 2026-02-21T22:29:51.317 INFO:tasks.rados.rados.0.trial178.stdout:3295: writing trial17818531-24 from 1962949 to 2498687 tid 2 2026-02-21T22:29:51.323 INFO:tasks.rados.rados.0.trial178.stdout:3295: writing trial17818531-24 from 3178862 to 3428892 tid 3 2026-02-21T22:29:51.324 INFO:tasks.rados.rados.0.trial178.stdout:3296: write (excl) oid 30 current snap is 331 2026-02-21T22:29:51.324 INFO:tasks.rados.rados.0.trial178.stdout:3296: seq_num 1157 ranges {645699=748749,2021431=679396,3206285=239414} 2026-02-21T22:29:51.333 INFO:tasks.rados.rados.0.trial178.stdout:3296: writing trial17818531-30 from 645699 to 1394448 tid 1 2026-02-21T22:29:51.341 INFO:tasks.rados.rados.0.trial178.stdout:3296: writing trial17818531-30 from 2021431 to 2700827 tid 2 2026-02-21T22:29:51.347 INFO:tasks.rados.rados.0.trial178.stdout:3296: writing trial17818531-30 from 3206285 to 3445699 tid 3 2026-02-21T22:29:51.347 INFO:tasks.rados.rados.0.trial178.stdout:3297: write (excl) oid 44 current snap is 331 2026-02-21T22:29:51.347 INFO:tasks.rados.rados.0.trial178.stdout:3297: seq_num 1158 ranges {662506=423998,1563560=539407,2717355=490285} 2026-02-21T22:29:51.351 INFO:tasks.rados.rados.0.trial178.stdout:3297: writing trial17818531-44 from 662506 to 1086504 tid 1 2026-02-21T22:29:51.355 INFO:tasks.rados.rados.0.trial178.stdout:3297: writing trial17818531-44 from 1563560 to 2102967 tid 2 2026-02-21T22:29:51.359 INFO:tasks.rados.rados.0.trial178.stdout:3297: writing trial17818531-44 from 2717355 to 3207640 tid 3 2026-02-21T22:29:51.359 INFO:tasks.rados.rados.0.trial178.stdout:3298: write (excl) oid 20 current snap is 331 2026-02-21T22:29:51.359 INFO:tasks.rados.rados.0.trial178.stdout:3298: seq_num 1159 ranges {679313=499247,1622042=683065,2744778=701557} 2026-02-21T22:29:51.363 INFO:tasks.rados.rados.0.trial178.stdout:3298: writing trial17818531-20 from 679313 to 1178560 tid 1 2026-02-21T22:29:51.368 INFO:tasks.rados.rados.0.trial178.stdout:3298: writing trial17818531-20 from 1622042 to 2305107 tid 2 2026-02-21T22:29:51.372 INFO:tasks.rados.rados.0.trial178.stdout:3298: writing trial17818531-20 from 2744778 to 3446335 tid 3 2026-02-21T22:29:51.372 INFO:tasks.rados.rados.0.trial178.stdout:3299: copy_from oid 22 from oid 36 current snap is 331 2026-02-21T22:29:51.372 INFO:tasks.rados.rados.0.trial178.stdout:3295: finishing write tid 1 to trial17818531-24 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3293: finishing copy_from racing read to trial17818531-43 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3293: finishing copy_from to trial17818531-43 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1464 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3295: finishing write tid 2 to trial17818531-24 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3295: finishing write tid 3 to trial17818531-24 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3295: finishing write tid 4 to trial17818531-24 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3293: done (6 left) 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3294: done (5 left) 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3300: write oid 2 current snap is 331 2026-02-21T22:29:51.373 INFO:tasks.rados.rados.0.trial178.stdout:3300: seq_num 1160 ranges {696120=574496,1680524=543076,2888554=512829} 2026-02-21T22:29:51.377 INFO:tasks.rados.rados.0.trial178.stdout:3300: writing trial17818531-2 from 696120 to 1270616 tid 1 2026-02-21T22:29:51.381 INFO:tasks.rados.rados.0.trial178.stdout:3300: writing trial17818531-2 from 1680524 to 2223600 tid 2 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3300: writing trial17818531-2 from 2888554 to 3401383 tid 3 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3301: copy_from oid 45 from oid 25 current snap is 331 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3295: finishing write tid 5 to trial17818531-24 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3295: oid 24 updating version 0 to 1800 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3295: oid 24 version 1800 is already newer than 1799 2026-02-21T22:29:51.386 INFO:tasks.rados.rados.0.trial178.stdout:3295: oid 24 version 1800 is already newer than 1797 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3295: oid 24 version 1800 is already newer than 1798 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1800 (ObjNum 1156 snap 331 seq_num 1156) dirty exists 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3295: left oid 24 (ObjNum 1156 snap 331 seq_num 1156) 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3296: finishing write tid 1 to trial17818531-30 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3295: done (6 left) 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3302: read oid 15 snap -1 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3302: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:51.387 INFO:tasks.rados.rados.0.trial178.stdout:3303: snap_remove snap 322 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3303: done (7 left) 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3304: read oid 42 snap -1 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3304: expect (ObjNum 1136 snap 325 seq_num 1136) 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3305: read oid 21 snap -1 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3305: expect (ObjNum 1137 snap 325 seq_num 1137) 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3306: snap_create 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: finishing write tid 2 to trial17818531-30 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: finishing write tid 3 to trial17818531-30 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: finishing write tid 4 to trial17818531-30 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: finishing write tid 5 to trial17818531-30 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: oid 30 updating version 0 to 1600 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: oid 30 updating version 1600 to 1602 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: oid 30 version 1602 is already newer than 1599 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: oid 30 version 1602 is already newer than 1601 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1602 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3296: left oid 30 (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3297: finishing write tid 1 to trial17818531-44 2026-02-21T22:29:52.201 INFO:tasks.rados.rados.0.trial178.stdout:3297: finishing write tid 2 to trial17818531-44 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: finishing write tid 3 to trial17818531-44 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: finishing write tid 4 to trial17818531-44 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: finishing write tid 5 to trial17818531-44 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: oid 44 updating version 0 to 1618 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: oid 44 version 1618 is already newer than 1616 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: oid 44 updating version 1618 to 1619 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: oid 44 version 1619 is already newer than 1617 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1619 (ObjNum 1158 snap 331 seq_num 1158) dirty exists 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3297: left oid 44 (ObjNum 1158 snap 331 seq_num 1158) 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: finishing write tid 1 to trial17818531-20 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: finishing write tid 2 to trial17818531-20 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: finishing write tid 3 to trial17818531-20 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: finishing write tid 4 to trial17818531-20 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: finishing write tid 5 to trial17818531-20 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: oid 20 updating version 0 to 1256 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: oid 20 version 1256 is already newer than 1255 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: oid 20 updating version 1256 to 1257 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: oid 20 updating version 1257 to 1258 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1258 (ObjNum 1159 snap 331 seq_num 1159) dirty exists 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3298: left oid 20 (ObjNum 1159 snap 331 seq_num 1159) 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3299: finishing copy_from to trial17818531-22 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3299: got expected ENOENT (src dne) 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3299: finishing copy_from racing read to trial17818531-22 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3300: finishing write tid 1 to trial17818531-2 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3300: finishing write tid 2 to trial17818531-2 2026-02-21T22:29:52.202 INFO:tasks.rados.rados.0.trial178.stdout:3302: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:29:52.211 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1097684 2026-02-21T22:29:52.273 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3300: finishing write tid 3 to trial17818531-2 2026-02-21T22:29:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3300: finishing write tid 4 to trial17818531-2 2026-02-21T22:29:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3300: finishing write tid 5 to trial17818531-2 2026-02-21T22:29:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3300: oid 2 updating version 0 to 1228 2026-02-21T22:29:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3300: oid 2 updating version 1228 to 1230 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3300: oid 2 version 1230 is already newer than 1227 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3300: oid 2 version 1230 is already newer than 1229 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1230 (ObjNum 1160 snap 331 seq_num 1160) dirty exists 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3300: left oid 2 (ObjNum 1160 snap 331 seq_num 1160) 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3301: finishing copy_from racing read to trial17818531-45 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3301: finishing copy_from to trial17818531-45 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1479 (ObjNum 1154 snap 329 seq_num 1154) dirty exists 2026-02-21T22:29:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3304: expect (ObjNum 1136 snap 325 seq_num 1136) 2026-02-21T22:29:52.332 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 80077 2026-02-21T22:29:52.379 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2384653 2026-02-21T22:29:52.388 INFO:tasks.rados.rados.0.trial178.stdout:3305: expect (ObjNum 1137 snap 325 seq_num 1137) 2026-02-21T22:29:52.407 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2381059 2026-02-21T22:29:52.417 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:53.204 INFO:tasks.rados.rados.0.trial178.stdout:3296: done (9 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3297: done (8 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3298: done (7 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3299: done (6 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3300: done (5 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3301: done (4 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3302: done (3 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3304: done (2 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3305: done (1 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3306: done (0 left) 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3307: write oid 45 current snap is 332 2026-02-21T22:29:53.205 INFO:tasks.rados.rados.0.trial178.stdout:3307: seq_num 1161 ranges {712927=649745,2139006=686734} 2026-02-21T22:29:53.210 INFO:tasks.rados.rados.0.trial178.stdout:3307: writing trial17818531-45 from 712927 to 1362672 tid 1 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3307: writing trial17818531-45 from 2139006 to 2825740 tid 2 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3308: copy_from oid 32 from oid 27 current snap is 332 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3307: finishing write tid 1 to trial17818531-45 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3309: read oid 9 snap -1 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3309: expect deleted 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3310: write (excl) oid 14 current snap is 332 2026-02-21T22:29:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3310: seq_num 1162 ranges {729734=724994,2197488=430392,3227047=302687} 2026-02-21T22:29:53.221 INFO:tasks.rados.rados.0.trial178.stdout:3310: writing trial17818531-14 from 729734 to 1454728 tid 1 2026-02-21T22:29:53.226 INFO:tasks.rados.rados.0.trial178.stdout:3310: writing trial17818531-14 from 2197488 to 2627880 tid 2 2026-02-21T22:29:53.229 INFO:tasks.rados.rados.0.trial178.stdout:3310: writing trial17818531-14 from 3227047 to 3529734 tid 3 2026-02-21T22:29:53.229 INFO:tasks.rados.rados.0.trial178.stdout:3311: write oid 50 current snap is 332 2026-02-21T22:29:53.229 INFO:tasks.rados.rados.0.trial178.stdout:3311: seq_num 1163 ranges {746541=400243,1739617=690403,3138117=408424} 2026-02-21T22:29:53.233 INFO:tasks.rados.rados.0.trial178.stdout:3311: writing trial17818531-50 from 746541 to 1146784 tid 1 2026-02-21T22:29:53.238 INFO:tasks.rados.rados.0.trial178.stdout:3311: writing trial17818531-50 from 1739617 to 2430020 tid 2 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3311: writing trial17818531-50 from 3138117 to 3546541 tid 3 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3312: copy_from oid 42 from oid 10 current snap is 332 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3308: finishing copy_from to trial17818531-32 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3308: got expected ENOENT (src dne) 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3308: finishing copy_from racing read to trial17818531-32 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: finishing write tid 2 to trial17818531-45 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: finishing write tid 3 to trial17818531-45 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: finishing write tid 4 to trial17818531-45 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: oid 45 updating version 0 to 1490 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: oid 45 version 1490 is already newer than 1489 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: oid 45 version 1490 is already newer than 1488 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1490 (ObjNum 1161 snap 332 seq_num 1161) dirty exists 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3307: left oid 45 (ObjNum 1161 snap 332 seq_num 1161) 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: finishing write tid 1 to trial17818531-14 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: finishing write tid 2 to trial17818531-14 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: finishing write tid 3 to trial17818531-14 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: finishing write tid 4 to trial17818531-14 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: finishing write tid 5 to trial17818531-14 2026-02-21T22:29:53.243 INFO:tasks.rados.rados.0.trial178.stdout:3310: oid 14 updating version 0 to 1468 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3310: oid 14 updating version 1468 to 1469 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3310: oid 14 version 1469 is already newer than 1467 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3310: oid 14 updating version 1469 to 1470 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1470 (ObjNum 1162 snap 332 seq_num 1162) dirty exists 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3310: left oid 14 (ObjNum 1162 snap 332 seq_num 1162) 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3311: finishing write tid 1 to trial17818531-50 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3311: finishing write tid 2 to trial17818531-50 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3307: done (5 left) 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3308: done (4 left) 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3309: done (3 left) 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3310: done (2 left) 2026-02-21T22:29:53.244 INFO:tasks.rados.rados.0.trial178.stdout:3313: snap_remove snap 327 2026-02-21T22:29:53.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:53.540 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:53.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:53.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:29:53.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:53.552 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:29:53.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:53.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:54.206 INFO:tasks.rados.rados.0.trial178.stdout:3313: done (2 left) 2026-02-21T22:29:54.206 INFO:tasks.rados.rados.0.trial178.stdout:3314: read oid 13 snap -1 2026-02-21T22:29:54.206 INFO:tasks.rados.rados.0.trial178.stdout:3314: expect deleted 2026-02-21T22:29:54.206 INFO:tasks.rados.rados.0.trial178.stdout:3315: write oid 2 current snap is 332 2026-02-21T22:29:54.206 INFO:tasks.rados.rados.0.trial178.stdout:3315: seq_num 1164 ranges {763348=591845,1914452=434061,2881893=674270} 2026-02-21T22:29:54.211 INFO:tasks.rados.rados.0.trial178.stdout:3315: writing trial17818531-2 from 763348 to 1355193 tid 1 2026-02-21T22:29:54.215 INFO:tasks.rados.rados.0.trial178.stdout:3315: writing trial17818531-2 from 1914452 to 2348513 tid 2 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3315: writing trial17818531-2 from 2881893 to 3556163 tid 3 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3316: snap_create 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3312: finishing copy_from to trial17818531-42 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3312: got expected ENOENT (src dne) 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3312: finishing copy_from racing read to trial17818531-42 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3311: finishing write tid 3 to trial17818531-50 2026-02-21T22:29:54.220 INFO:tasks.rados.rados.0.trial178.stdout:3311: finishing write tid 4 to trial17818531-50 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: finishing write tid 5 to trial17818531-50 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: oid 50 updating version 0 to 1620 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: oid 50 version 1620 is already newer than 1618 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: oid 50 updating version 1620 to 1621 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: oid 50 version 1621 is already newer than 1619 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1621 (ObjNum 1163 snap 332 seq_num 1163) dirty exists 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3311: left oid 50 (ObjNum 1163 snap 332 seq_num 1163) 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3315: finishing write tid 1 to trial17818531-2 2026-02-21T22:29:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3315: finishing write tid 2 to trial17818531-2 2026-02-21T22:29:54.232 INFO:tasks.rados.rados.0.trial178.stdout:3315: finishing write tid 3 to trial17818531-2 2026-02-21T22:29:54.232 INFO:tasks.rados.rados.0.trial178.stdout:3315: finishing write tid 4 to trial17818531-2 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: finishing write tid 5 to trial17818531-2 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: oid 2 updating version 0 to 1232 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: oid 2 version 1232 is already newer than 1231 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: oid 2 updating version 1232 to 1234 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: oid 2 version 1234 is already newer than 1233 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1234 (ObjNum 1164 snap 332 seq_num 1164) dirty exists 2026-02-21T22:29:54.233 INFO:tasks.rados.rados.0.trial178.stdout:3315: left oid 2 (ObjNum 1164 snap 332 seq_num 1164) 2026-02-21T22:29:55.206 INFO:tasks.rados.rados.0.trial178.stdout:3311: done (4 left) 2026-02-21T22:29:55.206 INFO:tasks.rados.rados.0.trial178.stdout:3312: done (3 left) 2026-02-21T22:29:55.207 INFO:tasks.rados.rados.0.trial178.stdout:3314: done (2 left) 2026-02-21T22:29:55.207 INFO:tasks.rados.rados.0.trial178.stdout:3315: done (1 left) 2026-02-21T22:29:55.207 INFO:tasks.rados.rados.0.trial178.stdout:3316: done (0 left) 2026-02-21T22:29:55.207 INFO:tasks.rados.rados.0.trial178.stdout:3317: delete oid 17 current snap is 333 2026-02-21T22:29:55.212 INFO:tasks.rados.rados.0.trial178.stdout:3317: done (0 left) 2026-02-21T22:29:55.212 INFO:tasks.rados.rados.0.trial178.stdout:3318: snap_remove snap 328 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3318: done (0 left) 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3319: copy_from oid 43 from oid 11 current snap is 333 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3320: read oid 19 snap 332 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3320: expect (ObjNum 1155 snap 330 seq_num 1155) 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3321: write (excl) oid 17 current snap is 333 2026-02-21T22:29:56.212 INFO:tasks.rados.rados.0.trial178.stdout:3321: seq_num 1166 ranges {796962=742343,2031416=437730,2936739=413167} 2026-02-21T22:29:56.222 INFO:tasks.rados.rados.0.trial178.stdout:3321: writing trial17818531-17 from 796962 to 1539305 tid 1 2026-02-21T22:29:56.228 INFO:tasks.rados.rados.0.trial178.stdout:3321: writing trial17818531-17 from 2031416 to 2469146 tid 2 2026-02-21T22:29:56.234 INFO:tasks.rados.rados.0.trial178.stdout:3321: writing trial17818531-17 from 2936739 to 3349906 tid 3 2026-02-21T22:29:56.234 INFO:tasks.rados.rados.0.trial178.stdout:3322: rollback oid 28 current snap is 333 2026-02-21T22:29:56.234 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 332 2026-02-21T22:29:56.234 INFO:tasks.rados.rados.0.trial178.stdout:3320: expect (ObjNum 1155 snap 330 seq_num 1155) 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:3319: finishing copy_from racing read to trial17818531-43 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:3319: finishing copy_from to trial17818531-43 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1575 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:3321: finishing write tid 1 to trial17818531-17 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:3321: finishing write tid 2 to trial17818531-17 2026-02-21T22:29:56.380 INFO:tasks.rados.rados.0.trial178.stdout:3321: finishing write tid 3 to trial17818531-17 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: finishing write tid 4 to trial17818531-17 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: finishing write tid 5 to trial17818531-17 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: oid 17 updating version 0 to 1271 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: oid 17 updating version 1271 to 1273 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: oid 17 updating version 1273 to 1274 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: oid 17 version 1274 is already newer than 1272 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1274 (ObjNum 1166 snap 333 seq_num 1166) dirty exists 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3321: left oid 17 (ObjNum 1166 snap 333 seq_num 1166) 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3322: finishing rollback tid 0 to trial17818531-28 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3322: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3322: finishing rollback tid 2 to trial17818531-28 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1208 (ObjNum 1147 snap 327 seq_num 1147) dirty exists 2026-02-21T22:29:56.381 INFO:tasks.rados.rados.0.trial178.stdout:3319: done (3 left) 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3320: done (2 left) 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3321: done (1 left) 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3322: done (0 left) 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3323: read oid 49 snap -1 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3323: expect (ObjNum 1080 snap 313 seq_num 1080) 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3324: rollback oid 40 current snap is 333 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 331 2026-02-21T22:29:56.382 INFO:tasks.rados.rados.0.trial178.stdout:3325: snap_create 2026-02-21T22:29:56.384 INFO:tasks.rados.rados.0.trial178.stdout:3324: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:29:56.384 INFO:tasks.rados.rados.0.trial178.stdout:3324: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:29:56.384 INFO:tasks.rados.rados.0.trial178.stdout:3324: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:29:56.384 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1624 (ObjNum 1134 snap 325 seq_num 1134) dirty exists 2026-02-21T22:29:56.384 INFO:tasks.rados.rados.0.trial178.stdout:3323: expect (ObjNum 1080 snap 313 seq_num 1080) 2026-02-21T22:29:56.386 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1176440 2026-02-21T22:29:57.201 INFO:tasks.rados.rados.0.trial178.stdout:3323: done (2 left) 2026-02-21T22:29:57.201 INFO:tasks.rados.rados.0.trial178.stdout:3324: done (1 left) 2026-02-21T22:29:57.201 INFO:tasks.rados.rados.0.trial178.stdout:3325: done (0 left) 2026-02-21T22:29:57.202 INFO:tasks.rados.rados.0.trial178.stdout:3326: rollback oid 50 current snap is 334 2026-02-21T22:29:57.202 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 50 to 331 2026-02-21T22:29:57.202 INFO:tasks.rados.rados.0.trial178.stdout:3327: copy_from oid 7 from oid 21 current snap is 334 2026-02-21T22:29:57.202 INFO:tasks.rados.rados.0.trial178.stdout:3328: write (excl) oid 29 current snap is 334 2026-02-21T22:29:57.202 INFO:tasks.rados.rados.0.trial178.stdout:3328: seq_num 1167 ranges {413769=417592,1573545=697741,2847809=624439} 2026-02-21T22:29:57.206 INFO:tasks.rados.rados.0.trial178.stdout:3328: writing trial17818531-29 from 413769 to 831361 tid 1 2026-02-21T22:29:57.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: writing trial17818531-29 from 1573545 to 2271286 tid 2 2026-02-21T22:29:57.224 INFO:tasks.rados.rados.0.trial178.stdout:3328: writing trial17818531-29 from 2847809 to 3472248 tid 3 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3329: write oid 26 current snap is 334 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3326: finishing rollback tid 0 to trial17818531-50 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3326: finishing rollback tid 1 to trial17818531-50 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3326: finishing rollback tid 2 to trial17818531-50 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1627 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3328: finishing write tid 1 to trial17818531-29 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3327: finishing copy_from racing read to trial17818531-7 2026-02-21T22:29:57.225 INFO:tasks.rados.rados.0.trial178.stdout:3329: seq_num 1168 ranges {430576=492841,1632027=441399,2475232=435711,3365397=265179} 2026-02-21T22:29:57.230 INFO:tasks.rados.rados.0.trial178.stdout:3329: writing trial17818531-26 from 430576 to 923417 tid 1 2026-02-21T22:29:57.238 INFO:tasks.rados.rados.0.trial178.stdout:3329: writing trial17818531-26 from 1632027 to 2073426 tid 2 2026-02-21T22:29:57.244 INFO:tasks.rados.rados.0.trial178.stdout:3329: writing trial17818531-26 from 2475232 to 2910943 tid 3 2026-02-21T22:29:57.249 INFO:tasks.rados.rados.0.trial178.stdout:3329: writing trial17818531-26 from 3365397 to 3630576 tid 4 2026-02-21T22:29:57.249 INFO:tasks.rados.rados.0.trial178.stdout:3326: done (3 left) 2026-02-21T22:29:57.249 INFO:tasks.rados.rados.0.trial178.stdout:3330: snap_remove snap 333 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3330: done (3 left) 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3331: rollback oid 40 current snap is 334 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 330 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3327: finishing copy_from to trial17818531-7 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1773 (ObjNum 1137 snap 325 seq_num 1137) dirty exists 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: finishing write tid 2 to trial17818531-29 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: finishing write tid 3 to trial17818531-29 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: finishing write tid 4 to trial17818531-29 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: finishing write tid 5 to trial17818531-29 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: oid 29 updating version 0 to 1621 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: oid 29 version 1621 is already newer than 1618 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: oid 29 version 1621 is already newer than 1619 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: oid 29 version 1621 is already newer than 1620 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1621 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3328: left oid 29 (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:29:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 1 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 2 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 3 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 4 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3327: done (3 left) 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3328: done (2 left) 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3332: delete oid 7 current snap is 334 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 5 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: finishing write tid 6 to trial17818531-26 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: oid 26 updating version 0 to 1237 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: oid 26 updating version 1237 to 1238 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: oid 26 updating version 1238 to 1239 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: oid 26 version 1239 is already newer than 1236 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: oid 26 version 1239 is already newer than 1235 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1239 (ObjNum 1168 snap 334 seq_num 1168) dirty exists 2026-02-21T22:29:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3329: left oid 26 (ObjNum 1168 snap 334 seq_num 1168) 2026-02-21T22:29:58.218 INFO:tasks.rados.rados.0.trial178.stdout:3331: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:29:58.218 INFO:tasks.rados.rados.0.trial178.stdout:3329: done (2 left) 2026-02-21T22:29:58.218 INFO:tasks.rados.rados.0.trial178.stdout:3332: done (1 left) 2026-02-21T22:29:58.219 INFO:tasks.rados.rados.0.trial178.stdout:3333: delete oid 19 current snap is 334 2026-02-21T22:29:58.219 INFO:tasks.rados.rados.0.trial178.stdout:3331: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:29:58.219 INFO:tasks.rados.rados.0.trial178.stdout:3331: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:29:58.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1630 (ObjNum 1134 snap 325 seq_num 1134) dirty exists 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:3331: done (1 left) 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:3333: done (0 left) 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:3334: rollback oid 13 current snap is 334 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 13 to 331 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:3335: write oid 48 current snap is 334 2026-02-21T22:29:58.221 INFO:tasks.rados.rados.0.trial178.stdout:3335: seq_num 1171 ranges {480997=434941,1407473=705079,2557501=785880} 2026-02-21T22:29:58.227 INFO:tasks.rados.rados.0.trial178.stdout:3335: writing trial17818531-48 from 480997 to 915938 tid 1 2026-02-21T22:29:58.236 INFO:tasks.rados.rados.0.trial178.stdout:3335: writing trial17818531-48 from 1407473 to 2112552 tid 2 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3335: writing trial17818531-48 from 2557501 to 3343381 tid 3 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3336: rollback oid 2 current snap is 334 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 329 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3334: finishing rollback tid 1 to trial17818531-13 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3335: finishing write tid 1 to trial17818531-48 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3335: finishing write tid 2 to trial17818531-48 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3334: done (2 left) 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3337: read oid 20 snap -1 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3337: expect (ObjNum 1159 snap 331 seq_num 1159) 2026-02-21T22:29:58.244 INFO:tasks.rados.rados.0.trial178.stdout:3338: snap_remove snap 330 2026-02-21T22:29:58.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:29:58.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:29:59.217 INFO:tasks.rados.rados.0.trial178.stdout:3338: done (3 left) 2026-02-21T22:29:59.217 INFO:tasks.rados.rados.0.trial178.stdout:3339: delete oid 19 current snap is 334 2026-02-21T22:29:59.217 INFO:tasks.rados.rados.0.trial178.stdout:3335: finishing write tid 3 to trial17818531-48 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: finishing write tid 4 to trial17818531-48 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: finishing write tid 5 to trial17818531-48 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: oid 48 updating version 0 to 1491 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: oid 48 updating version 1491 to 1492 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: oid 48 updating version 1492 to 1494 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: oid 48 version 1494 is already newer than 1493 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1494 (ObjNum 1171 snap 334 seq_num 1171) dirty exists 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3335: left oid 48 (ObjNum 1171 snap 334 seq_num 1171) 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3336: finishing rollback tid 0 to trial17818531-2 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3336: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1241 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3337: expect (ObjNum 1159 snap 331 seq_num 1159) 2026-02-21T22:29:59.218 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3335: done (3 left) 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3336: done (2 left) 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3337: done (1 left) 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3339: done (0 left) 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3340: write oid 18 current snap is 334 2026-02-21T22:29:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3340: seq_num 1173 ranges {514611=585439,1524437=592395,2895994=408424} 2026-02-21T22:29:59.380 INFO:tasks.rados.rados.0.trial178.stdout:3340: writing trial17818531-18 from 514611 to 1100050 tid 1 2026-02-21T22:29:59.387 INFO:tasks.rados.rados.0.trial178.stdout:3340: writing trial17818531-18 from 1524437 to 2116832 tid 2 2026-02-21T22:29:59.395 INFO:tasks.rados.rados.0.trial178.stdout:3340: writing trial17818531-18 from 2895994 to 3304418 tid 3 2026-02-21T22:29:59.395 INFO:tasks.rados.rados.0.trial178.stdout:3341: snap_create 2026-02-21T22:29:59.395 INFO:tasks.rados.rados.0.trial178.stdout:3340: finishing write tid 1 to trial17818531-18 2026-02-21T22:29:59.395 INFO:tasks.rados.rados.0.trial178.stdout:3340: finishing write tid 2 to trial17818531-18 2026-02-21T22:29:59.396 INFO:tasks.rados.rados.0.trial178.stdout:3340: finishing write tid 3 to trial17818531-18 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: finishing write tid 4 to trial17818531-18 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: finishing write tid 5 to trial17818531-18 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: oid 18 updating version 0 to 1603 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: oid 18 updating version 1603 to 1606 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: oid 18 version 1606 is already newer than 1605 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: oid 18 version 1606 is already newer than 1604 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1606 (ObjNum 1173 snap 334 seq_num 1173) dirty exists 2026-02-21T22:29:59.397 INFO:tasks.rados.rados.0.trial178.stdout:3340: left oid 18 (ObjNum 1173 snap 334 seq_num 1173) 2026-02-21T22:30:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3340: done (1 left) 2026-02-21T22:30:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3341: done (0 left) 2026-02-21T22:30:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3342: snap_create 2026-02-21T22:30:01.225 INFO:tasks.rados.rados.0.trial178.stdout:3342: done (0 left) 2026-02-21T22:30:01.225 INFO:tasks.rados.rados.0.trial178.stdout:3343: delete oid 41 current snap is 336 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3343: done (0 left) 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3344: copy_from oid 29 from oid 36 current snap is 336 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3345: rollback oid 11 current snap is 336 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 335 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3346: copy_from oid 35 from oid 43 current snap is 336 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3347: write oid 21 current snap is 336 2026-02-21T22:30:01.231 INFO:tasks.rados.rados.0.trial178.stdout:3347: seq_num 1175 ranges {548225=735937,1925048=596064,2950840=547321} 2026-02-21T22:30:01.240 INFO:tasks.rados.rados.0.trial178.stdout:3347: writing trial17818531-21 from 548225 to 1284162 tid 1 2026-02-21T22:30:01.248 INFO:tasks.rados.rados.0.trial178.stdout:3347: writing trial17818531-21 from 1925048 to 2521112 tid 2 2026-02-21T22:30:01.254 INFO:tasks.rados.rados.0.trial178.stdout:3347: writing trial17818531-21 from 2950840 to 3498161 tid 3 2026-02-21T22:30:01.254 INFO:tasks.rados.rados.0.trial178.stdout:3348: snap_remove snap 329 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3348: done (4 left) 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3349: rollback oid 40 current snap is 336 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 335 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3344: finishing copy_from to trial17818531-29 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3344: got expected ENOENT (src dne) 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3344: finishing copy_from racing read to trial17818531-29 2026-02-21T22:30:02.202 INFO:tasks.rados.rados.0.trial178.stdout:3345: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3345: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3345: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1599 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: finishing write tid 1 to trial17818531-21 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3346: finishing copy_from racing read to trial17818531-35 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3346: finishing copy_from to trial17818531-35 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1575 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: finishing write tid 2 to trial17818531-21 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: finishing write tid 3 to trial17818531-21 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: finishing write tid 4 to trial17818531-21 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: finishing write tid 5 to trial17818531-21 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: oid 21 updating version 0 to 1829 2026-02-21T22:30:02.203 INFO:tasks.rados.rados.0.trial178.stdout:3347: oid 21 version 1829 is already newer than 1828 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3347: oid 21 version 1829 is already newer than 1827 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3347: oid 21 updating version 1829 to 1830 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1830 (ObjNum 1175 snap 336 seq_num 1175) dirty exists 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3347: left oid 21 (ObjNum 1175 snap 336 seq_num 1175) 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3344: done (4 left) 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3345: done (3 left) 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3346: done (2 left) 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3347: done (1 left) 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3350: write oid 32 current snap is 336 2026-02-21T22:30:02.204 INFO:tasks.rados.rados.0.trial178.stdout:3350: seq_num 1176 ranges {565032=411186,1583530=456075,2578263=758593} 2026-02-21T22:30:02.207 INFO:tasks.rados.rados.0.trial178.stdout:3350: writing trial17818531-32 from 565032 to 976218 tid 1 2026-02-21T22:30:02.214 INFO:tasks.rados.rados.0.trial178.stdout:3350: writing trial17818531-32 from 1583530 to 2039605 tid 2 2026-02-21T22:30:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3350: writing trial17818531-32 from 2578263 to 3336856 tid 3 2026-02-21T22:30:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3351: read oid 50 snap -1 2026-02-21T22:30:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3351: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3352: snap_remove snap 331 2026-02-21T22:30:03.204 INFO:tasks.rados.rados.0.trial178.stdout:3352: done (3 left) 2026-02-21T22:30:03.204 INFO:tasks.rados.rados.0.trial178.stdout:3353: write (excl) oid 13 current snap is 336 2026-02-21T22:30:03.204 INFO:tasks.rados.rados.0.trial178.stdout:3353: seq_num 1177 ranges {581839=486435,1642012=599733,3005686=569865} 2026-02-21T22:30:03.211 INFO:tasks.rados.rados.0.trial178.stdout:3353: writing trial17818531-13 from 581839 to 1068274 tid 1 2026-02-21T22:30:03.219 INFO:tasks.rados.rados.0.trial178.stdout:3353: writing trial17818531-13 from 1642012 to 2241745 tid 2 2026-02-21T22:30:03.228 INFO:tasks.rados.rados.0.trial178.stdout:3353: writing trial17818531-13 from 3005686 to 3575551 tid 3 2026-02-21T22:30:03.228 INFO:tasks.rados.rados.0.trial178.stdout:3354: write oid 48 current snap is 336 2026-02-21T22:30:03.228 INFO:tasks.rados.rados.0.trial178.stdout:3354: seq_num 1178 ranges {598646=561684,1700494=459744,2633109=781137} 2026-02-21T22:30:03.235 INFO:tasks.rados.rados.0.trial178.stdout:3354: writing trial17818531-48 from 598646 to 1160330 tid 1 2026-02-21T22:30:03.242 INFO:tasks.rados.rados.0.trial178.stdout:3354: writing trial17818531-48 from 1700494 to 2160238 tid 2 2026-02-21T22:30:03.250 INFO:tasks.rados.rados.0.trial178.stdout:3354: writing trial17818531-48 from 2633109 to 3414246 tid 3 2026-02-21T22:30:03.251 INFO:tasks.rados.rados.0.trial178.stdout:3355: copy_from oid 15 from oid 28 current snap is 336 2026-02-21T22:30:03.251 INFO:tasks.rados.rados.0.trial178.stdout:3349: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:30:03.251 INFO:tasks.rados.rados.0.trial178.stdout:3356: write (excl) oid 38 current snap is 336 2026-02-21T22:30:03.251 INFO:tasks.rados.rados.0.trial178.stdout:3356: seq_num 1179 ranges {615453=753286,2158976=603402,3460532=354921} 2026-02-21T22:30:03.256 INFO:tasks.rados.rados.0.trial178.stdout:3356: writing trial17818531-38 from 615453 to 1368739 tid 1 2026-02-21T22:30:03.261 INFO:tasks.rados.rados.0.trial178.stdout:3356: writing trial17818531-38 from 2158976 to 2762378 tid 2 2026-02-21T22:30:03.266 INFO:tasks.rados.rados.0.trial178.stdout:3356: writing trial17818531-38 from 3460532 to 3815453 tid 3 2026-02-21T22:30:03.266 INFO:tasks.rados.rados.0.trial178.stdout:3357: rollback oid 17 current snap is 336 2026-02-21T22:30:03.266 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 334 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3349: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3349: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1633 (ObjNum 1134 snap 325 seq_num 1134) dirty exists 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3350: finishing write tid 1 to trial17818531-32 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3349: done (7 left) 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3358: write (excl) oid 4 current snap is 336 2026-02-21T22:30:03.267 INFO:tasks.rados.rados.0.trial178.stdout:3358: seq_num 1180 ranges {632260=428535,1817458=463413,2687955=520034,3709324=122936} 2026-02-21T22:30:03.269 INFO:tasks.rados.rados.0.trial178.stdout:3358: writing trial17818531-4 from 632260 to 1060795 tid 1 2026-02-21T22:30:03.274 INFO:tasks.rados.rados.0.trial178.stdout:3358: writing trial17818531-4 from 1817458 to 2280871 tid 2 2026-02-21T22:30:03.277 INFO:tasks.rados.rados.0.trial178.stdout:3358: writing trial17818531-4 from 2687955 to 3207989 tid 3 2026-02-21T22:30:03.279 INFO:tasks.rados.rados.0.trial178.stdout:3358: writing trial17818531-4 from 3709324 to 3832260 tid 4 2026-02-21T22:30:03.279 INFO:tasks.rados.rados.0.trial178.stdout:3359: write oid 11 current snap is 336 2026-02-21T22:30:03.279 INFO:tasks.rados.rados.0.trial178.stdout:3359: seq_num 1181 ranges {649067=503784,1875940=607071,3115378=731306} 2026-02-21T22:30:03.284 INFO:tasks.rados.rados.0.trial178.stdout:3359: writing trial17818531-11 from 649067 to 1152851 tid 1 2026-02-21T22:30:03.289 INFO:tasks.rados.rados.0.trial178.stdout:3359: writing trial17818531-11 from 1875940 to 2483011 tid 2 2026-02-21T22:30:03.294 INFO:tasks.rados.rados.0.trial178.stdout:3359: writing trial17818531-11 from 3115378 to 3846684 tid 3 2026-02-21T22:30:03.294 INFO:tasks.rados.rados.0.trial178.stdout:3360: write oid 18 current snap is 336 2026-02-21T22:30:03.294 INFO:tasks.rados.rados.0.trial178.stdout:3360: seq_num 1182 ranges {665874=579033,1934422=467082,3142801=542578} 2026-02-21T22:30:03.299 INFO:tasks.rados.rados.0.trial178.stdout:3360: writing trial17818531-18 from 665874 to 1244907 tid 1 2026-02-21T22:30:03.303 INFO:tasks.rados.rados.0.trial178.stdout:3360: writing trial17818531-18 from 1934422 to 2401504 tid 2 2026-02-21T22:30:03.308 INFO:tasks.rados.rados.0.trial178.stdout:3360: writing trial17818531-18 from 3142801 to 3685379 tid 3 2026-02-21T22:30:03.308 INFO:tasks.rados.rados.0.trial178.stdout:3361: snap_remove snap 335 2026-02-21T22:30:03.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:03.586 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:04.860 INFO:tasks.rados.rados.0.trial178.stdout:3361: done (10 left) 2026-02-21T22:30:04.861 INFO:tasks.rados.rados.0.trial178.stdout:3362: write oid 3 current snap is 336 2026-02-21T22:30:04.861 INFO:tasks.rados.rados.0.trial178.stdout:3362: seq_num 1183 ranges {682681=654282,1876551=610740,3053871=470203} 2026-02-21T22:30:04.866 INFO:tasks.rados.rados.0.trial178.stdout:3362: writing trial17818531-3 from 682681 to 1336963 tid 1 2026-02-21T22:30:04.870 INFO:tasks.rados.rados.0.trial178.stdout:3362: writing trial17818531-3 from 1876551 to 2487291 tid 2 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3362: writing trial17818531-3 from 3053871 to 3524074 tid 3 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3363: rollback oid 37 current snap is 336 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 332 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: finishing write tid 2 to trial17818531-32 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: finishing write tid 3 to trial17818531-32 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3364: copy_from oid 35 from oid 23 current snap is 336 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: finishing write tid 4 to trial17818531-32 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: finishing write tid 5 to trial17818531-32 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: oid 32 updating version 0 to 1607 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: oid 32 updating version 1607 to 1608 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: oid 32 updating version 1608 to 1609 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: oid 32 updating version 1609 to 1610 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1610 (ObjNum 1176 snap 336 seq_num 1176) dirty exists 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3350: left oid 32 (ObjNum 1176 snap 336 seq_num 1176) 2026-02-21T22:30:04.874 INFO:tasks.rados.rados.0.trial178.stdout:3351: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:04.945 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1293086 2026-02-21T22:30:04.960 INFO:tasks.rados.rados.0.trial178.stdout:3353: finishing write tid 1 to trial17818531-13 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: finishing write tid 2 to trial17818531-13 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: finishing write tid 3 to trial17818531-13 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: finishing write tid 4 to trial17818531-13 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: finishing write tid 5 to trial17818531-13 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: oid 13 updating version 0 to 1618 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: oid 13 updating version 1618 to 1620 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: oid 13 version 1620 is already newer than 1617 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: oid 13 version 1620 is already newer than 1619 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1620 (ObjNum 1177 snap 336 seq_num 1177) dirty exists 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: left oid 13 (ObjNum 1177 snap 336 seq_num 1177) 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: finishing write tid 1 to trial17818531-48 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: finishing write tid 2 to trial17818531-48 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: finishing write tid 3 to trial17818531-48 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: finishing write tid 4 to trial17818531-48 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: finishing write tid 5 to trial17818531-48 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: oid 48 updating version 0 to 1497 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: oid 48 updating version 1497 to 1498 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: oid 48 version 1498 is already newer than 1496 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: oid 48 version 1498 is already newer than 1495 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1498 (ObjNum 1178 snap 336 seq_num 1178) dirty exists 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3354: left oid 48 (ObjNum 1178 snap 336 seq_num 1178) 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3356: finishing write tid 1 to trial17818531-38 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3356: finishing write tid 2 to trial17818531-38 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3355: finishing copy_from racing read to trial17818531-15 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3350: done (12 left) 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3351: done (11 left) 2026-02-21T22:30:04.961 INFO:tasks.rados.rados.0.trial178.stdout:3353: done (10 left) 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3354: done (9 left) 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3365: snap_create 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3355: finishing copy_from to trial17818531-15 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1208 (ObjNum 1147 snap 327 seq_num 1147) dirty exists 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: finishing write tid 3 to trial17818531-38 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: finishing write tid 4 to trial17818531-38 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: finishing write tid 5 to trial17818531-38 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: oid 38 updating version 0 to 1209 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: oid 38 updating version 1209 to 1212 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: oid 38 version 1212 is already newer than 1211 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: oid 38 version 1212 is already newer than 1210 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1212 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3356: left oid 38 (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3357: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3357: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3357: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1296 (ObjNum 1166 snap 333 seq_num 1166) dirty exists 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 1 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 2 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 3 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 4 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 5 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: finishing write tid 6 to trial17818531-4 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: oid 4 updating version 0 to 1622 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: oid 4 version 1622 is already newer than 1621 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: oid 4 version 1622 is already newer than 1620 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: oid 4 updating version 1622 to 1623 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: oid 4 updating version 1623 to 1624 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1624 (ObjNum 1180 snap 336 seq_num 1180) dirty exists 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3358: left oid 4 (ObjNum 1180 snap 336 seq_num 1180) 2026-02-21T22:30:04.962 INFO:tasks.rados.rados.0.trial178.stdout:3359: finishing write tid 1 to trial17818531-11 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: finishing write tid 2 to trial17818531-11 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: finishing write tid 3 to trial17818531-11 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: finishing write tid 4 to trial17818531-11 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: finishing write tid 5 to trial17818531-11 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: oid 11 updating version 0 to 1623 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: oid 11 version 1623 is already newer than 1622 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: oid 11 version 1623 is already newer than 1621 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: oid 11 updating version 1623 to 1624 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1624 (ObjNum 1181 snap 336 seq_num 1181) dirty exists 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3359: left oid 11 (ObjNum 1181 snap 336 seq_num 1181) 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: finishing write tid 1 to trial17818531-18 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: finishing write tid 2 to trial17818531-18 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: finishing write tid 3 to trial17818531-18 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: finishing write tid 4 to trial17818531-18 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: finishing write tid 5 to trial17818531-18 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: oid 18 updating version 0 to 1610 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: oid 18 version 1610 is already newer than 1607 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: oid 18 version 1610 is already newer than 1609 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: oid 18 version 1610 is already newer than 1608 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1610 (ObjNum 1182 snap 336 seq_num 1182) dirty exists 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3360: left oid 18 (ObjNum 1182 snap 336 seq_num 1182) 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3362: finishing write tid 1 to trial17818531-3 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3362: finishing write tid 2 to trial17818531-3 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3363: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 0 (ObjNum 33 snap 0 seq_num 101576336) dirty dne 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3362: finishing write tid 3 to trial17818531-3 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3362: finishing write tid 4 to trial17818531-3 2026-02-21T22:30:04.963 INFO:tasks.rados.rados.0.trial178.stdout:3362: finishing write tid 5 to trial17818531-3 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3362: oid 3 updating version 0 to 1627 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3362: oid 3 version 1627 is already newer than 1626 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3362: oid 3 updating version 1627 to 1632 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3362: oid 3 updating version 1632 to 1633 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1633 (ObjNum 1183 snap 336 seq_num 1183) dirty exists 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3362: left oid 3 (ObjNum 1183 snap 336 seq_num 1183) 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3364: finishing copy_from to trial17818531-35 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3364: got expected ENOENT (src dne) 2026-02-21T22:30:04.964 INFO:tasks.rados.rados.0.trial178.stdout:3364: finishing copy_from racing read to trial17818531-35 2026-02-21T22:30:05.861 INFO:tasks.rados.rados.0.trial178.stdout:3355: done (9 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3356: done (8 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3357: done (7 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3358: done (6 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3359: done (5 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3360: done (4 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3362: done (3 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3363: done (2 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3364: done (1 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3365: done (0 left) 2026-02-21T22:30:05.862 INFO:tasks.rados.rados.0.trial178.stdout:3366: snap_remove snap 334 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3366: done (0 left) 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3367: rollback oid 46 current snap is 337 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 332 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3368: read oid 40 snap -1 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3368: expect (ObjNum 1134 snap 325 seq_num 1134) 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3369: read oid 45 snap -1 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3369: expect (ObjNum 1161 snap 332 seq_num 1161) 2026-02-21T22:30:06.862 INFO:tasks.rados.rados.0.trial178.stdout:3370: read oid 10 snap -1 2026-02-21T22:30:06.863 INFO:tasks.rados.rados.0.trial178.stdout:3370: expect deleted 2026-02-21T22:30:06.863 INFO:tasks.rados.rados.0.trial178.stdout:3371: rollback oid 5 current snap is 337 2026-02-21T22:30:06.863 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 332 2026-02-21T22:30:06.863 INFO:tasks.rados.rados.0.trial178.stdout:3372: write oid 18 current snap is 337 2026-02-21T22:30:06.863 INFO:tasks.rados.rados.0.trial178.stdout:3372: seq_num 1184 ranges {699488=729531,1935033=470751,3081294=681475} 2026-02-21T22:30:06.872 INFO:tasks.rados.rados.0.trial178.stdout:3372: writing trial17818531-18 from 699488 to 1429019 tid 1 2026-02-21T22:30:06.879 INFO:tasks.rados.rados.0.trial178.stdout:3372: writing trial17818531-18 from 1935033 to 2405784 tid 2 2026-02-21T22:30:06.888 INFO:tasks.rados.rados.0.trial178.stdout:3372: writing trial17818531-18 from 3081294 to 3762769 tid 3 2026-02-21T22:30:06.888 INFO:tasks.rados.rados.0.trial178.stdout:3373: snap_remove snap 336 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:3373: done (6 left) 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:3374: rollback oid 25 current snap is 337 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 25 to 332 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:3367: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:3371: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:30:07.202 INFO:tasks.rados.rados.0.trial178.stdout:3369: expect (ObjNum 1161 snap 332 seq_num 1161) 2026-02-21T22:30:07.399 INFO:tasks.rados.rados.0.trial178.stdout:3371: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:30:07.399 INFO:tasks.rados.rados.0.trial178.stdout:3371: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:30:07.399 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1613 (ObjNum 1133 snap 324 seq_num 1133) dirty exists 2026-02-21T22:30:07.399 INFO:tasks.rados.rados.0.trial178.stdout:3368: expect (ObjNum 1134 snap 325 seq_num 1134) 2026-02-21T22:30:07.399 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 391602 2026-02-21T22:30:07.420 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 281741 2026-02-21T22:30:07.455 INFO:tasks.rados.rados.0.trial178.stdout:3372: finishing write tid 1 to trial17818531-18 2026-02-21T22:30:07.455 INFO:tasks.rados.rados.0.trial178.stdout:3372: finishing write tid 2 to trial17818531-18 2026-02-21T22:30:07.455 INFO:tasks.rados.rados.0.trial178.stdout:3372: finishing write tid 3 to trial17818531-18 2026-02-21T22:30:07.455 INFO:tasks.rados.rados.0.trial178.stdout:3367: done (6 left) 2026-02-21T22:30:07.455 INFO:tasks.rados.rados.0.trial178.stdout:3368: done (5 left) 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3369: done (4 left) 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3370: done (3 left) 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3371: done (2 left) 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3375: delete oid 26 current snap is 337 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: finishing write tid 4 to trial17818531-18 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: finishing write tid 5 to trial17818531-18 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: oid 18 updating version 0 to 1616 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: oid 18 version 1616 is already newer than 1615 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: oid 18 version 1616 is already newer than 1614 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: oid 18 updating version 1616 to 1617 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1617 (ObjNum 1184 snap 337 seq_num 1184) dirty exists 2026-02-21T22:30:07.456 INFO:tasks.rados.rados.0.trial178.stdout:3372: left oid 18 (ObjNum 1184 snap 337 seq_num 1184) 2026-02-21T22:30:07.457 INFO:tasks.rados.rados.0.trial178.stdout:3374: finishing rollback tid 0 to trial17818531-25 2026-02-21T22:30:07.457 INFO:tasks.rados.rados.0.trial178.stdout:3374: finishing rollback tid 1 to trial17818531-25 2026-02-21T22:30:07.457 INFO:tasks.rados.rados.0.trial178.stdout:3374: finishing rollback tid 2 to trial17818531-25 2026-02-21T22:30:07.457 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1482 (ObjNum 1154 snap 329 seq_num 1154) dirty exists 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3372: done (2 left) 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3374: done (1 left) 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3375: done (0 left) 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3376: read oid 21 snap -1 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3376: expect (ObjNum 1175 snap 336 seq_num 1175) 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3377: read oid 33 snap -1 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3377: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3378: write oid 13 current snap is 337 2026-02-21T22:30:07.458 INFO:tasks.rados.rados.0.trial178.stdout:3378: seq_num 1186 ranges {733102=596382,1768350=758067,3136140=704019} 2026-02-21T22:30:07.465 INFO:tasks.rados.rados.0.trial178.stdout:3378: writing trial17818531-13 from 733102 to 1329484 tid 1 2026-02-21T22:30:07.473 INFO:tasks.rados.rados.0.trial178.stdout:3378: writing trial17818531-13 from 1768350 to 2526417 tid 2 2026-02-21T22:30:07.482 INFO:tasks.rados.rados.0.trial178.stdout:3378: writing trial17818531-13 from 3136140 to 3840159 tid 3 2026-02-21T22:30:07.482 INFO:tasks.rados.rados.0.trial178.stdout:3379: write oid 18 current snap is 337 2026-02-21T22:30:07.482 INFO:tasks.rados.rados.0.trial178.stdout:3379: seq_num 1187 ranges {749909=671631,2110479=618078,3163563=515291} 2026-02-21T22:30:07.491 INFO:tasks.rados.rados.0.trial178.stdout:3379: writing trial17818531-18 from 749909 to 1421540 tid 1 2026-02-21T22:30:07.500 INFO:tasks.rados.rados.0.trial178.stdout:3379: writing trial17818531-18 from 2110479 to 2728557 tid 2 2026-02-21T22:30:07.505 INFO:tasks.rados.rados.0.trial178.stdout:3379: writing trial17818531-18 from 3163563 to 3678854 tid 3 2026-02-21T22:30:07.505 INFO:tasks.rados.rados.0.trial178.stdout:3380: snap_remove snap 332 2026-02-21T22:30:08.208 INFO:tasks.rados.rados.0.trial178.stdout:3380: done (4 left) 2026-02-21T22:30:08.208 INFO:tasks.rados.rados.0.trial178.stdout:3381: snap_create 2026-02-21T22:30:08.208 INFO:tasks.rados.rados.0.trial178.stdout:3377: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:08.293 INFO:tasks.rados.rados.0.trial178.stdout:3376: expect (ObjNum 1175 snap 336 seq_num 1175) 2026-02-21T22:30:08.319 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2016882 2026-02-21T22:30:08.423 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3055726 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: finishing write tid 1 to trial17818531-13 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: finishing write tid 2 to trial17818531-13 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: finishing write tid 3 to trial17818531-13 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: finishing write tid 4 to trial17818531-13 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: finishing write tid 5 to trial17818531-13 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: oid 13 updating version 0 to 1644 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: oid 13 updating version 1644 to 1647 2026-02-21T22:30:08.432 INFO:tasks.rados.rados.0.trial178.stdout:3378: oid 13 version 1647 is already newer than 1646 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3378: oid 13 version 1647 is already newer than 1645 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1647 (ObjNum 1186 snap 337 seq_num 1186) dirty exists 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3378: left oid 13 (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: finishing write tid 1 to trial17818531-18 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: finishing write tid 2 to trial17818531-18 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: finishing write tid 3 to trial17818531-18 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: finishing write tid 4 to trial17818531-18 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: finishing write tid 5 to trial17818531-18 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: oid 18 updating version 0 to 1618 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: oid 18 updating version 1618 to 1621 2026-02-21T22:30:08.433 INFO:tasks.rados.rados.0.trial178.stdout:3379: oid 18 version 1621 is already newer than 1620 2026-02-21T22:30:08.434 INFO:tasks.rados.rados.0.trial178.stdout:3379: oid 18 version 1621 is already newer than 1619 2026-02-21T22:30:08.434 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1621 (ObjNum 1187 snap 337 seq_num 1187) dirty exists 2026-02-21T22:30:08.434 INFO:tasks.rados.rados.0.trial178.stdout:3379: left oid 18 (ObjNum 1187 snap 337 seq_num 1187) 2026-02-21T22:30:08.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:08.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3376: done (4 left) 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3377: done (3 left) 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3378: done (2 left) 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3379: done (1 left) 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3381: done (0 left) 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3382: rollback oid 31 current snap is 338 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 337 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3383: write (excl) oid 4 current snap is 338 2026-02-21T22:30:09.209 INFO:tasks.rados.rados.0.trial178.stdout:3383: seq_num 1188 ranges {766716=746880,2168961=761736,3474633=442916} 2026-02-21T22:30:09.219 INFO:tasks.rados.rados.0.trial178.stdout:3383: writing trial17818531-4 from 766716 to 1513596 tid 1 2026-02-21T22:30:09.229 INFO:tasks.rados.rados.0.trial178.stdout:3383: writing trial17818531-4 from 2168961 to 2930697 tid 2 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3383: writing trial17818531-4 from 3474633 to 3917549 tid 3 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3384: snap_create 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3382: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3382: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3382: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1485 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:30:09.235 INFO:tasks.rados.rados.0.trial178.stdout:3383: finishing write tid 1 to trial17818531-4 2026-02-21T22:30:09.236 INFO:tasks.rados.rados.0.trial178.stdout:3383: finishing write tid 2 to trial17818531-4 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: finishing write tid 3 to trial17818531-4 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: finishing write tid 4 to trial17818531-4 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: finishing write tid 5 to trial17818531-4 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: oid 4 updating version 0 to 1625 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: oid 4 updating version 1625 to 1626 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: oid 4 updating version 1626 to 1628 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:3383: oid 4 version 1628 is already newer than 1627 2026-02-21T22:30:09.237 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1628 (ObjNum 1188 snap 338 seq_num 1188) dirty exists 2026-02-21T22:30:09.238 INFO:tasks.rados.rados.0.trial178.stdout:3383: left oid 4 (ObjNum 1188 snap 338 seq_num 1188) 2026-02-21T22:30:10.214 INFO:tasks.rados.rados.0.trial178.stdout:3382: done (2 left) 2026-02-21T22:30:10.214 INFO:tasks.rados.rados.0.trial178.stdout:3383: done (1 left) 2026-02-21T22:30:10.214 INFO:tasks.rados.rados.0.trial178.stdout:3384: done (0 left) 2026-02-21T22:30:10.214 INFO:tasks.rados.rados.0.trial178.stdout:3385: snap_create 2026-02-21T22:30:11.215 INFO:tasks.rados.rados.0.trial178.stdout:3385: done (0 left) 2026-02-21T22:30:11.215 INFO:tasks.rados.rados.0.trial178.stdout:3386: snap_create 2026-02-21T22:30:12.202 INFO:tasks.rados.rados.0.trial178.stdout:3386: done (0 left) 2026-02-21T22:30:12.202 INFO:tasks.rados.rados.0.trial178.stdout:3387: snap_create 2026-02-21T22:30:13.205 INFO:tasks.rados.rados.0.trial178.stdout:3387: done (0 left) 2026-02-21T22:30:13.205 INFO:tasks.rados.rados.0.trial178.stdout:3388: write (excl) oid 14 current snap is 342 2026-02-21T22:30:13.205 INFO:tasks.rados.rados.0.trial178.stdout:3388: seq_num 1189 ranges {783523=422129,1827443=621747,3218409=654188} 2026-02-21T22:30:13.212 INFO:tasks.rados.rados.0.trial178.stdout:3388: writing trial17818531-14 from 783523 to 1205652 tid 1 2026-02-21T22:30:13.221 INFO:tasks.rados.rados.0.trial178.stdout:3388: writing trial17818531-14 from 1827443 to 2449190 tid 2 2026-02-21T22:30:13.230 INFO:tasks.rados.rados.0.trial178.stdout:3388: writing trial17818531-14 from 3218409 to 3872597 tid 3 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3389: read oid 50 snap -1 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3389: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3390: read oid 49 snap 341 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3390: expect (ObjNum 1080 snap 313 seq_num 1080) 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3391: read oid 15 snap -1 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3391: expect (ObjNum 1147 snap 327 seq_num 1147) 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3392: copy_from oid 48 from oid 7 current snap is 342 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3388: finishing write tid 1 to trial17818531-14 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3388: finishing write tid 2 to trial17818531-14 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3393: read oid 3 snap -1 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3393: expect (ObjNum 1183 snap 336 seq_num 1183) 2026-02-21T22:30:13.231 INFO:tasks.rados.rados.0.trial178.stdout:3394: snap_create 2026-02-21T22:30:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3392: finishing copy_from to trial17818531-48 2026-02-21T22:30:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3392: got expected ENOENT (src dne) 2026-02-21T22:30:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3392: finishing copy_from racing read to trial17818531-48 2026-02-21T22:30:13.236 INFO:tasks.rados.rados.0.trial178.stdout:3388: finishing write tid 3 to trial17818531-14 2026-02-21T22:30:13.238 INFO:tasks.rados.rados.0.trial178.stdout:3390: expect (ObjNum 1080 snap 313 seq_num 1080) 2026-02-21T22:30:13.238 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:13.266 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:13.289 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:13.305 INFO:tasks.rados.rados.0.trial178.stdout:3389: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:13.305 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:13.331 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:13.357 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 470635 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: finishing write tid 4 to trial17818531-14 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: finishing write tid 5 to trial17818531-14 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: oid 14 updating version 0 to 1471 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: oid 14 updating version 1471 to 1473 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: oid 14 updating version 1473 to 1474 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: oid 14 version 1474 is already newer than 1472 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1474 (ObjNum 1189 snap 342 seq_num 1189) dirty exists 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3388: left oid 14 (ObjNum 1189 snap 342 seq_num 1189) 2026-02-21T22:30:13.380 INFO:tasks.rados.rados.0.trial178.stdout:3391: expect (ObjNum 1147 snap 327 seq_num 1147) 2026-02-21T22:30:13.444 INFO:tasks.rados.rados.0.trial178.stdout:3393: expect (ObjNum 1183 snap 336 seq_num 1183) 2026-02-21T22:30:13.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:13.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3388: done (6 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3389: done (5 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3390: done (4 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3391: done (3 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3392: done (2 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3393: done (1 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3394: done (0 left) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3395: copy_from oid 48 from oid 45 current snap is 343 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3396: read oid 11 snap -1 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3396: expect (ObjNum 1181 snap 336 seq_num 1181) 2026-02-21T22:30:14.864 INFO:tasks.rados.rados.0.trial178.stdout:3397: delete oid 10 current snap is 343 2026-02-21T22:30:14.872 INFO:tasks.rados.rados.0.trial178.stdout:3397: done (2 left) 2026-02-21T22:30:14.872 INFO:tasks.rados.rados.0.trial178.stdout:3398: read oid 3 snap -1 2026-02-21T22:30:14.872 INFO:tasks.rados.rados.0.trial178.stdout:3398: expect (ObjNum 1183 snap 336 seq_num 1183) 2026-02-21T22:30:14.872 INFO:tasks.rados.rados.0.trial178.stdout:3399: snap_create 2026-02-21T22:30:14.873 INFO:tasks.rados.rados.0.trial178.stdout:3396: expect (ObjNum 1181 snap 336 seq_num 1181) 2026-02-21T22:30:14.892 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1847060 2026-02-21T22:30:14.988 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1558798 2026-02-21T22:30:15.017 INFO:tasks.rados.rados.0.trial178.stdout:3395: finishing copy_from racing read to trial17818531-48 2026-02-21T22:30:15.017 INFO:tasks.rados.rados.0.trial178.stdout:3395: finishing copy_from to trial17818531-48 2026-02-21T22:30:15.017 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1490 (ObjNum 1161 snap 332 seq_num 1161) dirty exists 2026-02-21T22:30:15.017 INFO:tasks.rados.rados.0.trial178.stdout:3398: expect (ObjNum 1183 snap 336 seq_num 1183) 2026-02-21T22:30:15.035 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2119232 2026-02-21T22:30:15.113 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3301746 2026-02-21T22:30:15.121 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3395: done (3 left) 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3396: done (2 left) 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3398: done (1 left) 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3399: done (0 left) 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3400: rollback oid 46 current snap is 344 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 343 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3401: rollback oid 10 current snap is 344 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 343 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3402: copy_from oid 45 from oid 29 current snap is 344 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3403: write oid 9 current snap is 344 2026-02-21T22:30:15.865 INFO:tasks.rados.rados.0.trial178.stdout:3403: seq_num 1191 ranges {417137=572627,1428054=589083} 2026-02-21T22:30:15.871 INFO:tasks.rados.rados.0.trial178.stdout:3403: writing trial17818531-9 from 417137 to 989764 tid 1 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:3403: writing trial17818531-9 from 1428054 to 2017137 tid 2 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:3404: delete oid 25 current snap is 344 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:3400: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:3401: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-21T22:30:15.878 INFO:tasks.rados.rados.0.trial178.stdout:3403: finishing write tid 1 to trial17818531-9 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: finishing write tid 2 to trial17818531-9 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: finishing write tid 3 to trial17818531-9 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: finishing write tid 4 to trial17818531-9 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: oid 9 updating version 0 to 1476 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: oid 9 version 1476 is already newer than 1475 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: oid 9 updating version 1476 to 1477 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1477 (ObjNum 1191 snap 344 seq_num 1191) dirty exists 2026-02-21T22:30:15.881 INFO:tasks.rados.rados.0.trial178.stdout:3403: left oid 9 (ObjNum 1191 snap 344 seq_num 1191) 2026-02-21T22:30:15.882 INFO:tasks.rados.rados.0.trial178.stdout:3402: finishing copy_from racing read to trial17818531-45 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3400: done (4 left) 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3401: done (3 left) 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3403: done (2 left) 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3404: done (1 left) 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3405: read oid 2 snap -1 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3405: expect deleted 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3406: snap_create 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:3402: finishing copy_from to trial17818531-45 2026-02-21T22:30:15.883 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1621 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3402: done (2 left) 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3405: done (1 left) 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3406: done (0 left) 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3407: copy_from oid 40 from oid 23 current snap is 345 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3408: write (excl) oid 10 current snap is 345 2026-02-21T22:30:16.866 INFO:tasks.rados.rados.0.trial178.stdout:3408: seq_num 1193 ranges {450751=723125,1945018=105733} 2026-02-21T22:30:16.874 INFO:tasks.rados.rados.0.trial178.stdout:3408: writing trial17818531-10 from 450751 to 1173876 tid 1 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3408: writing trial17818531-10 from 1945018 to 2050751 tid 2 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3409: write (excl) oid 2 current snap is 345 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3407: finishing copy_from to trial17818531-40 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3407: got expected ENOENT (src dne) 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3407: finishing copy_from racing read to trial17818531-40 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3408: finishing write tid 1 to trial17818531-10 2026-02-21T22:30:16.881 INFO:tasks.rados.rados.0.trial178.stdout:3409: seq_num 1194 ranges {467558=514727,1719853=347705} 2026-02-21T22:30:16.887 INFO:tasks.rados.rados.0.trial178.stdout:3409: writing trial17818531-2 from 467558 to 982285 tid 1 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3409: writing trial17818531-2 from 1719853 to 2067558 tid 2 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3407: done (2 left) 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3410: delete oid 35 current snap is 345 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3408: finishing write tid 2 to trial17818531-10 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3408: finishing write tid 3 to trial17818531-10 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3408: finishing write tid 4 to trial17818531-10 2026-02-21T22:30:16.895 INFO:tasks.rados.rados.0.trial178.stdout:3408: oid 10 updating version 0 to 1243 2026-02-21T22:30:16.896 INFO:tasks.rados.rados.0.trial178.stdout:3408: oid 10 updating version 1243 to 1244 2026-02-21T22:30:16.896 INFO:tasks.rados.rados.0.trial178.stdout:3408: oid 10 updating version 1244 to 1245 2026-02-21T22:30:16.896 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1245 (ObjNum 1193 snap 345 seq_num 1193) dirty exists 2026-02-21T22:30:16.896 INFO:tasks.rados.rados.0.trial178.stdout:3408: left oid 10 (ObjNum 1193 snap 345 seq_num 1193) 2026-02-21T22:30:16.896 INFO:tasks.rados.rados.0.trial178.stdout:3409: finishing write tid 1 to trial17818531-2 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: finishing write tid 2 to trial17818531-2 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: finishing write tid 3 to trial17818531-2 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: finishing write tid 4 to trial17818531-2 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: oid 2 updating version 0 to 1248 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: oid 2 version 1248 is already newer than 1246 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: oid 2 version 1248 is already newer than 1247 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1248 (ObjNum 1194 snap 345 seq_num 1194) dirty exists 2026-02-21T22:30:16.897 INFO:tasks.rados.rados.0.trial178.stdout:3409: left oid 2 (ObjNum 1194 snap 345 seq_num 1194) 2026-02-21T22:30:16.898 INFO:tasks.rados.rados.0.trial178.stdout:3408: done (2 left) 2026-02-21T22:30:16.898 INFO:tasks.rados.rados.0.trial178.stdout:3409: done (1 left) 2026-02-21T22:30:16.898 INFO:tasks.rados.rados.0.trial178.stdout:3410: done (0 left) 2026-02-21T22:30:16.898 INFO:tasks.rados.rados.0.trial178.stdout:3411: copy_from oid 3 from oid 16 current snap is 345 2026-02-21T22:30:16.898 INFO:tasks.rados.rados.0.trial178.stdout:3412: snap_create 2026-02-21T22:30:16.913 INFO:tasks.rados.rados.0.trial178.stdout:3411: finishing copy_from racing read to trial17818531-3 2026-02-21T22:30:16.913 INFO:tasks.rados.rados.0.trial178.stdout:3411: finishing copy_from to trial17818531-3 2026-02-21T22:30:16.913 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1612 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:30:17.204 INFO:tasks.rados.rados.0.trial178.stdout:3411: done (1 left) 2026-02-21T22:30:17.204 INFO:tasks.rados.rados.0.trial178.stdout:3412: done (0 left) 2026-02-21T22:30:17.204 INFO:tasks.rados.rados.0.trial178.stdout:3413: snap_create 2026-02-21T22:30:18.209 INFO:tasks.rados.rados.0.trial178.stdout:3413: done (0 left) 2026-02-21T22:30:18.210 INFO:tasks.rados.rados.0.trial178.stdout:3414: write oid 12 current snap is 347 2026-02-21T22:30:18.210 INFO:tasks.rados.rados.0.trial178.stdout:3414: seq_num 1196 ranges {501172=665225,1720464=380708} 2026-02-21T22:30:18.217 INFO:tasks.rados.rados.0.trial178.stdout:3414: writing trial17818531-12 from 501172 to 1166397 tid 1 2026-02-21T22:30:18.223 INFO:tasks.rados.rados.0.trial178.stdout:3414: writing trial17818531-12 from 1720464 to 2101172 tid 2 2026-02-21T22:30:18.224 INFO:tasks.rados.rados.0.trial178.stdout:3415: read oid 33 snap -1 2026-02-21T22:30:18.224 INFO:tasks.rados.rados.0.trial178.stdout:3415: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:18.224 INFO:tasks.rados.rados.0.trial178.stdout:3416: write (excl) oid 39 current snap is 347 2026-02-21T22:30:18.224 INFO:tasks.rados.rados.0.trial178.stdout:3414: finishing write tid 1 to trial17818531-12 2026-02-21T22:30:18.224 INFO:tasks.rados.rados.0.trial178.stdout:3416: seq_num 1197 ranges {517979=740474,1778946=339033} 2026-02-21T22:30:18.232 INFO:tasks.rados.rados.0.trial178.stdout:3416: writing trial17818531-39 from 517979 to 1258453 tid 1 2026-02-21T22:30:18.237 INFO:tasks.rados.rados.0.trial178.stdout:3416: writing trial17818531-39 from 1778946 to 2117979 tid 2 2026-02-21T22:30:18.237 INFO:tasks.rados.rados.0.trial178.stdout:3417: snap_remove snap 341 2026-02-21T22:30:18.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:18.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:19.210 INFO:tasks.rados.rados.0.trial178.stdout:3417: done (3 left) 2026-02-21T22:30:19.211 INFO:tasks.rados.rados.0.trial178.stdout:3418: read oid 31 snap 344 2026-02-21T22:30:19.211 INFO:tasks.rados.rados.0.trial178.stdout:3418: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:30:19.211 INFO:tasks.rados.rados.0.trial178.stdout:3419: write (excl) oid 27 current snap is 347 2026-02-21T22:30:19.211 INFO:tasks.rados.rados.0.trial178.stdout:3419: seq_num 1198 ranges {534786=415723,1437428=697358} 2026-02-21T22:30:19.216 INFO:tasks.rados.rados.0.trial178.stdout:3419: writing trial17818531-27 from 534786 to 950509 tid 1 2026-02-21T22:30:19.225 INFO:tasks.rados.rados.0.trial178.stdout:3419: writing trial17818531-27 from 1437428 to 2134786 tid 2 2026-02-21T22:30:19.225 INFO:tasks.rados.rados.0.trial178.stdout:3420: write (excl) oid 20 current snap is 347 2026-02-21T22:30:19.225 INFO:tasks.rados.rados.0.trial178.stdout:3420: seq_num 1199 ranges {551593=490972,1779557=372036} 2026-02-21T22:30:19.232 INFO:tasks.rados.rados.0.trial178.stdout:3420: writing trial17818531-20 from 551593 to 1042565 tid 1 2026-02-21T22:30:19.240 INFO:tasks.rados.rados.0.trial178.stdout:3420: writing trial17818531-20 from 1779557 to 2151593 tid 2 2026-02-21T22:30:19.240 INFO:tasks.rados.rados.0.trial178.stdout:3421: read oid 3 snap -1 2026-02-21T22:30:19.240 INFO:tasks.rados.rados.0.trial178.stdout:3421: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:30:19.241 INFO:tasks.rados.rados.0.trial178.stdout:3422: read oid 22 snap -1 2026-02-21T22:30:19.241 INFO:tasks.rados.rados.0.trial178.stdout:3422: expect (ObjNum 1127 snap 323 seq_num 1127) 2026-02-21T22:30:19.241 INFO:tasks.rados.rados.0.trial178.stdout:3423: write (excl) oid 5 current snap is 347 2026-02-21T22:30:19.241 INFO:tasks.rados.rados.0.trial178.stdout:3423: seq_num 1200 ranges {568400=566221,1838039=330361} 2026-02-21T22:30:19.247 INFO:tasks.rados.rados.0.trial178.stdout:3423: writing trial17818531-5 from 568400 to 1134621 tid 1 2026-02-21T22:30:19.255 INFO:tasks.rados.rados.0.trial178.stdout:3423: writing trial17818531-5 from 1838039 to 2168400 tid 2 2026-02-21T22:30:19.255 INFO:tasks.rados.rados.0.trial178.stdout:3424: write (excl) oid 46 current snap is 347 2026-02-21T22:30:19.255 INFO:tasks.rados.rados.0.trial178.stdout:3424: seq_num 1201 ranges {585207=641470,1896521=288686} 2026-02-21T22:30:19.259 INFO:tasks.rados.rados.0.trial178.stdout:3424: writing trial17818531-46 from 585207 to 1226677 tid 1 2026-02-21T22:30:19.263 INFO:tasks.rados.rados.0.trial178.stdout:3424: writing trial17818531-46 from 1896521 to 2185207 tid 2 2026-02-21T22:30:19.263 INFO:tasks.rados.rados.0.trial178.stdout:3425: write oid 11 current snap is 347 2026-02-21T22:30:19.263 INFO:tasks.rados.rados.0.trial178.stdout:3425: seq_num 1202 ranges {602014=433072,1671356=530658} 2026-02-21T22:30:19.267 INFO:tasks.rados.rados.0.trial178.stdout:3425: writing trial17818531-11 from 602014 to 1035086 tid 1 2026-02-21T22:30:19.271 INFO:tasks.rados.rados.0.trial178.stdout:3425: writing trial17818531-11 from 1671356 to 2202014 tid 2 2026-02-21T22:30:19.271 INFO:tasks.rados.rados.0.trial178.stdout:3426: delete oid 9 current snap is 347 2026-02-21T22:30:19.271 INFO:tasks.rados.rados.0.trial178.stdout:3415: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:19.278 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 985233 2026-02-21T22:30:19.339 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: finishing write tid 2 to trial17818531-12 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: finishing write tid 3 to trial17818531-12 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: finishing write tid 4 to trial17818531-12 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: oid 12 updating version 0 to 1631 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: oid 12 version 1631 is already newer than 1630 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: oid 12 version 1631 is already newer than 1629 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1631 (ObjNum 1196 snap 347 seq_num 1196) dirty exists 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: left oid 12 (ObjNum 1196 snap 347 seq_num 1196) 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3416: finishing write tid 1 to trial17818531-39 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3416: finishing write tid 2 to trial17818531-39 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3416: finishing write tid 3 to trial17818531-39 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3414: done (11 left) 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3415: done (10 left) 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3426: done (9 left) 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3427: write (excl) oid 19 current snap is 347 2026-02-21T22:30:19.366 INFO:tasks.rados.rados.0.trial178.stdout:3427: seq_num 1204 ranges {635628=583570,1671967=563661} 2026-02-21T22:30:19.371 INFO:tasks.rados.rados.0.trial178.stdout:3427: writing trial17818531-19 from 635628 to 1219198 tid 1 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3427: writing trial17818531-19 from 1671967 to 2235628 tid 2 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3428: read oid 9 snap -1 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3428: expect deleted 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3429: read oid 50 snap 346 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3429: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3430: read oid 25 snap -1 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3430: expect deleted 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3431: read oid 26 snap -1 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3431: expect deleted 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3432: write (excl) oid 23 current snap is 347 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3416: finishing write tid 4 to trial17818531-39 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3416: oid 39 updating version 0 to 1480 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3416: oid 39 version 1480 is already newer than 1478 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3416: oid 39 version 1480 is already newer than 1479 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1480 (ObjNum 1197 snap 347 seq_num 1197) dirty exists 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3416: left oid 39 (ObjNum 1197 snap 347 seq_num 1197) 2026-02-21T22:30:19.379 INFO:tasks.rados.rados.0.trial178.stdout:3418: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:30:19.407 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1543214 2026-02-21T22:30:19.452 INFO:tasks.rados.rados.0.trial178.stdout:3419: finishing write tid 1 to trial17818531-27 2026-02-21T22:30:19.452 INFO:tasks.rados.rados.0.trial178.stdout:3419: finishing write tid 2 to trial17818531-27 2026-02-21T22:30:19.452 INFO:tasks.rados.rados.0.trial178.stdout:3432: seq_num 1205 ranges {652435=658819,1730449=521986} 2026-02-21T22:30:19.461 INFO:tasks.rados.rados.0.trial178.stdout:3432: writing trial17818531-23 from 652435 to 1311254 tid 1 2026-02-21T22:30:19.467 INFO:tasks.rados.rados.0.trial178.stdout:3432: writing trial17818531-23 from 1730449 to 2252435 tid 2 2026-02-21T22:30:19.467 INFO:tasks.rados.rados.0.trial178.stdout:3416: done (14 left) 2026-02-21T22:30:19.467 INFO:tasks.rados.rados.0.trial178.stdout:3418: done (13 left) 2026-02-21T22:30:19.467 INFO:tasks.rados.rados.0.trial178.stdout:3433: copy_from oid 33 from oid 24 current snap is 347 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: finishing write tid 3 to trial17818531-27 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: finishing write tid 4 to trial17818531-27 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: oid 27 updating version 0 to 1481 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: oid 27 updating version 1481 to 1483 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: oid 27 version 1483 is already newer than 1482 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1483 (ObjNum 1198 snap 347 seq_num 1198) dirty exists 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3419: left oid 27 (ObjNum 1198 snap 347 seq_num 1198) 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: finishing write tid 1 to trial17818531-20 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: finishing write tid 2 to trial17818531-20 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: finishing write tid 3 to trial17818531-20 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: finishing write tid 4 to trial17818531-20 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: oid 20 updating version 0 to 1316 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: oid 20 updating version 1316 to 1317 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: oid 20 updating version 1317 to 1318 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1318 (ObjNum 1199 snap 347 seq_num 1199) dirty exists 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3420: left oid 20 (ObjNum 1199 snap 347 seq_num 1199) 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:3421: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:30:19.468 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 228714 2026-02-21T22:30:19.537 INFO:tasks.rados.rados.0.trial178.stdout:3423: finishing write tid 1 to trial17818531-5 2026-02-21T22:30:19.537 INFO:tasks.rados.rados.0.trial178.stdout:3419: done (13 left) 2026-02-21T22:30:19.538 INFO:tasks.rados.rados.0.trial178.stdout:3420: done (12 left) 2026-02-21T22:30:19.538 INFO:tasks.rados.rados.0.trial178.stdout:3421: done (11 left) 2026-02-21T22:30:19.538 INFO:tasks.rados.rados.0.trial178.stdout:3434: rollback oid 48 current snap is 347 2026-02-21T22:30:19.538 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 338 2026-02-21T22:30:19.538 INFO:tasks.rados.rados.0.trial178.stdout:3422: expect (ObjNum 1127 snap 323 seq_num 1127) 2026-02-21T22:30:19.548 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2927502 2026-02-21T22:30:19.548 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 531729 2026-02-21T22:30:19.568 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:19.590 INFO:tasks.rados.rados.0.trial178.stdout:3423: finishing write tid 2 to trial17818531-5 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: finishing write tid 3 to trial17818531-5 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: finishing write tid 4 to trial17818531-5 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: oid 5 updating version 0 to 1623 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: oid 5 version 1623 is already newer than 1622 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: oid 5 updating version 1623 to 1624 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1624 (ObjNum 1200 snap 347 seq_num 1200) dirty exists 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3423: left oid 5 (ObjNum 1200 snap 347 seq_num 1200) 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3424: finishing write tid 1 to trial17818531-46 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3424: finishing write tid 2 to trial17818531-46 2026-02-21T22:30:19.591 INFO:tasks.rados.rados.0.trial178.stdout:3424: finishing write tid 3 to trial17818531-46 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3424: finishing write tid 4 to trial17818531-46 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3424: oid 46 updating version 0 to 1395 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3424: oid 46 version 1395 is already newer than 1394 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3424: oid 46 version 1395 is already newer than 1393 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1395 (ObjNum 1201 snap 347 seq_num 1201) dirty exists 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3424: left oid 46 (ObjNum 1201 snap 347 seq_num 1201) 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: finishing write tid 1 to trial17818531-11 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: finishing write tid 2 to trial17818531-11 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: finishing write tid 3 to trial17818531-11 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: finishing write tid 4 to trial17818531-11 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: oid 11 updating version 0 to 1667 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: oid 11 version 1667 is already newer than 1665 2026-02-21T22:30:19.592 INFO:tasks.rados.rados.0.trial178.stdout:3425: oid 11 version 1667 is already newer than 1666 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1667 (ObjNum 1202 snap 347 seq_num 1202) dirty exists 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3425: left oid 11 (ObjNum 1202 snap 347 seq_num 1202) 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: finishing write tid 1 to trial17818531-19 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: finishing write tid 2 to trial17818531-19 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: finishing write tid 3 to trial17818531-19 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: finishing write tid 4 to trial17818531-19 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: oid 19 updating version 0 to 1848 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: oid 19 updating version 1848 to 1849 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: oid 19 version 1849 is already newer than 1847 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1849 (ObjNum 1204 snap 347 seq_num 1204) dirty exists 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3427: left oid 19 (ObjNum 1204 snap 347 seq_num 1204) 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:3429: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:19.593 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:19.615 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:19.641 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3432: finishing write tid 1 to trial17818531-23 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3432: finishing write tid 2 to trial17818531-23 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3422: done (11 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3423: done (10 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3424: done (9 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3425: done (8 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3427: done (7 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3428: done (6 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3429: done (5 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3430: done (4 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3431: done (3 left) 2026-02-21T22:30:19.667 INFO:tasks.rados.rados.0.trial178.stdout:3435: read oid 44 snap -1 2026-02-21T22:30:19.668 INFO:tasks.rados.rados.0.trial178.stdout:3435: expect (ObjNum 1158 snap 331 seq_num 1158) 2026-02-21T22:30:19.668 INFO:tasks.rados.rados.0.trial178.stdout:3436: snap_remove snap 346 2026-02-21T22:30:20.217 INFO:tasks.rados.rados.0.trial178.stdout:3436: done (4 left) 2026-02-21T22:30:20.217 INFO:tasks.rados.rados.0.trial178.stdout:3437: rollback oid 2 current snap is 347 2026-02-21T22:30:20.217 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 342 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: finishing write tid 3 to trial17818531-23 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: finishing write tid 4 to trial17818531-23 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: oid 23 updating version 0 to 1396 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: oid 23 updating version 1396 to 1397 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: oid 23 updating version 1397 to 1398 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1398 (ObjNum 1205 snap 347 seq_num 1205) dirty exists 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: left oid 23 (ObjNum 1205 snap 347 seq_num 1205) 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3432: done (4 left) 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3438: read oid 26 snap 344 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3438: expect deleted 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3439: write (excl) oid 37 current snap is 347 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3433: finishing copy_from racing read to trial17818531-33 2026-02-21T22:30:20.218 INFO:tasks.rados.rados.0.trial178.stdout:3433: finishing copy_from to trial17818531-33 2026-02-21T22:30:20.219 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1800 (ObjNum 1156 snap 331 seq_num 1156) dirty exists 2026-02-21T22:30:20.219 INFO:tasks.rados.rados.0.trial178.stdout:3434: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:30:20.219 INFO:tasks.rados.rados.0.trial178.stdout:3439: seq_num 1206 ranges {669242=734068,2188931=80311} 2026-02-21T22:30:20.227 INFO:tasks.rados.rados.0.trial178.stdout:3439: writing trial17818531-37 from 669242 to 1403310 tid 1 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3439: writing trial17818531-37 from 2188931 to 2269242 tid 2 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3433: done (5 left) 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3440: read oid 17 snap 345 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3440: expect (ObjNum 1166 snap 333 seq_num 1166) 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3441: copy_from oid 19 from oid 29 current snap is 347 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3434: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3434: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1624 (ObjNum 1178 snap 336 seq_num 1178) dirty exists 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3434: done (6 left) 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3442: copy_from oid 3 from oid 18 current snap is 347 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:3435: expect (ObjNum 1158 snap 331 seq_num 1158) 2026-02-21T22:30:20.233 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:20.289 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 867737 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3437: finishing rollback tid 0 to trial17818531-2 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3437: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 1250 (ObjNum 81 snap 0 seq_num 1) dirty dne 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: finishing write tid 1 to trial17818531-37 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: finishing write tid 2 to trial17818531-37 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: finishing write tid 3 to trial17818531-37 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: finishing write tid 4 to trial17818531-37 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: oid 37 updating version 0 to 1627 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: oid 37 version 1627 is already newer than 1625 2026-02-21T22:30:20.309 INFO:tasks.rados.rados.0.trial178.stdout:3439: oid 37 version 1627 is already newer than 1626 2026-02-21T22:30:20.310 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1627 (ObjNum 1206 snap 347 seq_num 1206) dirty exists 2026-02-21T22:30:20.310 INFO:tasks.rados.rados.0.trial178.stdout:3439: left oid 37 (ObjNum 1206 snap 347 seq_num 1206) 2026-02-21T22:30:20.310 INFO:tasks.rados.rados.0.trial178.stdout:3440: expect (ObjNum 1166 snap 333 seq_num 1166) 2026-02-21T22:30:20.380 INFO:tasks.rados.rados.0.trial178.stdout:3441: finishing copy_from racing read to trial17818531-19 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3441: finishing copy_from to trial17818531-19 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1621 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3442: finishing copy_from racing read to trial17818531-3 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3442: finishing copy_from to trial17818531-3 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1621 (ObjNum 1187 snap 337 seq_num 1187) dirty exists 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3435: done (6 left) 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3437: done (5 left) 2026-02-21T22:30:20.381 INFO:tasks.rados.rados.0.trial178.stdout:3438: done (4 left) 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3439: done (3 left) 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3440: done (2 left) 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3441: done (1 left) 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3442: done (0 left) 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3443: write (excl) oid 36 current snap is 347 2026-02-21T22:30:20.382 INFO:tasks.rados.rados.0.trial178.stdout:3443: seq_num 1207 ranges {686049=409317,1847413=438636} 2026-02-21T22:30:20.387 INFO:tasks.rados.rados.0.trial178.stdout:3443: writing trial17818531-36 from 686049 to 1095366 tid 1 2026-02-21T22:30:20.395 INFO:tasks.rados.rados.0.trial178.stdout:3443: writing trial17818531-36 from 1847413 to 2286049 tid 2 2026-02-21T22:30:20.395 INFO:tasks.rados.rados.0.trial178.stdout:3444: delete oid 27 current snap is 347 2026-02-21T22:30:20.396 INFO:tasks.rados.rados.0.trial178.stdout:3443: finishing write tid 1 to trial17818531-36 2026-02-21T22:30:20.397 INFO:tasks.rados.rados.0.trial178.stdout:3444: done (1 left) 2026-02-21T22:30:20.397 INFO:tasks.rados.rados.0.trial178.stdout:3445: write oid 24 current snap is 347 2026-02-21T22:30:20.397 INFO:tasks.rados.rados.0.trial178.stdout:3445: seq_num 1209 ranges {719663=676168,1964377=355286} 2026-02-21T22:30:20.406 INFO:tasks.rados.rados.0.trial178.stdout:3445: writing trial17818531-24 from 719663 to 1395831 tid 1 2026-02-21T22:30:20.412 INFO:tasks.rados.rados.0.trial178.stdout:3445: writing trial17818531-24 from 1964377 to 2319663 tid 2 2026-02-21T22:30:20.412 INFO:tasks.rados.rados.0.trial178.stdout:3446: snap_remove snap 343 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3446: done (2 left) 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3447: read oid 33 snap 338 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3447: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3448: read oid 2 snap 345 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3448: expect (ObjNum 1194 snap 345 seq_num 1194) 2026-02-21T22:30:21.222 INFO:tasks.rados.rados.0.trial178.stdout:3449: delete oid 22 current snap is 347 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: finishing write tid 2 to trial17818531-36 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: finishing write tid 3 to trial17818531-36 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: finishing write tid 4 to trial17818531-36 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: oid 36 updating version 0 to 1110 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: oid 36 version 1110 is already newer than 1109 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: oid 36 version 1110 is already newer than 1108 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1110 (ObjNum 1207 snap 347 seq_num 1207) dirty exists 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3443: left oid 36 (ObjNum 1207 snap 347 seq_num 1207) 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3445: finishing write tid 1 to trial17818531-24 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3445: finishing write tid 2 to trial17818531-24 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3445: finishing write tid 3 to trial17818531-24 2026-02-21T22:30:21.223 INFO:tasks.rados.rados.0.trial178.stdout:3445: finishing write tid 4 to trial17818531-24 2026-02-21T22:30:21.224 INFO:tasks.rados.rados.0.trial178.stdout:3445: oid 24 updating version 0 to 1854 2026-02-21T22:30:21.224 INFO:tasks.rados.rados.0.trial178.stdout:3445: oid 24 version 1854 is already newer than 1852 2026-02-21T22:30:21.224 INFO:tasks.rados.rados.0.trial178.stdout:3445: oid 24 version 1854 is already newer than 1853 2026-02-21T22:30:21.224 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1854 (ObjNum 1209 snap 347 seq_num 1209) dirty exists 2026-02-21T22:30:21.224 INFO:tasks.rados.rados.0.trial178.stdout:3445: left oid 24 (ObjNum 1209 snap 347 seq_num 1209) 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3443: done (4 left) 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3445: done (3 left) 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3449: done (2 left) 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3450: copy_from oid 29 from oid 4 current snap is 347 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3451: read oid 45 snap -1 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3451: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3452: rollback oid 12 current snap is 347 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 12 to 344 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3453: rollback oid 22 current snap is 347 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 342 2026-02-21T22:30:21.227 INFO:tasks.rados.rados.0.trial178.stdout:3454: snap_remove snap 337 2026-02-21T22:30:22.205 INFO:tasks.rados.rados.0.trial178.stdout:3454: done (6 left) 2026-02-21T22:30:22.205 INFO:tasks.rados.rados.0.trial178.stdout:3455: read oid 11 snap -1 2026-02-21T22:30:22.205 INFO:tasks.rados.rados.0.trial178.stdout:3455: expect (ObjNum 1202 snap 347 seq_num 1202) 2026-02-21T22:30:22.205 INFO:tasks.rados.rados.0.trial178.stdout:3456: snap_create 2026-02-21T22:30:22.205 INFO:tasks.rados.rados.0.trial178.stdout:3447: expect (ObjNum 1085 snap 313 seq_num 1085) 2026-02-21T22:30:22.273 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:22.300 INFO:tasks.rados.rados.0.trial178.stdout:3448: expect (ObjNum 1194 snap 345 seq_num 1194) 2026-02-21T22:30:22.308 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2018697 2026-02-21T22:30:22.308 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:22.340 INFO:tasks.rados.rados.0.trial178.stdout:3453: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:30:22.340 INFO:tasks.rados.rados.0.trial178.stdout:3453: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:30:22.340 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1636 (ObjNum 1127 snap 323 seq_num 1127) dirty exists 2026-02-21T22:30:22.340 INFO:tasks.rados.rados.0.trial178.stdout:3451: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:22.495 INFO:tasks.rados.rados.0.trial178.stdout:3452: finishing rollback tid 0 to trial17818531-12 2026-02-21T22:30:22.495 INFO:tasks.rados.rados.0.trial178.stdout:3452: finishing rollback tid 1 to trial17818531-12 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:3452: finishing rollback tid 2 to trial17818531-12 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1634 (ObjNum 1099 snap 313 seq_num 1099) dirty exists 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:3450: finishing copy_from racing read to trial17818531-29 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:3450: finishing copy_from to trial17818531-29 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1628 (ObjNum 1188 snap 338 seq_num 1188) dirty exists 2026-02-21T22:30:22.496 INFO:tasks.rados.rados.0.trial178.stdout:3455: expect (ObjNum 1202 snap 347 seq_num 1202) 2026-02-21T22:30:22.530 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:22.565 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:23.209 INFO:tasks.rados.rados.0.trial178.stdout:3447: done (7 left) 2026-02-21T22:30:23.209 INFO:tasks.rados.rados.0.trial178.stdout:3448: done (6 left) 2026-02-21T22:30:23.209 INFO:tasks.rados.rados.0.trial178.stdout:3450: done (5 left) 2026-02-21T22:30:23.209 INFO:tasks.rados.rados.0.trial178.stdout:3451: done (4 left) 2026-02-21T22:30:23.209 INFO:tasks.rados.rados.0.trial178.stdout:3452: done (3 left) 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3453: done (2 left) 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3455: done (1 left) 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3456: done (0 left) 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3457: read oid 46 snap 340 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3457: expect deleted 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3458: write (excl) oid 43 current snap is 348 2026-02-21T22:30:23.210 INFO:tasks.rados.rados.0.trial178.stdout:3458: seq_num 1211 ranges {753277=426666,1681341=545753} 2026-02-21T22:30:23.218 INFO:tasks.rados.rados.0.trial178.stdout:3458: writing trial17818531-43 from 753277 to 1179943 tid 1 2026-02-21T22:30:23.227 INFO:tasks.rados.rados.0.trial178.stdout:3458: writing trial17818531-43 from 1681341 to 2227094 tid 2 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3459: read oid 33 snap -1 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3459: expect (ObjNum 1156 snap 331 seq_num 1156) 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3460: read oid 13 snap 344 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3460: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3461: copy_from oid 36 from oid 20 current snap is 348 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3458: finishing write tid 1 to trial17818531-43 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3457: done (4 left) 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3462: write oid 41 current snap is 348 2026-02-21T22:30:23.228 INFO:tasks.rados.rados.0.trial178.stdout:3462: seq_num 1212 ranges {770084=501915,2023470=346614} 2026-02-21T22:30:23.235 INFO:tasks.rados.rados.0.trial178.stdout:3462: writing trial17818531-41 from 770084 to 1271999 tid 1 2026-02-21T22:30:23.243 INFO:tasks.rados.rados.0.trial178.stdout:3462: writing trial17818531-41 from 2023470 to 2370084 tid 2 2026-02-21T22:30:23.243 INFO:tasks.rados.rados.0.trial178.stdout:3463: copy_from oid 4 from oid 21 current snap is 348 2026-02-21T22:30:23.243 INFO:tasks.rados.rados.0.trial178.stdout:3458: finishing write tid 2 to trial17818531-43 2026-02-21T22:30:23.243 INFO:tasks.rados.rados.0.trial178.stdout:3458: finishing write tid 3 to trial17818531-43 2026-02-21T22:30:23.243 INFO:tasks.rados.rados.0.trial178.stdout:3464: snap_remove snap 347 2026-02-21T22:30:23.538 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:30:23] 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:30:23, mode upmap, max misplaced 0.050000 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['rbd', '.mgr', 'unique_pool_0'] 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:30:23.539 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:30:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:23.553 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:30:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:30:23.573 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:30:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:30:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:30:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:30:23.574 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.575 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:30:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:30:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:30:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:30:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "716", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 716, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 357, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:30:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.000334238123010706 of space, bias 1.0, pg target 0.1002714369032118 quantized to 32 (current 16) 2026-02-21T22:30:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:30:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:24.838 INFO:tasks.rados.rados.0.trial178.stdout:3464: done (6 left) 2026-02-21T22:30:24.838 INFO:tasks.rados.rados.0.trial178.stdout:3465: write oid 49 current snap is 348 2026-02-21T22:30:24.839 INFO:tasks.rados.rados.0.trial178.stdout:3465: seq_num 1213 ranges {786891=577164,2081952=304939} 2026-02-21T22:30:24.847 INFO:tasks.rados.rados.0.trial178.stdout:3465: writing trial17818531-49 from 786891 to 1364055 tid 1 2026-02-21T22:30:24.854 INFO:tasks.rados.rados.0.trial178.stdout:3465: writing trial17818531-49 from 2081952 to 2386891 tid 2 2026-02-21T22:30:24.854 INFO:tasks.rados.rados.0.trial178.stdout:3466: snap_remove snap 338 2026-02-21T22:30:25.839 INFO:tasks.rados.rados.0.trial178.stdout:3466: done (7 left) 2026-02-21T22:30:25.839 INFO:tasks.rados.rados.0.trial178.stdout:3467: write (excl) oid 15 current snap is 348 2026-02-21T22:30:25.839 INFO:tasks.rados.rados.0.trial178.stdout:3467: seq_num 1214 ranges {403698=652413,1740434=663264} 2026-02-21T22:30:25.846 INFO:tasks.rados.rados.0.trial178.stdout:3467: writing trial17818531-15 from 403698 to 1056111 tid 1 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3467: writing trial17818531-15 from 1740434 to 2403698 tid 2 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3468: write (excl) oid 46 current snap is 348 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3458: finishing write tid 4 to trial17818531-43 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3458: oid 43 updating version 0 to 1576 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3458: oid 43 updating version 1576 to 1578 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3458: oid 43 version 1578 is already newer than 1577 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1578 (ObjNum 1211 snap 348 seq_num 1211) dirty exists 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3458: left oid 43 (ObjNum 1211 snap 348 seq_num 1211) 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:3460: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:30:25.855 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:25.934 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3621134 2026-02-21T22:30:25.976 INFO:tasks.rados.rados.0.trial178.stdout:3459: expect (ObjNum 1156 snap 331 seq_num 1156) 2026-02-21T22:30:25.990 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3461: finishing copy_from racing read to trial17818531-36 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3461: finishing copy_from to trial17818531-36 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1318 (ObjNum 1199 snap 347 seq_num 1199) dirty exists 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3462: finishing write tid 1 to trial17818531-41 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3462: finishing write tid 2 to trial17818531-41 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3462: finishing write tid 3 to trial17818531-41 2026-02-21T22:30:26.042 INFO:tasks.rados.rados.0.trial178.stdout:3468: seq_num 1215 ranges {420505=727662,1798916=553091} 2026-02-21T22:30:26.049 INFO:tasks.rados.rados.0.trial178.stdout:3468: writing trial17818531-46 from 420505 to 1148167 tid 1 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3468: writing trial17818531-46 from 1798916 to 2352007 tid 2 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3458: done (8 left) 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3459: done (7 left) 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3460: done (6 left) 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3461: done (5 left) 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3469: delete oid 42 current snap is 348 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3462: finishing write tid 4 to trial17818531-41 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3462: oid 41 updating version 0 to 1326 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3462: oid 41 updating version 1326 to 1327 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3462: oid 41 version 1327 is already newer than 1325 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1327 (ObjNum 1212 snap 348 seq_num 1212) dirty exists 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3462: left oid 41 (ObjNum 1212 snap 348 seq_num 1212) 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3463: finishing copy_from racing read to trial17818531-4 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3463: finishing copy_from to trial17818531-4 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1830 (ObjNum 1175 snap 336 seq_num 1175) dirty exists 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3465: finishing write tid 1 to trial17818531-49 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3465: finishing write tid 2 to trial17818531-49 2026-02-21T22:30:26.058 INFO:tasks.rados.rados.0.trial178.stdout:3465: finishing write tid 3 to trial17818531-49 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3465: finishing write tid 4 to trial17818531-49 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3465: oid 49 updating version 0 to 1487 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3465: oid 49 updating version 1487 to 1489 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3465: oid 49 version 1489 is already newer than 1488 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1489 (ObjNum 1213 snap 348 seq_num 1213) dirty exists 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3465: left oid 49 (ObjNum 1213 snap 348 seq_num 1213) 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: finishing write tid 1 to trial17818531-15 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: finishing write tid 2 to trial17818531-15 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: finishing write tid 3 to trial17818531-15 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: finishing write tid 4 to trial17818531-15 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: oid 15 updating version 0 to 1332 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: oid 15 updating version 1332 to 1333 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: oid 15 version 1333 is already newer than 1331 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1333 (ObjNum 1214 snap 348 seq_num 1214) dirty exists 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3467: left oid 15 (ObjNum 1214 snap 348 seq_num 1214) 2026-02-21T22:30:26.059 INFO:tasks.rados.rados.0.trial178.stdout:3468: finishing write tid 1 to trial17818531-46 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: finishing write tid 2 to trial17818531-46 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: finishing write tid 3 to trial17818531-46 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: finishing write tid 4 to trial17818531-46 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: oid 46 updating version 0 to 1400 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: oid 46 version 1400 is already newer than 1399 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:3468: oid 46 updating version 1400 to 1401 2026-02-21T22:30:26.062 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1401 (ObjNum 1215 snap 348 seq_num 1215) dirty exists 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3468: left oid 46 (ObjNum 1215 snap 348 seq_num 1215) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3462: done (5 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3463: done (4 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3465: done (3 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3467: done (2 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3468: done (1 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3469: done (0 left) 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3470: read oid 25 snap -1 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3470: expect deleted 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3471: copy_from oid 41 from oid 44 current snap is 348 2026-02-21T22:30:26.063 INFO:tasks.rados.rados.0.trial178.stdout:3472: delete oid 8 current snap is 348 2026-02-21T22:30:26.064 INFO:tasks.rados.rados.0.trial178.stdout:3470: done (2 left) 2026-02-21T22:30:26.065 INFO:tasks.rados.rados.0.trial178.stdout:3472: done (1 left) 2026-02-21T22:30:26.065 INFO:tasks.rados.rados.0.trial178.stdout:3473: write (excl) oid 32 current snap is 348 2026-02-21T22:30:26.065 INFO:tasks.rados.rados.0.trial178.stdout:3473: seq_num 1218 ranges {470926=669762,1574362=700418} 2026-02-21T22:30:26.073 INFO:tasks.rados.rados.0.trial178.stdout:3473: writing trial17818531-32 from 470926 to 1140688 tid 1 2026-02-21T22:30:26.083 INFO:tasks.rados.rados.0.trial178.stdout:3473: writing trial17818531-32 from 1574362 to 2274780 tid 2 2026-02-21T22:30:26.083 INFO:tasks.rados.rados.0.trial178.stdout:3474: rollback oid 35 current snap is 348 2026-02-21T22:30:26.083 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 35 to 344 2026-02-21T22:30:26.083 INFO:tasks.rados.rados.0.trial178.stdout:3471: finishing copy_from racing read to trial17818531-41 2026-02-21T22:30:26.083 INFO:tasks.rados.rados.0.trial178.stdout:3471: finishing copy_from to trial17818531-41 2026-02-21T22:30:26.084 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1619 (ObjNum 1158 snap 331 seq_num 1158) dirty exists 2026-02-21T22:30:26.084 INFO:tasks.rados.rados.0.trial178.stdout:3473: finishing write tid 1 to trial17818531-32 2026-02-21T22:30:26.084 INFO:tasks.rados.rados.0.trial178.stdout:3471: done (2 left) 2026-02-21T22:30:26.084 INFO:tasks.rados.rados.0.trial178.stdout:3475: write (excl) oid 6 current snap is 348 2026-02-21T22:30:26.084 INFO:tasks.rados.rados.0.trial178.stdout:3475: seq_num 1219 ranges {487733=745011,1632844=560429} 2026-02-21T22:30:26.091 INFO:tasks.rados.rados.0.trial178.stdout:3475: writing trial17818531-6 from 487733 to 1232744 tid 1 2026-02-21T22:30:26.098 INFO:tasks.rados.rados.0.trial178.stdout:3475: writing trial17818531-6 from 1632844 to 2193273 tid 2 2026-02-21T22:30:26.098 INFO:tasks.rados.rados.0.trial178.stdout:3476: snap_remove snap 345 2026-02-21T22:30:26.842 INFO:tasks.rados.rados.0.trial178.stdout:3476: done (3 left) 2026-02-21T22:30:26.842 INFO:tasks.rados.rados.0.trial178.stdout:3477: write (excl) oid 49 current snap is 348 2026-02-21T22:30:26.842 INFO:tasks.rados.rados.0.trial178.stdout:3477: seq_num 1220 ranges {504540=420260,1574973=704087} 2026-02-21T22:30:26.848 INFO:tasks.rados.rados.0.trial178.stdout:3477: writing trial17818531-49 from 504540 to 924800 tid 1 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3477: writing trial17818531-49 from 1574973 to 2279060 tid 2 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3478: rollback oid 40 current snap is 348 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 339 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3473: finishing write tid 2 to trial17818531-32 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3473: finishing write tid 3 to trial17818531-32 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3474: finishing rollback tid 1 to trial17818531-35 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3473: finishing write tid 4 to trial17818531-32 2026-02-21T22:30:26.858 INFO:tasks.rados.rados.0.trial178.stdout:3473: oid 32 updating version 0 to 1692 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3473: oid 32 version 1692 is already newer than 1691 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3473: oid 32 updating version 1692 to 1693 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1693 (ObjNum 1218 snap 348 seq_num 1218) dirty exists 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3473: left oid 32 (ObjNum 1218 snap 348 seq_num 1218) 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3474: finishing rollback tid 2 to trial17818531-35 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1695 (ObjNum 1093 snap 313 seq_num 1093) dirty exists 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3475: finishing write tid 1 to trial17818531-6 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3475: finishing write tid 2 to trial17818531-6 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3475: finishing write tid 3 to trial17818531-6 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3473: done (4 left) 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3474: done (3 left) 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3479: write oid 12 current snap is 348 2026-02-21T22:30:26.859 INFO:tasks.rados.rados.0.trial178.stdout:3479: seq_num 1221 ranges {521347=495509,1633455=564098} 2026-02-21T22:30:26.864 INFO:tasks.rados.rados.0.trial178.stdout:3479: writing trial17818531-12 from 521347 to 1016856 tid 1 2026-02-21T22:30:26.872 INFO:tasks.rados.rados.0.trial178.stdout:3479: writing trial17818531-12 from 1633455 to 2197553 tid 2 2026-02-21T22:30:26.872 INFO:tasks.rados.rados.0.trial178.stdout:3480: snap_remove snap 342 2026-02-21T22:30:27.205 INFO:tasks.rados.rados.0.trial178.stdout:3480: done (4 left) 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3481: delete oid 14 current snap is 348 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3475: finishing write tid 4 to trial17818531-6 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3475: oid 6 updating version 0 to 1253 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3475: oid 6 version 1253 is already newer than 1252 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3475: oid 6 version 1253 is already newer than 1251 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1253 (ObjNum 1219 snap 348 seq_num 1219) dirty exists 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3475: left oid 6 (ObjNum 1219 snap 348 seq_num 1219) 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: finishing write tid 1 to trial17818531-49 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: finishing write tid 2 to trial17818531-49 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3478: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: finishing write tid 3 to trial17818531-49 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3478: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3478: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1639 (ObjNum 1134 snap 325 seq_num 1134) dirty exists 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: finishing write tid 4 to trial17818531-49 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: oid 49 updating version 0 to 1491 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: oid 49 version 1491 is already newer than 1490 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:3477: oid 49 updating version 1491 to 1492 2026-02-21T22:30:27.206 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1492 (ObjNum 1220 snap 348 seq_num 1220) dirty exists 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3477: left oid 49 (ObjNum 1220 snap 348 seq_num 1220) 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: finishing write tid 1 to trial17818531-12 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: finishing write tid 2 to trial17818531-12 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: finishing write tid 3 to trial17818531-12 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: finishing write tid 4 to trial17818531-12 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: oid 12 updating version 0 to 1832 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: oid 12 version 1832 is already newer than 1831 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: oid 12 updating version 1832 to 1833 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1833 (ObjNum 1221 snap 348 seq_num 1221) dirty exists 2026-02-21T22:30:27.207 INFO:tasks.rados.rados.0.trial178.stdout:3479: left oid 12 (ObjNum 1221 snap 348 seq_num 1221) 2026-02-21T22:30:27.210 INFO:tasks.rados.rados.0.trial178.stdout:3475: done (4 left) 2026-02-21T22:30:27.210 INFO:tasks.rados.rados.0.trial178.stdout:3477: done (3 left) 2026-02-21T22:30:27.210 INFO:tasks.rados.rados.0.trial178.stdout:3478: done (2 left) 2026-02-21T22:30:27.211 INFO:tasks.rados.rados.0.trial178.stdout:3479: done (1 left) 2026-02-21T22:30:27.211 INFO:tasks.rados.rados.0.trial178.stdout:3481: done (0 left) 2026-02-21T22:30:27.211 INFO:tasks.rados.rados.0.trial178.stdout:3482: delete oid 34 current snap is 348 2026-02-21T22:30:27.213 INFO:tasks.rados.rados.0.trial178.stdout:3482: done (0 left) 2026-02-21T22:30:27.214 INFO:tasks.rados.rados.0.trial178.stdout:3483: rollback oid 10 current snap is 348 2026-02-21T22:30:27.214 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 339 2026-02-21T22:30:27.214 INFO:tasks.rados.rados.0.trial178.stdout:3484: read oid 14 snap 340 2026-02-21T22:30:27.214 INFO:tasks.rados.rados.0.trial178.stdout:3484: expect (ObjNum 1162 snap 332 seq_num 1162) 2026-02-21T22:30:27.214 INFO:tasks.rados.rados.0.trial178.stdout:3485: delete oid 49 current snap is 348 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3483: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3483: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1255 (ObjNum 289 snap 0 seq_num 1398762627) dirty dne 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3483: done (2 left) 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3485: done (1 left) 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3486: rollback oid 7 current snap is 348 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 7 to 344 2026-02-21T22:30:27.215 INFO:tasks.rados.rados.0.trial178.stdout:3487: delete oid 4 current snap is 348 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3487: done (2 left) 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3488: read oid 8 snap 340 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3488: expect (ObjNum 1101 snap 314 seq_num 1101) 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3489: read oid 12 snap 339 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3489: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:27.216 INFO:tasks.rados.rados.0.trial178.stdout:3490: delete oid 5 current snap is 348 2026-02-21T22:30:27.218 INFO:tasks.rados.rados.0.trial178.stdout:3490: done (4 left) 2026-02-21T22:30:27.218 INFO:tasks.rados.rados.0.trial178.stdout:3491: delete oid 33 current snap is 348 2026-02-21T22:30:27.219 INFO:tasks.rados.rados.0.trial178.stdout:3491: done (4 left) 2026-02-21T22:30:27.219 INFO:tasks.rados.rados.0.trial178.stdout:3492: write (excl) oid 50 current snap is 348 2026-02-21T22:30:27.219 INFO:tasks.rados.rados.0.trial178.stdout:3492: seq_num 1228 ranges {638996=738605,1926476=712520} 2026-02-21T22:30:27.229 INFO:tasks.rados.rados.0.trial178.stdout:3492: writing trial17818531-50 from 638996 to 1377601 tid 1 2026-02-21T22:30:27.237 INFO:tasks.rados.rados.0.trial178.stdout:3492: writing trial17818531-50 from 1926476 to 2638996 tid 2 2026-02-21T22:30:27.238 INFO:tasks.rados.rados.0.trial178.stdout:3493: read oid 16 snap 344 2026-02-21T22:30:27.238 INFO:tasks.rados.rados.0.trial178.stdout:3493: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:30:27.238 INFO:tasks.rados.rados.0.trial178.stdout:3494: copy_from oid 17 from oid 42 current snap is 348 2026-02-21T22:30:27.238 INFO:tasks.rados.rados.0.trial178.stdout:3484: expect (ObjNum 1162 snap 332 seq_num 1162) 2026-02-21T22:30:27.249 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1322191 2026-02-21T22:30:27.361 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2509911 2026-02-21T22:30:27.373 INFO:tasks.rados.rados.0.trial178.stdout:3489: expect (ObjNum 1099 snap 313 seq_num 1099) 2026-02-21T22:30:27.385 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1801037 2026-02-21T22:30:27.444 INFO:tasks.rados.rados.0.trial178.stdout:3486: finishing rollback tid 1 to trial17818531-7 2026-02-21T22:30:27.444 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 0 (ObjNum 305 snap 0 seq_num 1397636412) dirty dne 2026-02-21T22:30:27.444 INFO:tasks.rados.rados.0.trial178.stdout:3488: expect (ObjNum 1101 snap 314 seq_num 1101) 2026-02-21T22:30:27.444 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:27.504 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: finishing write tid 1 to trial17818531-50 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: finishing write tid 2 to trial17818531-50 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3484: done (6 left) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3486: done (5 left) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3488: done (4 left) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3489: done (3 left) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3495: snap_create 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: finishing write tid 3 to trial17818531-50 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3494: finishing copy_from to trial17818531-17 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3494: got expected ENOENT (src dne) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: finishing write tid 4 to trial17818531-50 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: oid 50 updating version 0 to 1640 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: oid 50 updating version 1640 to 1642 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: oid 50 version 1642 is already newer than 1641 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1642 (ObjNum 1228 snap 348 seq_num 1228) dirty exists 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3492: left oid 50 (ObjNum 1228 snap 348 seq_num 1228) 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3494: finishing copy_from racing read to trial17818531-17 2026-02-21T22:30:27.534 INFO:tasks.rados.rados.0.trial178.stdout:3493: expect (ObjNum 1050 snap 303 seq_num 1050) 2026-02-21T22:30:27.551 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:28.211 INFO:tasks.rados.rados.0.trial178.stdout:3492: done (3 left) 2026-02-21T22:30:28.211 INFO:tasks.rados.rados.0.trial178.stdout:3493: done (2 left) 2026-02-21T22:30:28.211 INFO:tasks.rados.rados.0.trial178.stdout:3494: done (1 left) 2026-02-21T22:30:28.211 INFO:tasks.rados.rados.0.trial178.stdout:3495: done (0 left) 2026-02-21T22:30:28.211 INFO:tasks.rados.rados.0.trial178.stdout:3496: snap_remove snap 339 2026-02-21T22:30:28.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:28.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:29.211 INFO:tasks.rados.rados.0.trial178.stdout:3496: done (0 left) 2026-02-21T22:30:29.212 INFO:tasks.rados.rados.0.trial178.stdout:3497: copy_from oid 40 from oid 13 current snap is 349 2026-02-21T22:30:29.212 INFO:tasks.rados.rados.0.trial178.stdout:3498: snap_create 2026-02-21T22:30:29.223 INFO:tasks.rados.rados.0.trial178.stdout:3497: finishing copy_from racing read to trial17818531-40 2026-02-21T22:30:29.223 INFO:tasks.rados.rados.0.trial178.stdout:3497: finishing copy_from to trial17818531-40 2026-02-21T22:30:29.224 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1647 (ObjNum 1186 snap 337 seq_num 1186) dirty exists 2026-02-21T22:30:30.216 INFO:tasks.rados.rados.0.trial178.stdout:3497: done (1 left) 2026-02-21T22:30:30.217 INFO:tasks.rados.rados.0.trial178.stdout:3498: done (0 left) 2026-02-21T22:30:30.217 INFO:tasks.rados.rados.0.trial178.stdout:3499: rollback oid 38 current snap is 350 2026-02-21T22:30:30.217 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 348 2026-02-21T22:30:30.217 INFO:tasks.rados.rados.0.trial178.stdout:3500: delete oid 27 current snap is 350 2026-02-21T22:30:30.220 INFO:tasks.rados.rados.0.trial178.stdout:3500: done (1 left) 2026-02-21T22:30:30.220 INFO:tasks.rados.rados.0.trial178.stdout:3501: read oid 32 snap 348 2026-02-21T22:30:30.220 INFO:tasks.rados.rados.0.trial178.stdout:3501: expect (ObjNum 1218 snap 348 seq_num 1218) 2026-02-21T22:30:30.220 INFO:tasks.rados.rados.0.trial178.stdout:3502: snap_remove snap 349 2026-02-21T22:30:31.218 INFO:tasks.rados.rados.0.trial178.stdout:3502: done (2 left) 2026-02-21T22:30:31.218 INFO:tasks.rados.rados.0.trial178.stdout:3503: write (excl) oid 6 current snap is 350 2026-02-21T22:30:31.218 INFO:tasks.rados.rados.0.trial178.stdout:3503: seq_num 1230 ranges {672610=489103,1643440=722432} 2026-02-21T22:30:31.226 INFO:tasks.rados.rados.0.trial178.stdout:3503: writing trial17818531-6 from 672610 to 1161713 tid 1 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3503: writing trial17818531-6 from 1643440 to 2365872 tid 2 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3504: write oid 23 current snap is 350 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3499: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3499: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3499: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1804 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:30:31.235 INFO:tasks.rados.rados.0.trial178.stdout:3501: expect (ObjNum 1218 snap 348 seq_num 1218) 2026-02-21T22:30:31.308 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:31.335 INFO:tasks.rados.rados.0.trial178.stdout:3503: finishing write tid 1 to trial17818531-6 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: finishing write tid 2 to trial17818531-6 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: finishing write tid 3 to trial17818531-6 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: finishing write tid 4 to trial17818531-6 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: oid 6 updating version 0 to 1257 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: oid 6 version 1257 is already newer than 1256 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: oid 6 updating version 1257 to 1258 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1258 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3503: left oid 6 (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:30:31.336 INFO:tasks.rados.rados.0.trial178.stdout:3504: seq_num 1231 ranges {689417=564352,1701922=466090} 2026-02-21T22:30:31.343 INFO:tasks.rados.rados.0.trial178.stdout:3504: writing trial17818531-23 from 689417 to 1253769 tid 1 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3504: writing trial17818531-23 from 1701922 to 2168012 tid 2 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3499: done (3 left) 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3501: done (2 left) 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3503: done (1 left) 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3505: write (excl) oid 27 current snap is 350 2026-02-21T22:30:31.350 INFO:tasks.rados.rados.0.trial178.stdout:3505: seq_num 1232 ranges {706224=755954,2160404=545820} 2026-02-21T22:30:31.359 INFO:tasks.rados.rados.0.trial178.stdout:3505: writing trial17818531-27 from 706224 to 1462178 tid 1 2026-02-21T22:30:31.368 INFO:tasks.rados.rados.0.trial178.stdout:3505: writing trial17818531-27 from 2160404 to 2706224 tid 2 2026-02-21T22:30:31.368 INFO:tasks.rados.rados.0.trial178.stdout:3506: write oid 46 current snap is 350 2026-02-21T22:30:31.368 INFO:tasks.rados.rados.0.trial178.stdout:3506: seq_num 1233 ranges {723031=431203,1818886=469759} 2026-02-21T22:30:31.375 INFO:tasks.rados.rados.0.trial178.stdout:3506: writing trial17818531-46 from 723031 to 1154234 tid 1 2026-02-21T22:30:31.382 INFO:tasks.rados.rados.0.trial178.stdout:3506: writing trial17818531-46 from 1818886 to 2288645 tid 2 2026-02-21T22:30:31.382 INFO:tasks.rados.rados.0.trial178.stdout:3507: read oid 44 snap -1 2026-02-21T22:30:31.382 INFO:tasks.rados.rados.0.trial178.stdout:3507: expect (ObjNum 1158 snap 331 seq_num 1158) 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3508: rollback oid 16 current snap is 350 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 340 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: finishing write tid 1 to trial17818531-23 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3509: delete oid 25 current snap is 350 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: finishing write tid 2 to trial17818531-23 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: finishing write tid 3 to trial17818531-23 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: finishing write tid 4 to trial17818531-23 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: oid 23 updating version 0 to 1404 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: oid 23 version 1404 is already newer than 1403 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: oid 23 version 1404 is already newer than 1402 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1404 (ObjNum 1231 snap 350 seq_num 1231) dirty exists 2026-02-21T22:30:31.383 INFO:tasks.rados.rados.0.trial178.stdout:3504: left oid 23 (ObjNum 1231 snap 350 seq_num 1231) 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: finishing write tid 1 to trial17818531-27 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: finishing write tid 2 to trial17818531-27 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: finishing write tid 3 to trial17818531-27 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: finishing write tid 4 to trial17818531-27 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: oid 27 updating version 0 to 1487 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: oid 27 updating version 1487 to 1489 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: oid 27 version 1489 is already newer than 1488 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1489 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3505: left oid 27 (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:30:31.384 INFO:tasks.rados.rados.0.trial178.stdout:3506: finishing write tid 1 to trial17818531-46 2026-02-21T22:30:31.385 INFO:tasks.rados.rados.0.trial178.stdout:3504: done (5 left) 2026-02-21T22:30:31.385 INFO:tasks.rados.rados.0.trial178.stdout:3505: done (4 left) 2026-02-21T22:30:31.385 INFO:tasks.rados.rados.0.trial178.stdout:3509: done (3 left) 2026-02-21T22:30:31.385 INFO:tasks.rados.rados.0.trial178.stdout:3510: write oid 20 current snap is 350 2026-02-21T22:30:31.385 INFO:tasks.rados.rados.0.trial178.stdout:3510: seq_num 1235 ranges {756645=581701,1935850=473428} 2026-02-21T22:30:31.388 INFO:tasks.rados.rados.0.trial178.stdout:3510: writing trial17818531-20 from 756645 to 1338346 tid 1 2026-02-21T22:30:31.392 INFO:tasks.rados.rados.0.trial178.stdout:3510: writing trial17818531-20 from 1935850 to 2409278 tid 2 2026-02-21T22:30:31.392 INFO:tasks.rados.rados.0.trial178.stdout:3511: write (excl) oid 41 current snap is 350 2026-02-21T22:30:31.392 INFO:tasks.rados.rados.0.trial178.stdout:3511: seq_num 1236 ranges {773452=656950,1877979=617086} 2026-02-21T22:30:31.397 INFO:tasks.rados.rados.0.trial178.stdout:3511: writing trial17818531-41 from 773452 to 1430402 tid 1 2026-02-21T22:30:31.402 INFO:tasks.rados.rados.0.trial178.stdout:3511: writing trial17818531-41 from 1877979 to 2495065 tid 2 2026-02-21T22:30:31.402 INFO:tasks.rados.rados.0.trial178.stdout:3512: snap_create 2026-02-21T22:30:31.402 INFO:tasks.rados.rados.0.trial178.stdout:3506: finishing write tid 2 to trial17818531-46 2026-02-21T22:30:31.402 INFO:tasks.rados.rados.0.trial178.stdout:3506: finishing write tid 3 to trial17818531-46 2026-02-21T22:30:31.402 INFO:tasks.rados.rados.0.trial178.stdout:3506: finishing write tid 4 to trial17818531-46 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:3506: oid 46 updating version 0 to 1405 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:3506: oid 46 updating version 1405 to 1406 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:3506: oid 46 updating version 1406 to 1407 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1407 (ObjNum 1233 snap 350 seq_num 1233) dirty exists 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:3506: left oid 46 (ObjNum 1233 snap 350 seq_num 1233) 2026-02-21T22:30:31.403 INFO:tasks.rados.rados.0.trial178.stdout:3507: expect (ObjNum 1158 snap 331 seq_num 1158) 2026-02-21T22:30:31.405 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1288203 2026-02-21T22:30:31.430 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1657697 2026-02-21T22:30:31.443 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:31.469 INFO:tasks.rados.rados.0.trial178.stdout:3508: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:30:31.469 INFO:tasks.rados.rados.0.trial178.stdout:3508: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:30:31.469 INFO:tasks.rados.rados.0.trial178.stdout:3508: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:30:31.469 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1837 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: finishing write tid 1 to trial17818531-20 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: finishing write tid 2 to trial17818531-20 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: finishing write tid 3 to trial17818531-20 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: finishing write tid 4 to trial17818531-20 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: oid 20 updating version 0 to 1620 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: oid 20 updating version 1620 to 1622 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: oid 20 version 1622 is already newer than 1621 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1622 (ObjNum 1235 snap 350 seq_num 1235) dirty exists 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3510: left oid 20 (ObjNum 1235 snap 350 seq_num 1235) 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: finishing write tid 1 to trial17818531-41 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: finishing write tid 2 to trial17818531-41 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: finishing write tid 3 to trial17818531-41 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: finishing write tid 4 to trial17818531-41 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: oid 41 updating version 0 to 1625 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: oid 41 version 1625 is already newer than 1624 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: oid 41 version 1625 is already newer than 1623 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1625 (ObjNum 1236 snap 350 seq_num 1236) dirty exists 2026-02-21T22:30:31.470 INFO:tasks.rados.rados.0.trial178.stdout:3511: left oid 41 (ObjNum 1236 snap 350 seq_num 1236) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3506: done (5 left) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3507: done (4 left) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3508: done (3 left) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3510: done (2 left) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3511: done (1 left) 2026-02-21T22:30:32.207 INFO:tasks.rados.rados.0.trial178.stdout:3512: done (0 left) 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:3513: copy_from oid 43 from oid 30 current snap is 351 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:3514: rollback oid 23 current snap is 351 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 348 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:3515: read oid 41 snap 350 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:3515: expect (ObjNum 1236 snap 350 seq_num 1236) 2026-02-21T22:30:32.208 INFO:tasks.rados.rados.0.trial178.stdout:3516: snap_remove snap 340 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:3516: done (3 left) 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:3517: copy_from oid 10 from oid 13 current snap is 351 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:3514: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:3514: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:3514: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:30:33.211 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1410 (ObjNum 1205 snap 347 seq_num 1205) dirty exists 2026-02-21T22:30:33.212 INFO:tasks.rados.rados.0.trial178.stdout:3515: expect (ObjNum 1236 snap 350 seq_num 1236) 2026-02-21T22:30:33.248 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2648325 2026-02-21T22:30:33.282 INFO:tasks.rados.rados.0.trial178.stdout:3513: finishing copy_from racing read to trial17818531-43 2026-02-21T22:30:33.282 INFO:tasks.rados.rados.0.trial178.stdout:3513: finishing copy_from to trial17818531-43 2026-02-21T22:30:33.282 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1602 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:30:33.282 INFO:tasks.rados.rados.0.trial178.stdout:3513: done (3 left) 2026-02-21T22:30:33.282 INFO:tasks.rados.rados.0.trial178.stdout:3514: done (2 left) 2026-02-21T22:30:33.283 INFO:tasks.rados.rados.0.trial178.stdout:3515: done (1 left) 2026-02-21T22:30:33.283 INFO:tasks.rados.rados.0.trial178.stdout:3518: write oid 46 current snap is 351 2026-02-21T22:30:33.283 INFO:tasks.rados.rados.0.trial178.stdout:3518: seq_num 1237 ranges {790259=732199,1936461=477097} 2026-02-21T22:30:33.292 INFO:tasks.rados.rados.0.trial178.stdout:3518: writing trial17818531-46 from 790259 to 1522458 tid 1 2026-02-21T22:30:33.298 INFO:tasks.rados.rados.0.trial178.stdout:3518: writing trial17818531-46 from 1936461 to 2413558 tid 2 2026-02-21T22:30:33.298 INFO:tasks.rados.rados.0.trial178.stdout:3519: snap_create 2026-02-21T22:30:33.298 INFO:tasks.rados.rados.0.trial178.stdout:3517: finishing copy_from racing read to trial17818531-10 2026-02-21T22:30:33.298 INFO:tasks.rados.rados.0.trial178.stdout:3517: finishing copy_from to trial17818531-10 2026-02-21T22:30:33.298 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1647 (ObjNum 1186 snap 337 seq_num 1186) dirty exists 2026-02-21T22:30:33.306 INFO:tasks.rados.rados.0.trial178.stdout:3518: finishing write tid 1 to trial17818531-46 2026-02-21T22:30:33.308 INFO:tasks.rados.rados.0.trial178.stdout:3518: finishing write tid 2 to trial17818531-46 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: finishing write tid 3 to trial17818531-46 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: finishing write tid 4 to trial17818531-46 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: oid 46 updating version 0 to 1412 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: oid 46 version 1412 is already newer than 1411 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: oid 46 updating version 1412 to 1413 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1413 (ObjNum 1237 snap 351 seq_num 1237) dirty exists 2026-02-21T22:30:33.309 INFO:tasks.rados.rados.0.trial178.stdout:3518: left oid 46 (ObjNum 1237 snap 351 seq_num 1237) 2026-02-21T22:30:33.588 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:33.589 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:34.228 INFO:tasks.rados.rados.0.trial178.stdout:3517: done (2 left) 2026-02-21T22:30:34.228 INFO:tasks.rados.rados.0.trial178.stdout:3518: done (1 left) 2026-02-21T22:30:34.228 INFO:tasks.rados.rados.0.trial178.stdout:3519: done (0 left) 2026-02-21T22:30:34.228 INFO:tasks.rados.rados.0.trial178.stdout:3520: write oid 28 current snap is 352 2026-02-21T22:30:34.229 INFO:tasks.rados.rados.0.trial178.stdout:3520: seq_num 1238 ranges {407066=407448,1594943=620755} 2026-02-21T22:30:34.233 INFO:tasks.rados.rados.0.trial178.stdout:3520: writing trial17818531-28 from 407066 to 814514 tid 1 2026-02-21T22:30:34.244 INFO:tasks.rados.rados.0.trial178.stdout:3520: writing trial17818531-28 from 1594943 to 2215698 tid 2 2026-02-21T22:30:34.245 INFO:tasks.rados.rados.0.trial178.stdout:3521: read oid 19 snap -1 2026-02-21T22:30:34.245 INFO:tasks.rados.rados.0.trial178.stdout:3521: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:34.245 INFO:tasks.rados.rados.0.trial178.stdout:3522: read oid 35 snap -1 2026-02-21T22:30:34.245 INFO:tasks.rados.rados.0.trial178.stdout:3522: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:30:34.246 INFO:tasks.rados.rados.0.trial178.stdout:3523: rollback oid 47 current snap is 352 2026-02-21T22:30:34.246 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 47 to 344 2026-02-21T22:30:34.246 INFO:tasks.rados.rados.0.trial178.stdout:3520: finishing write tid 1 to trial17818531-28 2026-02-21T22:30:34.246 INFO:tasks.rados.rados.0.trial178.stdout:3524: snap_create 2026-02-21T22:30:34.247 INFO:tasks.rados.rados.0.trial178.stdout:3520: finishing write tid 2 to trial17818531-28 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:3520: finishing write tid 3 to trial17818531-28 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:3520: finishing write tid 4 to trial17818531-28 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:3520: oid 28 updating version 0 to 1806 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:3520: oid 28 version 1806 is already newer than 1805 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:3520: oid 28 updating version 1806 to 1807 2026-02-21T22:30:34.248 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1807 (ObjNum 1238 snap 352 seq_num 1238) dirty exists 2026-02-21T22:30:34.249 INFO:tasks.rados.rados.0.trial178.stdout:3520: left oid 28 (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:34.251 INFO:tasks.rados.rados.0.trial178.stdout:3522: expect (ObjNum 1093 snap 313 seq_num 1093) 2026-02-21T22:30:34.339 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1843627 2026-02-21T22:30:34.346 INFO:tasks.rados.rados.0.trial178.stdout:3521: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:34.375 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2656412 2026-02-21T22:30:34.397 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1192792 2026-02-21T22:30:34.490 INFO:tasks.rados.rados.0.trial178.stdout:3523: finishing rollback tid 0 to trial17818531-47 2026-02-21T22:30:34.490 INFO:tasks.rados.rados.0.trial178.stdout:3523: finishing rollback tid 1 to trial17818531-47 2026-02-21T22:30:34.490 INFO:tasks.rados.rados.0.trial178.stdout:3523: finishing rollback tid 2 to trial17818531-47 2026-02-21T22:30:34.490 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1893 (ObjNum 1151 snap 328 seq_num 1151) dirty exists 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3520: done (4 left) 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3521: done (3 left) 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3522: done (2 left) 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3523: done (1 left) 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3524: done (0 left) 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3525: read oid 25 snap 352 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3525: expect deleted 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3526: read oid 42 snap 352 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3526: expect deleted 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3527: read oid 8 snap 348 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3527: expect deleted 2026-02-21T22:30:35.230 INFO:tasks.rados.rados.0.trial178.stdout:3528: snap_remove snap 350 2026-02-21T22:30:36.235 INFO:tasks.rados.rados.0.trial178.stdout:3528: done (3 left) 2026-02-21T22:30:36.235 INFO:tasks.rados.rados.0.trial178.stdout:3529: snap_remove snap 351 2026-02-21T22:30:37.207 INFO:tasks.rados.rados.0.trial178.stdout:3529: done (3 left) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3530: read oid 1 snap -1 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3530: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3531: copy_from oid 47 from oid 41 current snap is 353 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3525: done (4 left) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3526: done (3 left) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3527: done (2 left) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3532: copy_from oid 5 from oid 31 current snap is 353 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3533: copy_from oid 22 from oid 27 current snap is 353 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3534: read oid 45 snap -1 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3534: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3535: rollback oid 16 current snap is 353 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 344 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3536: rollback oid 48 current snap is 353 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 48 to 352 2026-02-21T22:30:37.208 INFO:tasks.rados.rados.0.trial178.stdout:3537: delete oid 35 current snap is 353 2026-02-21T22:30:37.214 INFO:tasks.rados.rados.0.trial178.stdout:3537: done (7 left) 2026-02-21T22:30:37.214 INFO:tasks.rados.rados.0.trial178.stdout:3538: rollback oid 8 current snap is 353 2026-02-21T22:30:37.214 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 352 2026-02-21T22:30:37.214 INFO:tasks.rados.rados.0.trial178.stdout:3539: write (excl) oid 3 current snap is 353 2026-02-21T22:30:37.214 INFO:tasks.rados.rados.0.trial178.stdout:3539: seq_num 1240 ranges {440680=674299,1711907=624424} 2026-02-21T22:30:37.221 INFO:tasks.rados.rados.0.trial178.stdout:3539: writing trial17818531-3 from 440680 to 1114979 tid 1 2026-02-21T22:30:37.230 INFO:tasks.rados.rados.0.trial178.stdout:3539: writing trial17818531-3 from 1711907 to 2336331 tid 2 2026-02-21T22:30:37.230 INFO:tasks.rados.rados.0.trial178.stdout:3540: read oid 26 snap -1 2026-02-21T22:30:37.230 INFO:tasks.rados.rados.0.trial178.stdout:3540: expect deleted 2026-02-21T22:30:37.230 INFO:tasks.rados.rados.0.trial178.stdout:3541: snap_create 2026-02-21T22:30:37.230 INFO:tasks.rados.rados.0.trial178.stdout:3530: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:37.328 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1582575 2026-02-21T22:30:37.344 INFO:tasks.rados.rados.0.trial178.stdout:3532: finishing copy_from racing read to trial17818531-5 2026-02-21T22:30:37.344 INFO:tasks.rados.rados.0.trial178.stdout:3531: finishing copy_from racing read to trial17818531-47 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:3532: finishing copy_from to trial17818531-5 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1485 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:3538: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1864 (ObjNum 305 snap 0 seq_num 0) dirty dne 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:3533: finishing copy_from racing read to trial17818531-22 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:3533: finishing copy_from to trial17818531-22 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1489 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:37.345 INFO:tasks.rados.rados.0.trial178.stdout:3534: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:37.471 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2586906 2026-02-21T22:30:37.486 INFO:tasks.rados.rados.0.trial178.stdout:3531: finishing copy_from to trial17818531-47 2026-02-21T22:30:37.486 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1625 (ObjNum 1236 snap 350 seq_num 1236) dirty exists 2026-02-21T22:30:37.486 INFO:tasks.rados.rados.0.trial178.stdout:3535: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:30:37.486 INFO:tasks.rados.rados.0.trial178.stdout:3535: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3535: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1840 (ObjNum 1050 snap 303 seq_num 1050) dirty exists 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3536: finishing rollback tid 0 to trial17818531-48 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3536: finishing rollback tid 1 to trial17818531-48 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: finishing write tid 1 to trial17818531-3 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3536: finishing rollback tid 2 to trial17818531-48 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1630 (ObjNum 1178 snap 336 seq_num 1178) dirty exists 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: finishing write tid 2 to trial17818531-3 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: finishing write tid 3 to trial17818531-3 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: finishing write tid 4 to trial17818531-3 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: oid 3 updating version 0 to 1729 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: oid 3 updating version 1729 to 1731 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: oid 3 version 1731 is already newer than 1730 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1731 (ObjNum 1240 snap 353 seq_num 1240) dirty exists 2026-02-21T22:30:37.487 INFO:tasks.rados.rados.0.trial178.stdout:3539: left oid 3 (ObjNum 1240 snap 353 seq_num 1240) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3530: done (10 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3531: done (9 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3532: done (8 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3533: done (7 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3534: done (6 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3535: done (5 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3536: done (4 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3538: done (3 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3539: done (2 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3540: done (1 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3541: done (0 left) 2026-02-21T22:30:38.212 INFO:tasks.rados.rados.0.trial178.stdout:3542: read oid 45 snap -1 2026-02-21T22:30:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3542: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3543: write (excl) oid 25 current snap is 354 2026-02-21T22:30:38.213 INFO:tasks.rados.rados.0.trial178.stdout:3543: seq_num 1241 ranges {457487=749548,1770389=484435,2746287=111200} 2026-02-21T22:30:38.220 INFO:tasks.rados.rados.0.trial178.stdout:3543: writing trial17818531-25 from 457487 to 1207035 tid 1 2026-02-21T22:30:38.227 INFO:tasks.rados.rados.0.trial178.stdout:3543: writing trial17818531-25 from 1770389 to 2254824 tid 2 2026-02-21T22:30:38.231 INFO:tasks.rados.rados.0.trial178.stdout:3543: writing trial17818531-25 from 2746287 to 2857487 tid 3 2026-02-21T22:30:38.231 INFO:tasks.rados.rados.0.trial178.stdout:3544: delete oid 10 current snap is 354 2026-02-21T22:30:38.232 INFO:tasks.rados.rados.0.trial178.stdout:3542: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:30:38.284 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: finishing write tid 1 to trial17818531-25 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: finishing write tid 2 to trial17818531-25 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3542: done (2 left) 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3544: done (1 left) 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3545: delete oid 8 current snap is 354 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: finishing write tid 3 to trial17818531-25 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: finishing write tid 4 to trial17818531-25 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: finishing write tid 5 to trial17818531-25 2026-02-21T22:30:38.391 INFO:tasks.rados.rados.0.trial178.stdout:3543: oid 25 updating version 0 to 1497 2026-02-21T22:30:38.392 INFO:tasks.rados.rados.0.trial178.stdout:3543: oid 25 version 1497 is already newer than 1496 2026-02-21T22:30:38.392 INFO:tasks.rados.rados.0.trial178.stdout:3543: oid 25 version 1497 is already newer than 1494 2026-02-21T22:30:38.392 INFO:tasks.rados.rados.0.trial178.stdout:3543: oid 25 version 1497 is already newer than 1495 2026-02-21T22:30:38.392 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1497 (ObjNum 1241 snap 354 seq_num 1241) dirty exists 2026-02-21T22:30:38.392 INFO:tasks.rados.rados.0.trial178.stdout:3543: left oid 25 (ObjNum 1241 snap 354 seq_num 1241) 2026-02-21T22:30:38.393 INFO:tasks.rados.rados.0.trial178.stdout:3543: done (1 left) 2026-02-21T22:30:38.393 INFO:tasks.rados.rados.0.trial178.stdout:3545: done (0 left) 2026-02-21T22:30:38.393 INFO:tasks.rados.rados.0.trial178.stdout:3546: read oid 22 snap -1 2026-02-21T22:30:38.393 INFO:tasks.rados.rados.0.trial178.stdout:3546: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:30:38.393 INFO:tasks.rados.rados.0.trial178.stdout:3547: read oid 9 snap -1 2026-02-21T22:30:38.394 INFO:tasks.rados.rados.0.trial178.stdout:3547: expect deleted 2026-02-21T22:30:38.394 INFO:tasks.rados.rados.0.trial178.stdout:3548: snap_create 2026-02-21T22:30:38.401 INFO:tasks.rados.rados.0.trial178.stdout:3546: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:30:38.444 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1490527 2026-02-21T22:30:38.589 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:38.589 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:39.240 INFO:tasks.rados.rados.0.trial178.stdout:3546: done (2 left) 2026-02-21T22:30:39.240 INFO:tasks.rados.rados.0.trial178.stdout:3547: done (1 left) 2026-02-21T22:30:39.240 INFO:tasks.rados.rados.0.trial178.stdout:3548: done (0 left) 2026-02-21T22:30:39.240 INFO:tasks.rados.rados.0.trial178.stdout:3549: snap_create 2026-02-21T22:30:40.245 INFO:tasks.rados.rados.0.trial178.stdout:3549: done (0 left) 2026-02-21T22:30:40.246 INFO:tasks.rados.rados.0.trial178.stdout:3550: snap_remove snap 348 2026-02-21T22:30:41.246 INFO:tasks.rados.rados.0.trial178.stdout:3550: done (0 left) 2026-02-21T22:30:41.246 INFO:tasks.rados.rados.0.trial178.stdout:3551: copy_from oid 42 from oid 44 current snap is 356 2026-02-21T22:30:41.246 INFO:tasks.rados.rados.0.trial178.stdout:3552: snap_remove snap 354 2026-02-21T22:30:42.209 INFO:tasks.rados.rados.0.trial178.stdout:3552: done (1 left) 2026-02-21T22:30:42.209 INFO:tasks.rados.rados.0.trial178.stdout:3553: read oid 1 snap -1 2026-02-21T22:30:42.210 INFO:tasks.rados.rados.0.trial178.stdout:3553: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:42.210 INFO:tasks.rados.rados.0.trial178.stdout:3554: snap_remove snap 355 2026-02-21T22:30:43.212 INFO:tasks.rados.rados.0.trial178.stdout:3554: done (2 left) 2026-02-21T22:30:43.212 INFO:tasks.rados.rados.0.trial178.stdout:3555: copy_from oid 31 from oid 50 current snap is 356 2026-02-21T22:30:43.212 INFO:tasks.rados.rados.0.trial178.stdout:3551: finishing copy_from racing read to trial17818531-42 2026-02-21T22:30:43.213 INFO:tasks.rados.rados.0.trial178.stdout:3551: finishing copy_from to trial17818531-42 2026-02-21T22:30:43.213 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1619 (ObjNum 1158 snap 331 seq_num 1158) dirty exists 2026-02-21T22:30:43.213 INFO:tasks.rados.rados.0.trial178.stdout:3551: done (2 left) 2026-02-21T22:30:43.213 INFO:tasks.rados.rados.0.trial178.stdout:3556: write oid 13 current snap is 356 2026-02-21T22:30:43.213 INFO:tasks.rados.rados.0.trial178.stdout:3556: seq_num 1244 ranges {507908=575295,1829482=631762} 2026-02-21T22:30:43.219 INFO:tasks.rados.rados.0.trial178.stdout:3556: writing trial17818531-13 from 507908 to 1083203 tid 1 2026-02-21T22:30:43.228 INFO:tasks.rados.rados.0.trial178.stdout:3556: writing trial17818531-13 from 1829482 to 2461244 tid 2 2026-02-21T22:30:43.228 INFO:tasks.rados.rados.0.trial178.stdout:3557: write oid 23 current snap is 356 2026-02-21T22:30:43.229 INFO:tasks.rados.rados.0.trial178.stdout:3557: seq_num 1245 ranges {524715=650544,1887964=491773} 2026-02-21T22:30:43.236 INFO:tasks.rados.rados.0.trial178.stdout:3557: writing trial17818531-23 from 524715 to 1175259 tid 1 2026-02-21T22:30:43.245 INFO:tasks.rados.rados.0.trial178.stdout:3557: writing trial17818531-23 from 1887964 to 2379737 tid 2 2026-02-21T22:30:43.245 INFO:tasks.rados.rados.0.trial178.stdout:3558: copy_from oid 3 from oid 6 current snap is 356 2026-02-21T22:30:43.245 INFO:tasks.rados.rados.0.trial178.stdout:3553: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:3555: finishing copy_from racing read to trial17818531-31 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:3553: done (4 left) 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:3559: snap_create 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:3555: finishing copy_from to trial17818531-31 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1642 (ObjNum 1228 snap 348 seq_num 1228) dirty exists 2026-02-21T22:30:43.353 INFO:tasks.rados.rados.0.trial178.stdout:3556: finishing write tid 1 to trial17818531-13 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: finishing write tid 2 to trial17818531-13 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: finishing write tid 3 to trial17818531-13 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: finishing write tid 4 to trial17818531-13 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: oid 13 updating version 0 to 1739 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: oid 13 version 1739 is already newer than 1737 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: oid 13 version 1739 is already newer than 1738 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 13 v 1739 (ObjNum 1244 snap 356 seq_num 1244) dirty exists 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3556: left oid 13 (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3557: finishing write tid 1 to trial17818531-23 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3557: finishing write tid 2 to trial17818531-23 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3557: finishing write tid 3 to trial17818531-23 2026-02-21T22:30:43.354 INFO:tasks.rados.rados.0.trial178.stdout:3557: finishing write tid 4 to trial17818531-23 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3557: oid 23 updating version 0 to 1415 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3557: oid 23 updating version 1415 to 1416 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3557: oid 23 version 1416 is already newer than 1414 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1416 (ObjNum 1245 snap 356 seq_num 1245) dirty exists 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3557: left oid 23 (ObjNum 1245 snap 356 seq_num 1245) 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3558: finishing copy_from racing read to trial17818531-3 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:3558: finishing copy_from to trial17818531-3 2026-02-21T22:30:43.355 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1258 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:30:43.589 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:43.589 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3555: done (4 left) 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3556: done (3 left) 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3557: done (2 left) 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3558: done (1 left) 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3559: done (0 left) 2026-02-21T22:30:44.214 INFO:tasks.rados.rados.0.trial178.stdout:3560: snap_remove snap 344 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:3560: done (0 left) 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:3561: rollback oid 23 current snap is 357 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 356 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:3562: copy_from oid 14 from oid 10 current snap is 357 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:3563: write (excl) oid 47 current snap is 357 2026-02-21T22:30:45.214 INFO:tasks.rados.rados.0.trial178.stdout:3563: seq_num 1246 ranges {541522=725793,1946446=635431} 2026-02-21T22:30:45.223 INFO:tasks.rados.rados.0.trial178.stdout:3563: writing trial17818531-47 from 541522 to 1267315 tid 1 2026-02-21T22:30:45.232 INFO:tasks.rados.rados.0.trial178.stdout:3563: writing trial17818531-47 from 1946446 to 2581877 tid 2 2026-02-21T22:30:45.232 INFO:tasks.rados.rados.0.trial178.stdout:3564: snap_remove snap 352 2026-02-21T22:30:46.219 INFO:tasks.rados.rados.0.trial178.stdout:3564: done (3 left) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3565: read oid 13 snap -1 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3565: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3566: rollback oid 43 current snap is 357 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 43 to 353 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3562: finishing copy_from to trial17818531-14 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3562: got expected ENOENT (src dne) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3562: finishing copy_from racing read to trial17818531-14 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3562: done (4 left) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3567: read oid 10 snap 356 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3567: expect deleted 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3561: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3561: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3561: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1419 (ObjNum 1245 snap 356 seq_num 1245) dirty exists 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: finishing write tid 1 to trial17818531-47 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: finishing write tid 2 to trial17818531-47 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: finishing write tid 3 to trial17818531-47 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: finishing write tid 4 to trial17818531-47 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: oid 47 updating version 0 to 1915 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: oid 47 version 1915 is already newer than 1914 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: oid 47 updating version 1915 to 1916 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1916 (ObjNum 1246 snap 357 seq_num 1246) dirty exists 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: left oid 47 (ObjNum 1246 snap 357 seq_num 1246) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3561: done (4 left) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3563: done (3 left) 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3568: write (excl) oid 18 current snap is 357 2026-02-21T22:30:46.220 INFO:tasks.rados.rados.0.trial178.stdout:3568: seq_num 1247 ranges {558329=517395,1721281=779089} 2026-02-21T22:30:46.227 INFO:tasks.rados.rados.0.trial178.stdout:3568: writing trial17818531-18 from 558329 to 1075724 tid 1 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:3568: writing trial17818531-18 from 1721281 to 2500370 tid 2 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:3569: rollback oid 21 current snap is 357 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 353 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:3566: finishing rollback tid 0 to trial17818531-43 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:3566: finishing rollback tid 1 to trial17818531-43 2026-02-21T22:30:46.237 INFO:tasks.rados.rados.0.trial178.stdout:3566: finishing rollback tid 2 to trial17818531-43 2026-02-21T22:30:46.238 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1605 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:30:46.238 INFO:tasks.rados.rados.0.trial178.stdout:3565: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:30:46.277 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2644379 2026-02-21T22:30:46.321 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:46.359 INFO:tasks.rados.rados.0.trial178.stdout:3568: finishing write tid 1 to trial17818531-18 2026-02-21T22:30:46.359 INFO:tasks.rados.rados.0.trial178.stdout:3568: finishing write tid 2 to trial17818531-18 2026-02-21T22:30:46.359 INFO:tasks.rados.rados.0.trial178.stdout:3568: finishing write tid 3 to trial17818531-18 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3569: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3568: finishing write tid 4 to trial17818531-18 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3568: oid 18 updating version 0 to 1626 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3568: oid 18 updating version 1626 to 1627 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3568: oid 18 updating version 1627 to 1628 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1628 (ObjNum 1247 snap 357 seq_num 1247) dirty exists 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3568: left oid 18 (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3569: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3569: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1922 (ObjNum 1175 snap 336 seq_num 1175) dirty exists 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3565: done (4 left) 2026-02-21T22:30:46.360 INFO:tasks.rados.rados.0.trial178.stdout:3566: done (3 left) 2026-02-21T22:30:46.361 INFO:tasks.rados.rados.0.trial178.stdout:3567: done (2 left) 2026-02-21T22:30:46.361 INFO:tasks.rados.rados.0.trial178.stdout:3568: done (1 left) 2026-02-21T22:30:46.361 INFO:tasks.rados.rados.0.trial178.stdout:3569: done (0 left) 2026-02-21T22:30:46.361 INFO:tasks.rados.rados.0.trial178.stdout:3570: snap_create 2026-02-21T22:30:47.209 INFO:tasks.rados.rados.0.trial178.stdout:3570: done (0 left) 2026-02-21T22:30:47.209 INFO:tasks.rados.rados.0.trial178.stdout:3571: snap_remove snap 356 2026-02-21T22:30:48.223 INFO:tasks.rados.rados.0.trial178.stdout:3571: done (0 left) 2026-02-21T22:30:48.223 INFO:tasks.rados.rados.0.trial178.stdout:3572: write oid 32 current snap is 358 2026-02-21T22:30:48.223 INFO:tasks.rados.rados.0.trial178.stdout:3572: seq_num 1248 ranges {575136=592644,1663410=639100,2705542=269594} 2026-02-21T22:30:48.231 INFO:tasks.rados.rados.0.trial178.stdout:3572: writing trial17818531-32 from 575136 to 1167780 tid 1 2026-02-21T22:30:48.239 INFO:tasks.rados.rados.0.trial178.stdout:3572: writing trial17818531-32 from 1663410 to 2302510 tid 2 2026-02-21T22:30:48.244 INFO:tasks.rados.rados.0.trial178.stdout:3572: writing trial17818531-32 from 2705542 to 2975136 tid 3 2026-02-21T22:30:48.244 INFO:tasks.rados.rados.0.trial178.stdout:3573: write (excl) oid 15 current snap is 358 2026-02-21T22:30:48.244 INFO:tasks.rados.rados.0.trial178.stdout:3573: seq_num 1249 ranges {591943=667893,1721892=782758} 2026-02-21T22:30:48.253 INFO:tasks.rados.rados.0.trial178.stdout:3573: writing trial17818531-15 from 591943 to 1259836 tid 1 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3573: writing trial17818531-15 from 1721892 to 2504650 tid 2 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3574: write oid 48 current snap is 358 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3572: finishing write tid 1 to trial17818531-32 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3572: finishing write tid 2 to trial17818531-32 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3572: finishing write tid 3 to trial17818531-32 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3572: finishing write tid 4 to trial17818531-32 2026-02-21T22:30:48.261 INFO:tasks.rados.rados.0.trial178.stdout:3572: finishing write tid 5 to trial17818531-32 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3572: oid 32 updating version 0 to 1761 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3572: oid 32 version 1761 is already newer than 1760 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3572: oid 32 updating version 1761 to 1762 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3572: oid 32 version 1762 is already newer than 1759 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1762 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3572: left oid 32 (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3573: finishing write tid 1 to trial17818531-15 2026-02-21T22:30:48.262 INFO:tasks.rados.rados.0.trial178.stdout:3574: seq_num 1250 ranges {608750=743142,1780374=642769} 2026-02-21T22:30:48.270 INFO:tasks.rados.rados.0.trial178.stdout:3574: writing trial17818531-48 from 608750 to 1351892 tid 1 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3574: writing trial17818531-48 from 1780374 to 2423143 tid 2 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3572: done (2 left) 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3575: read oid 6 snap -1 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3575: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3576: read oid 24 snap -1 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3576: expect (ObjNum 1209 snap 347 seq_num 1209) 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3577: copy_from oid 9 from oid 27 current snap is 358 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: finishing write tid 2 to trial17818531-15 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: finishing write tid 3 to trial17818531-15 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: finishing write tid 4 to trial17818531-15 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: oid 15 updating version 0 to 1628 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: oid 15 version 1628 is already newer than 1626 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: oid 15 version 1628 is already newer than 1627 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1628 (ObjNum 1249 snap 358 seq_num 1249) dirty exists 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: left oid 15 (ObjNum 1249 snap 358 seq_num 1249) 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3574: finishing write tid 1 to trial17818531-48 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3573: done (4 left) 2026-02-21T22:30:48.278 INFO:tasks.rados.rados.0.trial178.stdout:3578: snap_remove snap 353 2026-02-21T22:30:48.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:48.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:49.227 INFO:tasks.rados.rados.0.trial178.stdout:3578: done (4 left) 2026-02-21T22:30:49.227 INFO:tasks.rados.rados.0.trial178.stdout:3579: copy_from oid 36 from oid 44 current snap is 358 2026-02-21T22:30:49.227 INFO:tasks.rados.rados.0.trial178.stdout:3575: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:30:49.316 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2039254 2026-02-21T22:30:49.324 INFO:tasks.rados.rados.0.trial178.stdout:3574: finishing write tid 2 to trial17818531-48 2026-02-21T22:30:49.324 INFO:tasks.rados.rados.0.trial178.stdout:3574: finishing write tid 3 to trial17818531-48 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3575: done (4 left) 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3580: copy_from oid 19 from oid 22 current snap is 358 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3574: finishing write tid 4 to trial17818531-48 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3574: oid 48 updating version 0 to 1631 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3574: oid 48 updating version 1631 to 1633 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3574: oid 48 version 1633 is already newer than 1632 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1633 (ObjNum 1250 snap 358 seq_num 1250) dirty exists 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3574: left oid 48 (ObjNum 1250 snap 358 seq_num 1250) 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:3576: expect (ObjNum 1209 snap 347 seq_num 1209) 2026-02-21T22:30:49.325 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 9146 2026-02-21T22:30:49.374 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:49.399 INFO:tasks.rados.rados.0.trial178.stdout:3577: finishing copy_from racing read to trial17818531-9 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3577: finishing copy_from to trial17818531-9 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1489 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3579: finishing copy_from racing read to trial17818531-36 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3579: finishing copy_from to trial17818531-36 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1619 (ObjNum 1158 snap 331 seq_num 1158) dirty exists 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3580: finishing copy_from racing read to trial17818531-19 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:3580: finishing copy_from to trial17818531-19 2026-02-21T22:30:49.400 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1489 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3574: done (4 left) 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3576: done (3 left) 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3577: done (2 left) 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3579: done (1 left) 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3580: done (0 left) 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3581: write oid 24 current snap is 358 2026-02-21T22:30:49.401 INFO:tasks.rados.rados.0.trial178.stdout:3581: seq_num 1251 ranges {625557=418391,1838856=786427} 2026-02-21T22:30:49.406 INFO:tasks.rados.rados.0.trial178.stdout:3581: writing trial17818531-24 from 625557 to 1043948 tid 1 2026-02-21T22:30:49.417 INFO:tasks.rados.rados.0.trial178.stdout:3581: writing trial17818531-24 from 1838856 to 2625283 tid 2 2026-02-21T22:30:49.417 INFO:tasks.rados.rados.0.trial178.stdout:3582: write oid 48 current snap is 358 2026-02-21T22:30:49.417 INFO:tasks.rados.rados.0.trial178.stdout:3582: seq_num 1252 ranges {642364=493640,1897338=646438} 2026-02-21T22:30:49.424 INFO:tasks.rados.rados.0.trial178.stdout:3582: writing trial17818531-48 from 642364 to 1136004 tid 1 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:3582: writing trial17818531-48 from 1897338 to 2543776 tid 2 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:3583: rollback oid 30 current snap is 358 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 30 to 357 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:3581: finishing write tid 1 to trial17818531-24 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:3581: finishing write tid 2 to trial17818531-24 2026-02-21T22:30:49.434 INFO:tasks.rados.rados.0.trial178.stdout:3581: finishing write tid 3 to trial17818531-24 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: finishing write tid 4 to trial17818531-24 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: oid 24 updating version 0 to 1935 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: oid 24 version 1935 is already newer than 1934 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: oid 24 updating version 1935 to 1936 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1936 (ObjNum 1251 snap 358 seq_num 1251) dirty exists 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: left oid 24 (ObjNum 1251 snap 358 seq_num 1251) 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3581: done (2 left) 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3584: rollback oid 3 current snap is 358 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 357 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3582: finishing write tid 1 to trial17818531-48 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3585: rollback oid 17 current snap is 358 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 17 to 357 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3586: write (excl) oid 31 current snap is 358 2026-02-21T22:30:49.435 INFO:tasks.rados.rados.0.trial178.stdout:3586: seq_num 1253 ranges {659171=568889,1839467=790096} 2026-02-21T22:30:49.442 INFO:tasks.rados.rados.0.trial178.stdout:3586: writing trial17818531-31 from 659171 to 1228060 tid 1 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3586: writing trial17818531-31 from 1839467 to 2629563 tid 2 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3587: snap_create 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3584: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3582: finishing write tid 2 to trial17818531-48 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3582: finishing write tid 3 to trial17818531-48 2026-02-21T22:30:49.448 INFO:tasks.rados.rados.0.trial178.stdout:3582: finishing write tid 4 to trial17818531-48 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3582: oid 48 updating version 0 to 1634 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3582: oid 48 updating version 1634 to 1635 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3582: oid 48 updating version 1635 to 1636 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1636 (ObjNum 1252 snap 358 seq_num 1252) dirty exists 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3582: left oid 48 (ObjNum 1252 snap 358 seq_num 1252) 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3584: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3585: finishing rollback tid 0 to trial17818531-17 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3584: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1772 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3585: finishing rollback tid 1 to trial17818531-17 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3585: finishing rollback tid 2 to trial17818531-17 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1631 (ObjNum 1166 snap 333 seq_num 1166) dirty exists 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3583: finishing rollback tid 0 to trial17818531-30 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3583: finishing rollback tid 1 to trial17818531-30 2026-02-21T22:30:49.449 INFO:tasks.rados.rados.0.trial178.stdout:3583: finishing rollback tid 2 to trial17818531-30 2026-02-21T22:30:49.450 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1631 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:30:49.450 INFO:tasks.rados.rados.0.trial178.stdout:3586: finishing write tid 1 to trial17818531-31 2026-02-21T22:30:49.451 INFO:tasks.rados.rados.0.trial178.stdout:3586: finishing write tid 2 to trial17818531-31 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: finishing write tid 3 to trial17818531-31 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: finishing write tid 4 to trial17818531-31 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: oid 31 updating version 0 to 1645 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: oid 31 version 1645 is already newer than 1644 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: oid 31 version 1645 is already newer than 1643 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1645 (ObjNum 1253 snap 358 seq_num 1253) dirty exists 2026-02-21T22:30:49.452 INFO:tasks.rados.rados.0.trial178.stdout:3586: left oid 31 (ObjNum 1253 snap 358 seq_num 1253) 2026-02-21T22:30:50.232 INFO:tasks.rados.rados.0.trial178.stdout:3582: done (5 left) 2026-02-21T22:30:50.232 INFO:tasks.rados.rados.0.trial178.stdout:3583: done (4 left) 2026-02-21T22:30:50.232 INFO:tasks.rados.rados.0.trial178.stdout:3584: done (3 left) 2026-02-21T22:30:50.232 INFO:tasks.rados.rados.0.trial178.stdout:3585: done (2 left) 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3586: done (1 left) 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3587: done (0 left) 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3588: read oid 13 snap -1 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3588: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3589: copy_from oid 14 from oid 34 current snap is 359 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3590: copy_from oid 17 from oid 15 current snap is 359 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3591: write (excl) oid 36 current snap is 359 2026-02-21T22:30:50.233 INFO:tasks.rados.rados.0.trial178.stdout:3591: seq_num 1254 ranges {675978=644138,1897949=650107} 2026-02-21T22:30:50.237 INFO:tasks.rados.rados.0.trial178.stdout:3591: writing trial17818531-36 from 675978 to 1320116 tid 1 2026-02-21T22:30:50.242 INFO:tasks.rados.rados.0.trial178.stdout:3591: writing trial17818531-36 from 1897949 to 2548056 tid 2 2026-02-21T22:30:50.242 INFO:tasks.rados.rados.0.trial178.stdout:3592: read oid 22 snap -1 2026-02-21T22:30:50.242 INFO:tasks.rados.rados.0.trial178.stdout:3592: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:30:50.242 INFO:tasks.rados.rados.0.trial178.stdout:3593: snap_remove snap 357 2026-02-21T22:30:51.234 INFO:tasks.rados.rados.0.trial178.stdout:3593: done (5 left) 2026-02-21T22:30:51.234 INFO:tasks.rados.rados.0.trial178.stdout:3594: delete oid 16 current snap is 359 2026-02-21T22:30:51.235 INFO:tasks.rados.rados.0.trial178.stdout:3589: finishing copy_from to trial17818531-14 2026-02-21T22:30:51.235 INFO:tasks.rados.rados.0.trial178.stdout:3589: got expected ENOENT (src dne) 2026-02-21T22:30:51.235 INFO:tasks.rados.rados.0.trial178.stdout:3589: finishing copy_from racing read to trial17818531-14 2026-02-21T22:30:51.235 INFO:tasks.rados.rados.0.trial178.stdout:3588: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:30:51.235 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:51.326 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 732322 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3591: finishing write tid 1 to trial17818531-36 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3591: finishing write tid 2 to trial17818531-36 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3588: done (5 left) 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3589: done (4 left) 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3594: done (3 left) 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3595: copy_from oid 9 from oid 30 current snap is 359 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3591: finishing write tid 3 to trial17818531-36 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3591: finishing write tid 4 to trial17818531-36 2026-02-21T22:30:51.357 INFO:tasks.rados.rados.0.trial178.stdout:3591: oid 36 updating version 0 to 1620 2026-02-21T22:30:51.358 INFO:tasks.rados.rados.0.trial178.stdout:3591: oid 36 updating version 1620 to 1622 2026-02-21T22:30:51.358 INFO:tasks.rados.rados.0.trial178.stdout:3591: oid 36 version 1622 is already newer than 1621 2026-02-21T22:30:51.358 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1622 (ObjNum 1254 snap 359 seq_num 1254) dirty exists 2026-02-21T22:30:51.358 INFO:tasks.rados.rados.0.trial178.stdout:3591: left oid 36 (ObjNum 1254 snap 359 seq_num 1254) 2026-02-21T22:30:51.358 INFO:tasks.rados.rados.0.trial178.stdout:3592: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:30:51.377 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:51.397 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3590: finishing copy_from racing read to trial17818531-17 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3590: finishing copy_from to trial17818531-17 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1628 (ObjNum 1249 snap 358 seq_num 1249) dirty exists 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3595: finishing copy_from racing read to trial17818531-9 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3595: finishing copy_from to trial17818531-9 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1631 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3590: done (3 left) 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3591: done (2 left) 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3592: done (1 left) 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3595: done (0 left) 2026-02-21T22:30:51.418 INFO:tasks.rados.rados.0.trial178.stdout:3596: rollback oid 22 current snap is 359 2026-02-21T22:30:51.419 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 358 2026-02-21T22:30:51.419 INFO:tasks.rados.rados.0.trial178.stdout:3597: rollback oid 19 current snap is 359 2026-02-21T22:30:51.419 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 358 2026-02-21T22:30:51.419 INFO:tasks.rados.rados.0.trial178.stdout:3598: delete oid 20 current snap is 359 2026-02-21T22:30:51.421 INFO:tasks.rados.rados.0.trial178.stdout:3596: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:30:51.421 INFO:tasks.rados.rados.0.trial178.stdout:3596: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:30:51.421 INFO:tasks.rados.rados.0.trial178.stdout:3596: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:30:51.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1650 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:51.422 INFO:tasks.rados.rados.0.trial178.stdout:3597: finishing rollback tid 0 to trial17818531-19 2026-02-21T22:30:51.422 INFO:tasks.rados.rados.0.trial178.stdout:3597: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:30:51.422 INFO:tasks.rados.rados.0.trial178.stdout:3597: finishing rollback tid 2 to trial17818531-19 2026-02-21T22:30:51.422 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1944 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:30:51.423 INFO:tasks.rados.rados.0.trial178.stdout:3596: done (2 left) 2026-02-21T22:30:51.423 INFO:tasks.rados.rados.0.trial178.stdout:3597: done (1 left) 2026-02-21T22:30:51.423 INFO:tasks.rados.rados.0.trial178.stdout:3598: done (0 left) 2026-02-21T22:30:51.423 INFO:tasks.rados.rados.0.trial178.stdout:3599: snap_remove snap 358 2026-02-21T22:30:52.211 INFO:tasks.rados.rados.0.trial178.stdout:3599: done (0 left) 2026-02-21T22:30:52.211 INFO:tasks.rados.rados.0.trial178.stdout:3600: delete oid 27 current snap is 359 2026-02-21T22:30:52.217 INFO:tasks.rados.rados.0.trial178.stdout:3600: done (0 left) 2026-02-21T22:30:52.217 INFO:tasks.rados.rados.0.trial178.stdout:3601: rollback oid 25 current snap is 359 2026-02-21T22:30:52.217 INFO:tasks.rados.rados.0.trial178.stdout:3601: done (0 left) 2026-02-21T22:30:52.217 INFO:tasks.rados.rados.0.trial178.stdout:3602: write oid 7 current snap is 359 2026-02-21T22:30:52.217 INFO:tasks.rados.rados.0.trial178.stdout:3602: seq_num 1258 ranges {743206=661487,2131877=541092} 2026-02-21T22:30:52.227 INFO:tasks.rados.rados.0.trial178.stdout:3602: writing trial17818531-7 from 743206 to 1404693 tid 1 2026-02-21T22:30:52.235 INFO:tasks.rados.rados.0.trial178.stdout:3602: writing trial17818531-7 from 2131877 to 2672969 tid 2 2026-02-21T22:30:52.235 INFO:tasks.rados.rados.0.trial178.stdout:3603: copy_from oid 39 from oid 30 current snap is 359 2026-02-21T22:30:52.235 INFO:tasks.rados.rados.0.trial178.stdout:3602: finishing write tid 1 to trial17818531-7 2026-02-21T22:30:52.235 INFO:tasks.rados.rados.0.trial178.stdout:3604: write (excl) oid 42 current snap is 359 2026-02-21T22:30:52.235 INFO:tasks.rados.rados.0.trial178.stdout:3604: seq_num 1259 ranges {760013=736736,2190359=401103} 2026-02-21T22:30:52.246 INFO:tasks.rados.rados.0.trial178.stdout:3604: writing trial17818531-42 from 760013 to 1496749 tid 1 2026-02-21T22:30:52.253 INFO:tasks.rados.rados.0.trial178.stdout:3604: writing trial17818531-42 from 2190359 to 2591462 tid 2 2026-02-21T22:30:52.253 INFO:tasks.rados.rados.0.trial178.stdout:3605: write (excl) oid 44 current snap is 359 2026-02-21T22:30:52.253 INFO:tasks.rados.rados.0.trial178.stdout:3605: seq_num 1260 ranges {776820=411985,1848841=544761,2918265=258555} 2026-02-21T22:30:52.261 INFO:tasks.rados.rados.0.trial178.stdout:3605: writing trial17818531-44 from 776820 to 1188805 tid 1 2026-02-21T22:30:52.268 INFO:tasks.rados.rados.0.trial178.stdout:3605: writing trial17818531-44 from 1848841 to 2393602 tid 2 2026-02-21T22:30:52.271 INFO:tasks.rados.rados.0.trial178.stdout:3605: writing trial17818531-44 from 2918265 to 3176820 tid 3 2026-02-21T22:30:52.271 INFO:tasks.rados.rados.0.trial178.stdout:3606: read oid 27 snap -1 2026-02-21T22:30:52.272 INFO:tasks.rados.rados.0.trial178.stdout:3606: expect deleted 2026-02-21T22:30:52.272 INFO:tasks.rados.rados.0.trial178.stdout:3607: write (excl) oid 29 current snap is 359 2026-02-21T22:30:52.272 INFO:tasks.rados.rados.0.trial178.stdout:3607: seq_num 1261 ranges {793627=487234,1790970=404772,2829335=364292} 2026-02-21T22:30:52.276 INFO:tasks.rados.rados.0.trial178.stdout:3607: writing trial17818531-29 from 793627 to 1280861 tid 1 2026-02-21T22:30:52.279 INFO:tasks.rados.rados.0.trial178.stdout:3607: writing trial17818531-29 from 1790970 to 2195742 tid 2 2026-02-21T22:30:52.283 INFO:tasks.rados.rados.0.trial178.stdout:3607: writing trial17818531-29 from 2829335 to 3193627 tid 3 2026-02-21T22:30:52.283 INFO:tasks.rados.rados.0.trial178.stdout:3608: write (excl) oid 35 current snap is 359 2026-02-21T22:30:52.283 INFO:tasks.rados.rados.0.trial178.stdout:3608: seq_num 1262 ranges {410434=678836,1565805=548430,2573111=637323} 2026-02-21T22:30:52.287 INFO:tasks.rados.rados.0.trial178.stdout:3608: writing trial17818531-35 from 410434 to 1089270 tid 1 2026-02-21T22:30:52.291 INFO:tasks.rados.rados.0.trial178.stdout:3608: writing trial17818531-35 from 1565805 to 2114235 tid 2 2026-02-21T22:30:52.296 INFO:tasks.rados.rados.0.trial178.stdout:3608: writing trial17818531-35 from 2573111 to 3210434 tid 3 2026-02-21T22:30:52.296 INFO:tasks.rados.rados.0.trial178.stdout:3609: delete oid 9 current snap is 359 2026-02-21T22:30:52.296 INFO:tasks.rados.rados.0.trial178.stdout:3602: finishing write tid 2 to trial17818531-7 2026-02-21T22:30:52.296 INFO:tasks.rados.rados.0.trial178.stdout:3602: finishing write tid 3 to trial17818531-7 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3602: finishing write tid 4 to trial17818531-7 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3602: oid 7 updating version 0 to 1949 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3602: oid 7 version 1949 is already newer than 1948 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3602: oid 7 version 1949 is already newer than 1947 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1949 (ObjNum 1258 snap 359 seq_num 1258) dirty exists 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3602: left oid 7 (ObjNum 1258 snap 359 seq_num 1258) 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3603: finishing copy_from racing read to trial17818531-39 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3604: finishing write tid 1 to trial17818531-42 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3603: finishing copy_from to trial17818531-39 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1631 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:30:52.297 INFO:tasks.rados.rados.0.trial178.stdout:3604: finishing write tid 2 to trial17818531-42 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: finishing write tid 3 to trial17818531-42 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: finishing write tid 4 to trial17818531-42 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: oid 42 updating version 0 to 1951 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: oid 42 version 1951 is already newer than 1950 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: oid 42 updating version 1951 to 1952 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 1952 (ObjNum 1259 snap 359 seq_num 1259) dirty exists 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3604: left oid 42 (ObjNum 1259 snap 359 seq_num 1259) 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: finishing write tid 1 to trial17818531-44 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: finishing write tid 2 to trial17818531-44 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: finishing write tid 3 to trial17818531-44 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: finishing write tid 4 to trial17818531-44 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: finishing write tid 5 to trial17818531-44 2026-02-21T22:30:52.298 INFO:tasks.rados.rados.0.trial178.stdout:3605: oid 44 updating version 0 to 1844 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3605: oid 44 version 1844 is already newer than 1843 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3605: oid 44 updating version 1844 to 1845 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3605: oid 44 version 1845 is already newer than 1842 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1845 (ObjNum 1260 snap 359 seq_num 1260) dirty exists 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3605: left oid 44 (ObjNum 1260 snap 359 seq_num 1260) 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: finishing write tid 1 to trial17818531-29 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: finishing write tid 2 to trial17818531-29 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: finishing write tid 3 to trial17818531-29 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: finishing write tid 4 to trial17818531-29 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: finishing write tid 5 to trial17818531-29 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: oid 29 updating version 0 to 1631 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: oid 29 updating version 1631 to 1632 2026-02-21T22:30:52.299 INFO:tasks.rados.rados.0.trial178.stdout:3607: oid 29 updating version 1632 to 1633 2026-02-21T22:30:52.300 INFO:tasks.rados.rados.0.trial178.stdout:3607: oid 29 version 1633 is already newer than 1630 2026-02-21T22:30:52.300 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1633 (ObjNum 1261 snap 359 seq_num 1261) dirty exists 2026-02-21T22:30:52.300 INFO:tasks.rados.rados.0.trial178.stdout:3607: left oid 29 (ObjNum 1261 snap 359 seq_num 1261) 2026-02-21T22:30:52.300 INFO:tasks.rados.rados.0.trial178.stdout:3608: finishing write tid 1 to trial17818531-35 2026-02-21T22:30:52.300 INFO:tasks.rados.rados.0.trial178.stdout:3608: finishing write tid 2 to trial17818531-35 2026-02-21T22:30:52.301 INFO:tasks.rados.rados.0.trial178.stdout:3608: finishing write tid 3 to trial17818531-35 2026-02-21T22:30:52.301 INFO:tasks.rados.rados.0.trial178.stdout:3602: done (7 left) 2026-02-21T22:30:52.301 INFO:tasks.rados.rados.0.trial178.stdout:3603: done (6 left) 2026-02-21T22:30:52.301 INFO:tasks.rados.rados.0.trial178.stdout:3604: done (5 left) 2026-02-21T22:30:52.301 INFO:tasks.rados.rados.0.trial178.stdout:3605: done (4 left) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3606: done (3 left) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3607: done (2 left) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3609: done (1 left) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3610: read oid 29 snap -1 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3610: expect (ObjNum 1261 snap 359 seq_num 1261) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3611: read oid 28 snap -1 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3611: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3612: snap_create 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3608: finishing write tid 4 to trial17818531-35 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3608: finishing write tid 5 to trial17818531-35 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3608: oid 35 updating version 0 to 1777 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3608: oid 35 updating version 1777 to 1779 2026-02-21T22:30:52.302 INFO:tasks.rados.rados.0.trial178.stdout:3608: oid 35 updating version 1779 to 1780 2026-02-21T22:30:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3608: oid 35 version 1780 is already newer than 1778 2026-02-21T22:30:52.303 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1780 (ObjNum 1262 snap 359 seq_num 1262) dirty exists 2026-02-21T22:30:52.303 INFO:tasks.rados.rados.0.trial178.stdout:3608: left oid 35 (ObjNum 1262 snap 359 seq_num 1262) 2026-02-21T22:30:52.304 INFO:tasks.rados.rados.0.trial178.stdout:3611: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:52.304 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:52.354 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:52.412 INFO:tasks.rados.rados.0.trial178.stdout:3610: expect (ObjNum 1261 snap 359 seq_num 1261) 2026-02-21T22:30:53.215 INFO:tasks.rados.rados.0.trial178.stdout:3608: done (3 left) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3610: done (2 left) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3611: done (1 left) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3612: done (0 left) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3613: copy_from oid 17 from oid 13 current snap is 360 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3614: read oid 1 snap -1 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3614: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3615: read oid 28 snap 359 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3615: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3616: read oid 24 snap -1 2026-02-21T22:30:53.216 INFO:tasks.rados.rados.0.trial178.stdout:3616: expect (ObjNum 1251 snap 358 seq_num 1251) 2026-02-21T22:30:53.217 INFO:tasks.rados.rados.0.trial178.stdout:3617: snap_create 2026-02-21T22:30:53.218 INFO:tasks.rados.rados.0.trial178.stdout:3614: expect (ObjNum 1145 snap 326 seq_num 1145) 2026-02-21T22:30:53.298 INFO:tasks.rados.rados.0.trial178.stdout:3615: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:53.298 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:53.361 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1511241 2026-02-21T22:30:53.381 INFO:tasks.rados.rados.0.trial178.stdout:3616: expect (ObjNum 1251 snap 358 seq_num 1251) 2026-02-21T22:30:53.415 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 626509 2026-02-21T22:30:53.474 INFO:tasks.rados.rados.0.trial178.stdout:3613: finishing copy_from racing read to trial17818531-17 2026-02-21T22:30:53.475 INFO:tasks.rados.rados.0.trial178.stdout:3613: finishing copy_from to trial17818531-17 2026-02-21T22:30:53.475 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1739 (ObjNum 1244 snap 356 seq_num 1244) dirty exists 2026-02-21T22:30:53.541 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:53.541 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:53.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:53.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:30:53.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:53.554 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:30:53.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:53.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:54.217 INFO:tasks.rados.rados.0.trial178.stdout:3613: done (4 left) 2026-02-21T22:30:54.217 INFO:tasks.rados.rados.0.trial178.stdout:3614: done (3 left) 2026-02-21T22:30:54.217 INFO:tasks.rados.rados.0.trial178.stdout:3615: done (2 left) 2026-02-21T22:30:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3616: done (1 left) 2026-02-21T22:30:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3617: done (0 left) 2026-02-21T22:30:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3618: write oid 7 current snap is 361 2026-02-21T22:30:54.218 INFO:tasks.rados.rados.0.trial178.stdout:3618: seq_num 1264 ranges {444048=429334,1282769=552099,2627957=616091} 2026-02-21T22:30:54.221 INFO:tasks.rados.rados.0.trial178.stdout:3618: writing trial17818531-7 from 444048 to 873382 tid 1 2026-02-21T22:30:54.225 INFO:tasks.rados.rados.0.trial178.stdout:3618: writing trial17818531-7 from 1282769 to 1834868 tid 2 2026-02-21T22:30:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3618: writing trial17818531-7 from 2627957 to 3244048 tid 3 2026-02-21T22:30:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3619: snap_create 2026-02-21T22:30:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3618: finishing write tid 1 to trial17818531-7 2026-02-21T22:30:54.231 INFO:tasks.rados.rados.0.trial178.stdout:3618: finishing write tid 2 to trial17818531-7 2026-02-21T22:30:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3618: finishing write tid 3 to trial17818531-7 2026-02-21T22:30:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3618: finishing write tid 4 to trial17818531-7 2026-02-21T22:30:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3618: finishing write tid 5 to trial17818531-7 2026-02-21T22:30:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3618: oid 7 updating version 0 to 1956 2026-02-21T22:30:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3618: oid 7 version 1956 is already newer than 1954 2026-02-21T22:30:54.235 INFO:tasks.rados.rados.0.trial178.stdout:3618: oid 7 version 1956 is already newer than 1955 2026-02-21T22:30:54.235 INFO:tasks.rados.rados.0.trial178.stdout:3618: oid 7 updating version 1956 to 1957 2026-02-21T22:30:54.235 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1957 (ObjNum 1264 snap 361 seq_num 1264) dirty exists 2026-02-21T22:30:54.235 INFO:tasks.rados.rados.0.trial178.stdout:3618: left oid 7 (ObjNum 1264 snap 361 seq_num 1264) 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3618: done (1 left) 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3619: done (0 left) 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3620: rollback oid 49 current snap is 362 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 360 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3621: write oid 11 current snap is 362 2026-02-21T22:30:55.220 INFO:tasks.rados.rados.0.trial178.stdout:3621: seq_num 1265 ranges {460855=504583,1624898=412110,2539027=572508} 2026-02-21T22:30:55.224 INFO:tasks.rados.rados.0.trial178.stdout:3621: writing trial17818531-11 from 460855 to 965438 tid 1 2026-02-21T22:30:55.228 INFO:tasks.rados.rados.0.trial178.stdout:3621: writing trial17818531-11 from 1624898 to 2037008 tid 2 2026-02-21T22:30:55.232 INFO:tasks.rados.rados.0.trial178.stdout:3621: writing trial17818531-11 from 2539027 to 3111535 tid 3 2026-02-21T22:30:55.232 INFO:tasks.rados.rados.0.trial178.stdout:3622: snap_remove snap 359 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3622: done (2 left) 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3623: rollback oid 4 current snap is 362 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 4 to 361 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3620: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 0 (ObjNum 129 snap 0 seq_num 100608896) dirty dne 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3621: finishing write tid 1 to trial17818531-11 2026-02-21T22:30:56.225 INFO:tasks.rados.rados.0.trial178.stdout:3621: finishing write tid 2 to trial17818531-11 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: finishing write tid 3 to trial17818531-11 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3620: done (2 left) 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3624: read oid 28 snap -1 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3624: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3625: rollback oid 45 current snap is 362 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 360 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: finishing write tid 4 to trial17818531-11 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: finishing write tid 5 to trial17818531-11 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: oid 11 updating version 0 to 1784 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: oid 11 updating version 1784 to 1785 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: oid 11 version 1785 is already newer than 1783 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:3621: oid 11 version 1785 is already newer than 1782 2026-02-21T22:30:56.226 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1785 (ObjNum 1265 snap 362 seq_num 1265) dirty exists 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3621: left oid 11 (ObjNum 1265 snap 362 seq_num 1265) 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3621: done (3 left) 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3626: rollback oid 6 current snap is 362 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 6 to 360 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3627: read oid 29 snap -1 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3627: expect (ObjNum 1261 snap 359 seq_num 1261) 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3628: delete oid 46 current snap is 362 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:3623: finishing rollback tid 1 to trial17818531-4 2026-02-21T22:30:56.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 0 (ObjNum 129 snap 0 seq_num 101564712) dirty dne 2026-02-21T22:30:56.228 INFO:tasks.rados.rados.0.trial178.stdout:3625: finishing rollback tid 0 to trial17818531-45 2026-02-21T22:30:56.230 INFO:tasks.rados.rados.0.trial178.stdout:3623: done (5 left) 2026-02-21T22:30:56.230 INFO:tasks.rados.rados.0.trial178.stdout:3628: done (4 left) 2026-02-21T22:30:56.230 INFO:tasks.rados.rados.0.trial178.stdout:3629: write (excl) oid 4 current snap is 362 2026-02-21T22:30:56.230 INFO:tasks.rados.rados.0.trial178.stdout:3625: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:30:56.230 INFO:tasks.rados.rados.0.trial178.stdout:3629: seq_num 1267 ranges {494469=655081,1741862=415779,2710226=584243} 2026-02-21T22:30:56.237 INFO:tasks.rados.rados.0.trial178.stdout:3629: writing trial17818531-4 from 494469 to 1149550 tid 1 2026-02-21T22:30:56.244 INFO:tasks.rados.rados.0.trial178.stdout:3629: writing trial17818531-4 from 1741862 to 2157641 tid 2 2026-02-21T22:30:56.251 INFO:tasks.rados.rados.0.trial178.stdout:3629: writing trial17818531-4 from 2710226 to 3294469 tid 3 2026-02-21T22:30:56.251 INFO:tasks.rados.rados.0.trial178.stdout:3630: rollback oid 36 current snap is 362 2026-02-21T22:30:56.251 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 361 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:3625: finishing rollback tid 2 to trial17818531-45 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1639 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:3626: finishing rollback tid 0 to trial17818531-6 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:3626: finishing rollback tid 1 to trial17818531-6 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:3626: finishing rollback tid 2 to trial17818531-6 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1651 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:3624: expect (ObjNum 1238 snap 352 seq_num 1238) 2026-02-21T22:30:56.252 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 41311 2026-02-21T22:30:56.304 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1157681 2026-02-21T22:30:56.353 INFO:tasks.rados.rados.0.trial178.stdout:3627: expect (ObjNum 1261 snap 359 seq_num 1261) 2026-02-21T22:30:56.388 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: finishing write tid 1 to trial17818531-4 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: finishing write tid 2 to trial17818531-4 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: finishing write tid 3 to trial17818531-4 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: finishing write tid 4 to trial17818531-4 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: finishing write tid 5 to trial17818531-4 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: oid 4 updating version 0 to 1849 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: oid 4 version 1849 is already newer than 1847 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: oid 4 version 1849 is already newer than 1846 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: oid 4 version 1849 is already newer than 1848 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1849 (ObjNum 1267 snap 362 seq_num 1267) dirty exists 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3629: left oid 4 (ObjNum 1267 snap 362 seq_num 1267) 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3630: finishing rollback tid 0 to trial17818531-36 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3630: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:30:56.447 INFO:tasks.rados.rados.0.trial178.stdout:3630: finishing rollback tid 2 to trial17818531-36 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1625 (ObjNum 1254 snap 359 seq_num 1254) dirty exists 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3624: done (5 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3625: done (4 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3626: done (3 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3627: done (2 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3629: done (1 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3630: done (0 left) 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3631: write (excl) oid 9 current snap is 362 2026-02-21T22:30:56.448 INFO:tasks.rados.rados.0.trial178.stdout:3631: seq_num 1268 ranges {511276=730330,1800344=559437,3021296=289980} 2026-02-21T22:30:56.454 INFO:tasks.rados.rados.0.trial178.stdout:3631: writing trial17818531-9 from 511276 to 1241606 tid 1 2026-02-21T22:30:56.462 INFO:tasks.rados.rados.0.trial178.stdout:3631: writing trial17818531-9 from 1800344 to 2359781 tid 2 2026-02-21T22:30:56.468 INFO:tasks.rados.rados.0.trial178.stdout:3631: writing trial17818531-9 from 3021296 to 3311276 tid 3 2026-02-21T22:30:56.468 INFO:tasks.rados.rados.0.trial178.stdout:3632: rollback oid 44 current snap is 362 2026-02-21T22:30:56.469 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 361 2026-02-21T22:30:56.469 INFO:tasks.rados.rados.0.trial178.stdout:3631: finishing write tid 1 to trial17818531-9 2026-02-21T22:30:56.469 INFO:tasks.rados.rados.0.trial178.stdout:3631: finishing write tid 2 to trial17818531-9 2026-02-21T22:30:56.469 INFO:tasks.rados.rados.0.trial178.stdout:3633: write (excl) oid 23 current snap is 362 2026-02-21T22:30:56.469 INFO:tasks.rados.rados.0.trial178.stdout:3633: seq_num 1269 ranges {528083=405579,1342473=419448,2248719=733949} 2026-02-21T22:30:56.475 INFO:tasks.rados.rados.0.trial178.stdout:3633: writing trial17818531-23 from 528083 to 933662 tid 1 2026-02-21T22:30:56.480 INFO:tasks.rados.rados.0.trial178.stdout:3633: writing trial17818531-23 from 1342473 to 1761921 tid 2 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3633: writing trial17818531-23 from 2248719 to 2982668 tid 3 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3634: read oid 12 snap -1 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3634: expect (ObjNum 1221 snap 348 seq_num 1221) 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3635: snap_create 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3632: finishing rollback tid 0 to trial17818531-44 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3632: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3631: finishing write tid 3 to trial17818531-9 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3632: finishing rollback tid 2 to trial17818531-44 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1852 (ObjNum 1260 snap 359 seq_num 1260) dirty exists 2026-02-21T22:30:56.489 INFO:tasks.rados.rados.0.trial178.stdout:3631: finishing write tid 4 to trial17818531-9 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: finishing write tid 5 to trial17818531-9 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: oid 9 updating version 0 to 1635 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: oid 9 updating version 1635 to 1636 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: oid 9 version 1636 is already newer than 1634 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: oid 9 updating version 1636 to 1637 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1637 (ObjNum 1268 snap 362 seq_num 1268) dirty exists 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3631: left oid 9 (ObjNum 1268 snap 362 seq_num 1268) 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3633: finishing write tid 1 to trial17818531-23 2026-02-21T22:30:56.490 INFO:tasks.rados.rados.0.trial178.stdout:3633: finishing write tid 2 to trial17818531-23 2026-02-21T22:30:56.495 INFO:tasks.rados.rados.0.trial178.stdout:3633: finishing write tid 3 to trial17818531-23 2026-02-21T22:30:56.495 INFO:tasks.rados.rados.0.trial178.stdout:3633: finishing write tid 4 to trial17818531-23 2026-02-21T22:30:56.495 INFO:tasks.rados.rados.0.trial178.stdout:3634: expect (ObjNum 1221 snap 348 seq_num 1221) 2026-02-21T22:30:56.501 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 885848 2026-02-21T22:30:56.528 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:56.592 INFO:tasks.rados.rados.0.trial178.stdout:3633: finishing write tid 5 to trial17818531-23 2026-02-21T22:30:56.592 INFO:tasks.rados.rados.0.trial178.stdout:3633: oid 23 updating version 0 to 1422 2026-02-21T22:30:56.593 INFO:tasks.rados.rados.0.trial178.stdout:3633: oid 23 updating version 1422 to 1424 2026-02-21T22:30:56.593 INFO:tasks.rados.rados.0.trial178.stdout:3633: oid 23 version 1424 is already newer than 1421 2026-02-21T22:30:56.593 INFO:tasks.rados.rados.0.trial178.stdout:3633: oid 23 version 1424 is already newer than 1423 2026-02-21T22:30:56.593 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1424 (ObjNum 1269 snap 362 seq_num 1269) dirty exists 2026-02-21T22:30:56.593 INFO:tasks.rados.rados.0.trial178.stdout:3633: left oid 23 (ObjNum 1269 snap 362 seq_num 1269) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3631: done (4 left) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3632: done (3 left) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3633: done (2 left) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3634: done (1 left) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3635: done (0 left) 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3636: rollback oid 38 current snap is 363 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 362 2026-02-21T22:30:57.213 INFO:tasks.rados.rados.0.trial178.stdout:3637: read oid 27 snap -1 2026-02-21T22:30:57.214 INFO:tasks.rados.rados.0.trial178.stdout:3637: expect deleted 2026-02-21T22:30:57.214 INFO:tasks.rados.rados.0.trial178.stdout:3638: snap_remove snap 360 2026-02-21T22:30:58.216 INFO:tasks.rados.rados.0.trial178.stdout:3638: done (2 left) 2026-02-21T22:30:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3639: snap_create 2026-02-21T22:30:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3636: finishing rollback tid 0 to trial17818531-38 2026-02-21T22:30:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3636: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:30:58.217 INFO:tasks.rados.rados.0.trial178.stdout:3636: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:30:58.217 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1810 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:30:58.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:30:58.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:30:59.217 INFO:tasks.rados.rados.0.trial178.stdout:3636: done (2 left) 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3637: done (1 left) 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3639: done (0 left) 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3640: read oid 9 snap 363 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3640: expect (ObjNum 1268 snap 362 seq_num 1268) 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3641: rollback oid 20 current snap is 364 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 361 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3642: write (excl) oid 19 current snap is 364 2026-02-21T22:30:59.218 INFO:tasks.rados.rados.0.trial178.stdout:3642: seq_num 1270 ranges {544890=597181,1917308=563106,3076142=268748} 2026-02-21T22:30:59.225 INFO:tasks.rados.rados.0.trial178.stdout:3642: writing trial17818531-19 from 544890 to 1142071 tid 1 2026-02-21T22:30:59.234 INFO:tasks.rados.rados.0.trial178.stdout:3642: writing trial17818531-19 from 1917308 to 2480414 tid 2 2026-02-21T22:30:59.240 INFO:tasks.rados.rados.0.trial178.stdout:3642: writing trial17818531-19 from 3076142 to 3344890 tid 3 2026-02-21T22:30:59.240 INFO:tasks.rados.rados.0.trial178.stdout:3643: delete oid 12 current snap is 364 2026-02-21T22:30:59.240 INFO:tasks.rados.rados.0.trial178.stdout:3641: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:30:59.241 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 0 (ObjNum 129 snap 0 seq_num 100954520) dirty dne 2026-02-21T22:30:59.241 INFO:tasks.rados.rados.0.trial178.stdout:3640: expect (ObjNum 1268 snap 362 seq_num 1268) 2026-02-21T22:30:59.253 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1929758 2026-02-21T22:30:59.270 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3642: finishing write tid 1 to trial17818531-19 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3642: finishing write tid 2 to trial17818531-19 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3640: done (3 left) 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3641: done (2 left) 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3643: done (1 left) 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:3644: rollback oid 3 current snap is 364 2026-02-21T22:30:59.328 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 361 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: finishing write tid 3 to trial17818531-19 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: finishing write tid 4 to trial17818531-19 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: finishing write tid 5 to trial17818531-19 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: oid 19 updating version 0 to 1966 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: oid 19 version 1966 is already newer than 1963 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: oid 19 version 1966 is already newer than 1964 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: oid 19 version 1966 is already newer than 1965 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1966 (ObjNum 1270 snap 364 seq_num 1270) dirty exists 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: left oid 19 (ObjNum 1270 snap 364 seq_num 1270) 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3642: done (1 left) 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3645: delete oid 16 current snap is 364 2026-02-21T22:30:59.329 INFO:tasks.rados.rados.0.trial178.stdout:3645: done (1 left) 2026-02-21T22:30:59.330 INFO:tasks.rados.rados.0.trial178.stdout:3646: read oid 11 snap -1 2026-02-21T22:30:59.330 INFO:tasks.rados.rados.0.trial178.stdout:3646: expect (ObjNum 1265 snap 362 seq_num 1265) 2026-02-21T22:30:59.330 INFO:tasks.rados.rados.0.trial178.stdout:3647: write (excl) oid 20 current snap is 364 2026-02-21T22:30:59.330 INFO:tasks.rados.rados.0.trial178.stdout:3647: seq_num 1273 ranges {595311=422928,1576401=710433,3042058=353253} 2026-02-21T22:30:59.335 INFO:tasks.rados.rados.0.trial178.stdout:3647: writing trial17818531-20 from 595311 to 1018239 tid 1 2026-02-21T22:30:59.343 INFO:tasks.rados.rados.0.trial178.stdout:3647: writing trial17818531-20 from 1576401 to 2286834 tid 2 2026-02-21T22:30:59.351 INFO:tasks.rados.rados.0.trial178.stdout:3647: writing trial17818531-20 from 3042058 to 3395311 tid 3 2026-02-21T22:30:59.351 INFO:tasks.rados.rados.0.trial178.stdout:3648: write oid 14 current snap is 364 2026-02-21T22:30:59.352 INFO:tasks.rados.rados.0.trial178.stdout:3648: seq_num 1274 ranges {612118=498177,1634883=570444,2669481=706662} 2026-02-21T22:30:59.359 INFO:tasks.rados.rados.0.trial178.stdout:3648: writing trial17818531-14 from 612118 to 1110295 tid 1 2026-02-21T22:30:59.366 INFO:tasks.rados.rados.0.trial178.stdout:3648: writing trial17818531-14 from 1634883 to 2205327 tid 2 2026-02-21T22:30:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3648: writing trial17818531-14 from 2669481 to 3376143 tid 3 2026-02-21T22:30:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3649: write (excl) oid 33 current snap is 364 2026-02-21T22:30:59.373 INFO:tasks.rados.rados.0.trial178.stdout:3649: seq_num 1275 ranges {628925=573426,1693365=714102,3096904=332021} 2026-02-21T22:30:59.377 INFO:tasks.rados.rados.0.trial178.stdout:3649: writing trial17818531-33 from 628925 to 1202351 tid 1 2026-02-21T22:30:59.381 INFO:tasks.rados.rados.0.trial178.stdout:3649: writing trial17818531-33 from 1693365 to 2407467 tid 2 2026-02-21T22:30:59.385 INFO:tasks.rados.rados.0.trial178.stdout:3649: writing trial17818531-33 from 3096904 to 3428925 tid 3 2026-02-21T22:30:59.386 INFO:tasks.rados.rados.0.trial178.stdout:3650: write oid 50 current snap is 364 2026-02-21T22:30:59.386 INFO:tasks.rados.rados.0.trial178.stdout:3650: seq_num 1276 ranges {645732=648675,1751847=574113,3124327=321405} 2026-02-21T22:30:59.390 INFO:tasks.rados.rados.0.trial178.stdout:3650: writing trial17818531-50 from 645732 to 1294407 tid 1 2026-02-21T22:30:59.394 INFO:tasks.rados.rados.0.trial178.stdout:3650: writing trial17818531-50 from 1751847 to 2325960 tid 2 2026-02-21T22:30:59.398 INFO:tasks.rados.rados.0.trial178.stdout:3650: writing trial17818531-50 from 3124327 to 3445732 tid 3 2026-02-21T22:30:59.398 INFO:tasks.rados.rados.0.trial178.stdout:3651: snap_create 2026-02-21T22:30:59.398 INFO:tasks.rados.rados.0.trial178.stdout:3644: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:30:59.399 INFO:tasks.rados.rados.0.trial178.stdout:3644: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:30:59.399 INFO:tasks.rados.rados.0.trial178.stdout:3644: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:30:59.399 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1793 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:30:59.399 INFO:tasks.rados.rados.0.trial178.stdout:3646: expect (ObjNum 1265 snap 362 seq_num 1265) 2026-02-21T22:30:59.399 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: finishing write tid 1 to trial17818531-20 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: finishing write tid 2 to trial17818531-20 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: finishing write tid 3 to trial17818531-20 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: finishing write tid 4 to trial17818531-20 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: finishing write tid 5 to trial17818531-20 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: oid 20 updating version 0 to 1743 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: oid 20 version 1743 is already newer than 1741 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: oid 20 version 1743 is already newer than 1742 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: oid 20 version 1743 is already newer than 1740 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1743 (ObjNum 1273 snap 364 seq_num 1273) dirty exists 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3647: left oid 20 (ObjNum 1273 snap 364 seq_num 1273) 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: finishing write tid 1 to trial17818531-14 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: finishing write tid 2 to trial17818531-14 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: finishing write tid 3 to trial17818531-14 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: finishing write tid 4 to trial17818531-14 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: finishing write tid 5 to trial17818531-14 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: oid 14 updating version 0 to 1639 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: oid 14 updating version 1639 to 1640 2026-02-21T22:30:59.562 INFO:tasks.rados.rados.0.trial178.stdout:3648: oid 14 updating version 1640 to 1641 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3648: oid 14 version 1641 is already newer than 1638 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 14 v 1641 (ObjNum 1274 snap 364 seq_num 1274) dirty exists 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3648: left oid 14 (ObjNum 1274 snap 364 seq_num 1274) 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: finishing write tid 1 to trial17818531-33 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: finishing write tid 2 to trial17818531-33 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: finishing write tid 3 to trial17818531-33 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: finishing write tid 4 to trial17818531-33 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: finishing write tid 5 to trial17818531-33 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: oid 33 updating version 0 to 1812 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: oid 33 version 1812 is already newer than 1811 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: oid 33 updating version 1812 to 1813 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: oid 33 updating version 1813 to 1814 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1814 (ObjNum 1275 snap 364 seq_num 1275) dirty exists 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3649: left oid 33 (ObjNum 1275 snap 364 seq_num 1275) 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: finishing write tid 1 to trial17818531-50 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: finishing write tid 2 to trial17818531-50 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: finishing write tid 3 to trial17818531-50 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: finishing write tid 4 to trial17818531-50 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: finishing write tid 5 to trial17818531-50 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: oid 50 updating version 0 to 1651 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: oid 50 updating version 1651 to 1653 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: oid 50 updating version 1653 to 1654 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: oid 50 version 1654 is already newer than 1652 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1654 (ObjNum 1276 snap 364 seq_num 1276) dirty exists 2026-02-21T22:30:59.563 INFO:tasks.rados.rados.0.trial178.stdout:3650: left oid 50 (ObjNum 1276 snap 364 seq_num 1276) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3644: done (6 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3646: done (5 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3647: done (4 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3648: done (3 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3649: done (2 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3650: done (1 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3651: done (0 left) 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3652: rollback oid 27 current snap is 365 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 27 to 363 2026-02-21T22:31:00.224 INFO:tasks.rados.rados.0.trial178.stdout:3653: snap_create 2026-02-21T22:31:00.226 INFO:tasks.rados.rados.0.trial178.stdout:3652: finishing rollback tid 1 to trial17818531-27 2026-02-21T22:31:00.226 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 0 (ObjNum 1201 snap 0 seq_num 100624256) dirty dne 2026-02-21T22:31:01.225 INFO:tasks.rados.rados.0.trial178.stdout:3652: done (1 left) 2026-02-21T22:31:01.225 INFO:tasks.rados.rados.0.trial178.stdout:3653: done (0 left) 2026-02-21T22:31:01.225 INFO:tasks.rados.rados.0.trial178.stdout:3654: snap_remove snap 362 2026-02-21T22:31:02.212 INFO:tasks.rados.rados.0.trial178.stdout:3654: done (0 left) 2026-02-21T22:31:02.213 INFO:tasks.rados.rados.0.trial178.stdout:3655: snap_create 2026-02-21T22:31:03.215 INFO:tasks.rados.rados.0.trial178.stdout:3655: done (0 left) 2026-02-21T22:31:03.216 INFO:tasks.rados.rados.0.trial178.stdout:3656: snap_create 2026-02-21T22:31:03.590 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:03.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:04.216 INFO:tasks.rados.rados.0.trial178.stdout:3656: done (0 left) 2026-02-21T22:31:04.216 INFO:tasks.rados.rados.0.trial178.stdout:3657: write oid 6 current snap is 368 2026-02-21T22:31:04.216 INFO:tasks.rados.rados.0.trial178.stdout:3657: seq_num 1277 ranges {662539=723924,2093976=717771,3435397=27142} 2026-02-21T22:31:04.225 INFO:tasks.rados.rados.0.trial178.stdout:3657: writing trial17818531-6 from 662539 to 1386463 tid 1 2026-02-21T22:31:04.235 INFO:tasks.rados.rados.0.trial178.stdout:3657: writing trial17818531-6 from 2093976 to 2811747 tid 2 2026-02-21T22:31:04.240 INFO:tasks.rados.rados.0.trial178.stdout:3657: writing trial17818531-6 from 3435397 to 3462539 tid 3 2026-02-21T22:31:04.240 INFO:tasks.rados.rados.0.trial178.stdout:3658: snap_create 2026-02-21T22:31:04.240 INFO:tasks.rados.rados.0.trial178.stdout:3657: finishing write tid 1 to trial17818531-6 2026-02-21T22:31:04.240 INFO:tasks.rados.rados.0.trial178.stdout:3657: finishing write tid 2 to trial17818531-6 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: finishing write tid 3 to trial17818531-6 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: finishing write tid 4 to trial17818531-6 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: finishing write tid 5 to trial17818531-6 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: oid 6 updating version 0 to 1653 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: oid 6 updating version 1653 to 1654 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: oid 6 version 1654 is already newer than 1652 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: oid 6 updating version 1654 to 1655 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1655 (ObjNum 1277 snap 368 seq_num 1277) dirty exists 2026-02-21T22:31:04.241 INFO:tasks.rados.rados.0.trial178.stdout:3657: left oid 6 (ObjNum 1277 snap 368 seq_num 1277) 2026-02-21T22:31:05.217 INFO:tasks.rados.rados.0.trial178.stdout:3657: done (1 left) 2026-02-21T22:31:05.218 INFO:tasks.rados.rados.0.trial178.stdout:3658: done (0 left) 2026-02-21T22:31:05.218 INFO:tasks.rados.rados.0.trial178.stdout:3659: delete oid 6 current snap is 369 2026-02-21T22:31:05.223 INFO:tasks.rados.rados.0.trial178.stdout:3659: done (0 left) 2026-02-21T22:31:05.223 INFO:tasks.rados.rados.0.trial178.stdout:3660: delete oid 15 current snap is 369 2026-02-21T22:31:05.226 INFO:tasks.rados.rados.0.trial178.stdout:3660: done (0 left) 2026-02-21T22:31:05.226 INFO:tasks.rados.rados.0.trial178.stdout:3661: read oid 47 snap 366 2026-02-21T22:31:05.227 INFO:tasks.rados.rados.0.trial178.stdout:3661: expect (ObjNum 1246 snap 357 seq_num 1246) 2026-02-21T22:31:05.227 INFO:tasks.rados.rados.0.trial178.stdout:3662: delete oid 49 current snap is 369 2026-02-21T22:31:05.228 INFO:tasks.rados.rados.0.trial178.stdout:3662: done (1 left) 2026-02-21T22:31:05.228 INFO:tasks.rados.rados.0.trial178.stdout:3663: write oid 19 current snap is 369 2026-02-21T22:31:05.228 INFO:tasks.rados.rados.0.trial178.stdout:3663: seq_num 1281 ranges {729767=741273,1927904=725109,3145089=384678} 2026-02-21T22:31:05.237 INFO:tasks.rados.rados.0.trial178.stdout:3663: writing trial17818531-19 from 729767 to 1471040 tid 1 2026-02-21T22:31:05.246 INFO:tasks.rados.rados.0.trial178.stdout:3663: writing trial17818531-19 from 1927904 to 2653013 tid 2 2026-02-21T22:31:05.252 INFO:tasks.rados.rados.0.trial178.stdout:3663: writing trial17818531-19 from 3145089 to 3529767 tid 3 2026-02-21T22:31:05.252 INFO:tasks.rados.rados.0.trial178.stdout:3664: delete oid 16 current snap is 369 2026-02-21T22:31:05.252 INFO:tasks.rados.rados.0.trial178.stdout:3661: expect (ObjNum 1246 snap 357 seq_num 1246) 2026-02-21T22:31:05.292 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2569478 2026-02-21T22:31:05.306 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3663: finishing write tid 1 to trial17818531-19 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3663: finishing write tid 2 to trial17818531-19 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3663: finishing write tid 3 to trial17818531-19 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3663: finishing write tid 4 to trial17818531-19 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3661: done (2 left) 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3664: done (1 left) 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3665: write (excl) oid 37 current snap is 369 2026-02-21T22:31:05.384 INFO:tasks.rados.rados.0.trial178.stdout:3665: seq_num 1283 ranges {763381=491771,2044868=728778,3199935=363446} 2026-02-21T22:31:05.392 INFO:tasks.rados.rados.0.trial178.stdout:3665: writing trial17818531-37 from 763381 to 1255152 tid 1 2026-02-21T22:31:05.403 INFO:tasks.rados.rados.0.trial178.stdout:3665: writing trial17818531-37 from 2044868 to 2773646 tid 2 2026-02-21T22:31:05.408 INFO:tasks.rados.rados.0.trial178.stdout:3665: writing trial17818531-37 from 3199935 to 3563381 tid 3 2026-02-21T22:31:05.409 INFO:tasks.rados.rados.0.trial178.stdout:3666: write (excl) oid 39 current snap is 369 2026-02-21T22:31:05.409 INFO:tasks.rados.rados.0.trial178.stdout:3666: seq_num 1284 ranges {780188=567020,2103350=472436,3227358=352830} 2026-02-21T22:31:05.418 INFO:tasks.rados.rados.0.trial178.stdout:3666: writing trial17818531-39 from 780188 to 1347208 tid 1 2026-02-21T22:31:05.426 INFO:tasks.rados.rados.0.trial178.stdout:3666: writing trial17818531-39 from 2103350 to 2575786 tid 2 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3666: writing trial17818531-39 from 3227358 to 3580188 tid 3 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3667: delete oid 43 current snap is 369 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3663: finishing write tid 5 to trial17818531-19 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3663: oid 19 updating version 0 to 1970 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3663: oid 19 updating version 1970 to 1973 2026-02-21T22:31:05.433 INFO:tasks.rados.rados.0.trial178.stdout:3663: oid 19 version 1973 is already newer than 1972 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3663: oid 19 version 1973 is already newer than 1971 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 1973 (ObjNum 1281 snap 369 seq_num 1281) dirty exists 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3663: left oid 19 (ObjNum 1281 snap 369 seq_num 1281) 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: finishing write tid 1 to trial17818531-37 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: finishing write tid 2 to trial17818531-37 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: finishing write tid 3 to trial17818531-37 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: finishing write tid 4 to trial17818531-37 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: finishing write tid 5 to trial17818531-37 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: oid 37 updating version 0 to 1641 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: oid 37 version 1641 is already newer than 1640 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: oid 37 updating version 1641 to 1642 2026-02-21T22:31:05.434 INFO:tasks.rados.rados.0.trial178.stdout:3665: oid 37 updating version 1642 to 1643 2026-02-21T22:31:05.435 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1643 (ObjNum 1283 snap 369 seq_num 1283) dirty exists 2026-02-21T22:31:05.435 INFO:tasks.rados.rados.0.trial178.stdout:3665: left oid 37 (ObjNum 1283 snap 369 seq_num 1283) 2026-02-21T22:31:05.435 INFO:tasks.rados.rados.0.trial178.stdout:3666: finishing write tid 1 to trial17818531-39 2026-02-21T22:31:05.435 INFO:tasks.rados.rados.0.trial178.stdout:3666: finishing write tid 2 to trial17818531-39 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: finishing write tid 3 to trial17818531-39 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: finishing write tid 4 to trial17818531-39 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: finishing write tid 5 to trial17818531-39 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: oid 39 updating version 0 to 1644 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: oid 39 version 1644 is already newer than 1642 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: oid 39 updating version 1644 to 1645 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: oid 39 version 1645 is already newer than 1643 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1645 (ObjNum 1284 snap 369 seq_num 1284) dirty exists 2026-02-21T22:31:05.436 INFO:tasks.rados.rados.0.trial178.stdout:3666: left oid 39 (ObjNum 1284 snap 369 seq_num 1284) 2026-02-21T22:31:05.437 INFO:tasks.rados.rados.0.trial178.stdout:3663: done (3 left) 2026-02-21T22:31:05.437 INFO:tasks.rados.rados.0.trial178.stdout:3665: done (2 left) 2026-02-21T22:31:05.437 INFO:tasks.rados.rados.0.trial178.stdout:3666: done (1 left) 2026-02-21T22:31:05.437 INFO:tasks.rados.rados.0.trial178.stdout:3667: done (0 left) 2026-02-21T22:31:05.437 INFO:tasks.rados.rados.0.trial178.stdout:3668: snap_create 2026-02-21T22:31:06.225 INFO:tasks.rados.rados.0.trial178.stdout:3668: done (0 left) 2026-02-21T22:31:06.225 INFO:tasks.rados.rados.0.trial178.stdout:3669: read oid 42 snap -1 2026-02-21T22:31:06.225 INFO:tasks.rados.rados.0.trial178.stdout:3669: expect (ObjNum 1259 snap 359 seq_num 1259) 2026-02-21T22:31:06.225 INFO:tasks.rados.rados.0.trial178.stdout:3670: write (excl) oid 43 current snap is 370 2026-02-21T22:31:06.225 INFO:tasks.rados.rados.0.trial178.stdout:3670: seq_num 1286 ranges {413802=433871,1420314=476105,2482204=790985} 2026-02-21T22:31:06.231 INFO:tasks.rados.rados.0.trial178.stdout:3670: writing trial17818531-43 from 413802 to 847673 tid 1 2026-02-21T22:31:06.238 INFO:tasks.rados.rados.0.trial178.stdout:3670: writing trial17818531-43 from 1420314 to 1896419 tid 2 2026-02-21T22:31:06.247 INFO:tasks.rados.rados.0.trial178.stdout:3670: writing trial17818531-43 from 2482204 to 3273189 tid 3 2026-02-21T22:31:06.247 INFO:tasks.rados.rados.0.trial178.stdout:3671: snap_create 2026-02-21T22:31:06.247 INFO:tasks.rados.rados.0.trial178.stdout:3670: finishing write tid 1 to trial17818531-43 2026-02-21T22:31:06.247 INFO:tasks.rados.rados.0.trial178.stdout:3669: expect (ObjNum 1259 snap 359 seq_num 1259) 2026-02-21T22:31:06.247 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:06.346 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2689573 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: finishing write tid 2 to trial17818531-43 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: finishing write tid 3 to trial17818531-43 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: finishing write tid 4 to trial17818531-43 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: finishing write tid 5 to trial17818531-43 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: oid 43 updating version 0 to 1608 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: oid 43 updating version 1608 to 1609 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: oid 43 version 1609 is already newer than 1607 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: oid 43 updating version 1609 to 1610 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1610 (ObjNum 1286 snap 370 seq_num 1286) dirty exists 2026-02-21T22:31:06.357 INFO:tasks.rados.rados.0.trial178.stdout:3670: left oid 43 (ObjNum 1286 snap 370 seq_num 1286) 2026-02-21T22:31:07.213 INFO:tasks.rados.rados.0.trial178.stdout:3669: done (2 left) 2026-02-21T22:31:07.213 INFO:tasks.rados.rados.0.trial178.stdout:3670: done (1 left) 2026-02-21T22:31:07.213 INFO:tasks.rados.rados.0.trial178.stdout:3671: done (0 left) 2026-02-21T22:31:07.213 INFO:tasks.rados.rados.0.trial178.stdout:3672: delete oid 14 current snap is 371 2026-02-21T22:31:07.218 INFO:tasks.rados.rados.0.trial178.stdout:3672: done (0 left) 2026-02-21T22:31:07.218 INFO:tasks.rados.rados.0.trial178.stdout:3673: snap_create 2026-02-21T22:31:08.216 INFO:tasks.rados.rados.0.trial178.stdout:3673: done (0 left) 2026-02-21T22:31:08.217 INFO:tasks.rados.rados.0.trial178.stdout:3674: read oid 20 snap -1 2026-02-21T22:31:08.217 INFO:tasks.rados.rados.0.trial178.stdout:3674: expect (ObjNum 1273 snap 364 seq_num 1273) 2026-02-21T22:31:08.217 INFO:tasks.rados.rados.0.trial178.stdout:3675: snap_remove snap 367 2026-02-21T22:31:08.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:08.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:09.217 INFO:tasks.rados.rados.0.trial178.stdout:3675: done (1 left) 2026-02-21T22:31:09.217 INFO:tasks.rados.rados.0.trial178.stdout:3676: snap_create 2026-02-21T22:31:09.217 INFO:tasks.rados.rados.0.trial178.stdout:3674: expect (ObjNum 1273 snap 364 seq_num 1273) 2026-02-21T22:31:09.230 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1977535 2026-02-21T22:31:09.250 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:09.283 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 639661 2026-02-21T22:31:10.223 INFO:tasks.rados.rados.0.trial178.stdout:3674: done (1 left) 2026-02-21T22:31:10.223 INFO:tasks.rados.rados.0.trial178.stdout:3676: done (0 left) 2026-02-21T22:31:10.223 INFO:tasks.rados.rados.0.trial178.stdout:3677: rollback oid 26 current snap is 373 2026-02-21T22:31:10.223 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 26 to 372 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3678: read oid 36 snap -1 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3678: expect (ObjNum 1254 snap 359 seq_num 1254) 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3679: read oid 34 snap -1 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3679: expect deleted 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3680: copy_from oid 4 from oid 38 current snap is 373 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3681: read oid 5 snap 366 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3681: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:31:10.224 INFO:tasks.rados.rados.0.trial178.stdout:3682: snap_remove snap 361 2026-02-21T22:31:11.226 INFO:tasks.rados.rados.0.trial178.stdout:3682: done (5 left) 2026-02-21T22:31:11.226 INFO:tasks.rados.rados.0.trial178.stdout:3683: copy_from oid 27 from oid 9 current snap is 373 2026-02-21T22:31:11.226 INFO:tasks.rados.rados.0.trial178.stdout:3677: finishing rollback tid 1 to trial17818531-26 2026-02-21T22:31:11.226 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 0 (ObjNum 33 snap 0 seq_num 3) dirty dne 2026-02-21T22:31:11.227 INFO:tasks.rados.rados.0.trial178.stdout:3678: expect (ObjNum 1254 snap 359 seq_num 1254) 2026-02-21T22:31:11.312 INFO:tasks.rados.rados.0.trial178.stdout:3681: expect (ObjNum 997 snap 282 seq_num 997) 2026-02-21T22:31:11.312 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:11.354 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2160206 2026-02-21T22:31:11.358 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 593830 2026-02-21T22:31:11.383 INFO:tasks.rados.rados.0.trial178.stdout:3680: finishing copy_from racing read to trial17818531-4 2026-02-21T22:31:11.383 INFO:tasks.rados.rados.0.trial178.stdout:3680: finishing copy_from to trial17818531-4 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1810 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3683: finishing copy_from racing read to trial17818531-27 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3683: finishing copy_from to trial17818531-27 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1637 (ObjNum 1268 snap 362 seq_num 1268) dirty exists 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3677: done (5 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3678: done (4 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3679: done (3 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3680: done (2 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3681: done (1 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3683: done (0 left) 2026-02-21T22:31:11.384 INFO:tasks.rados.rados.0.trial178.stdout:3684: write oid 1 current snap is 373 2026-02-21T22:31:11.385 INFO:tasks.rados.rados.0.trial178.stdout:3684: seq_num 1288 ranges {447416=584369,1537278=479774,2537050=529882,3524784=122632} 2026-02-21T22:31:11.390 INFO:tasks.rados.rados.0.trial178.stdout:3684: writing trial17818531-1 from 447416 to 1031785 tid 1 2026-02-21T22:31:11.397 INFO:tasks.rados.rados.0.trial178.stdout:3684: writing trial17818531-1 from 1537278 to 2017052 tid 2 2026-02-21T22:31:11.404 INFO:tasks.rados.rados.0.trial178.stdout:3684: writing trial17818531-1 from 2537050 to 3066932 tid 3 2026-02-21T22:31:11.408 INFO:tasks.rados.rados.0.trial178.stdout:3684: writing trial17818531-1 from 3524784 to 3647416 tid 4 2026-02-21T22:31:11.408 INFO:tasks.rados.rados.0.trial178.stdout:3685: delete oid 38 current snap is 373 2026-02-21T22:31:11.409 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 1 to trial17818531-1 2026-02-21T22:31:11.409 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 2 to trial17818531-1 2026-02-21T22:31:11.409 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 3 to trial17818531-1 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 4 to trial17818531-1 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 5 to trial17818531-1 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: finishing write tid 6 to trial17818531-1 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: oid 1 updating version 0 to 1644 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: oid 1 updating version 1644 to 1646 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: oid 1 updating version 1646 to 1647 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: oid 1 version 1647 is already newer than 1645 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: oid 1 updating version 1647 to 1648 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1648 (ObjNum 1288 snap 373 seq_num 1288) dirty exists 2026-02-21T22:31:11.411 INFO:tasks.rados.rados.0.trial178.stdout:3684: left oid 1 (ObjNum 1288 snap 373 seq_num 1288) 2026-02-21T22:31:11.412 INFO:tasks.rados.rados.0.trial178.stdout:3684: done (1 left) 2026-02-21T22:31:11.412 INFO:tasks.rados.rados.0.trial178.stdout:3685: done (0 left) 2026-02-21T22:31:11.412 INFO:tasks.rados.rados.0.trial178.stdout:3686: read oid 24 snap -1 2026-02-21T22:31:11.412 INFO:tasks.rados.rados.0.trial178.stdout:3686: expect (ObjNum 1251 snap 358 seq_num 1251) 2026-02-21T22:31:11.412 INFO:tasks.rados.rados.0.trial178.stdout:3687: snap_remove snap 372 2026-02-21T22:31:12.216 INFO:tasks.rados.rados.0.trial178.stdout:3687: done (1 left) 2026-02-21T22:31:12.216 INFO:tasks.rados.rados.0.trial178.stdout:3688: write (excl) oid 27 current snap is 373 2026-02-21T22:31:12.216 INFO:tasks.rados.rados.0.trial178.stdout:3688: seq_num 1290 ranges {481030=734867,1937889=483443,2875543=552426} 2026-02-21T22:31:12.224 INFO:tasks.rados.rados.0.trial178.stdout:3688: writing trial17818531-27 from 481030 to 1215897 tid 1 2026-02-21T22:31:12.232 INFO:tasks.rados.rados.0.trial178.stdout:3688: writing trial17818531-27 from 1937889 to 2421332 tid 2 2026-02-21T22:31:12.239 INFO:tasks.rados.rados.0.trial178.stdout:3688: writing trial17818531-27 from 2875543 to 3427969 tid 3 2026-02-21T22:31:12.239 INFO:tasks.rados.rados.0.trial178.stdout:3689: delete oid 43 current snap is 373 2026-02-21T22:31:12.239 INFO:tasks.rados.rados.0.trial178.stdout:3686: expect (ObjNum 1251 snap 358 seq_num 1251) 2026-02-21T22:31:12.270 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2821515 2026-02-21T22:31:12.294 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2155941 2026-02-21T22:31:12.306 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:12.338 INFO:tasks.rados.rados.0.trial178.stdout:3688: finishing write tid 1 to trial17818531-27 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: finishing write tid 2 to trial17818531-27 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: finishing write tid 3 to trial17818531-27 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: finishing write tid 4 to trial17818531-27 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: finishing write tid 5 to trial17818531-27 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: oid 27 updating version 0 to 1649 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: oid 27 version 1649 is already newer than 1647 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: oid 27 version 1649 is already newer than 1648 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: oid 27 updating version 1649 to 1650 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1650 (ObjNum 1290 snap 373 seq_num 1290) dirty exists 2026-02-21T22:31:12.339 INFO:tasks.rados.rados.0.trial178.stdout:3688: left oid 27 (ObjNum 1290 snap 373 seq_num 1290) 2026-02-21T22:31:12.340 INFO:tasks.rados.rados.0.trial178.stdout:3686: done (2 left) 2026-02-21T22:31:12.340 INFO:tasks.rados.rados.0.trial178.stdout:3688: done (1 left) 2026-02-21T22:31:12.340 INFO:tasks.rados.rados.0.trial178.stdout:3689: done (0 left) 2026-02-21T22:31:12.340 INFO:tasks.rados.rados.0.trial178.stdout:3690: snap_remove snap 369 2026-02-21T22:31:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3690: done (0 left) 2026-02-21T22:31:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3691: rollback oid 23 current snap is 373 2026-02-21T22:31:13.233 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 370 2026-02-21T22:31:13.233 INFO:tasks.rados.rados.0.trial178.stdout:3692: delete oid 2 current snap is 373 2026-02-21T22:31:13.234 INFO:tasks.rados.rados.0.trial178.stdout:3692: done (1 left) 2026-02-21T22:31:13.234 INFO:tasks.rados.rados.0.trial178.stdout:3693: snap_create 2026-02-21T22:31:13.235 INFO:tasks.rados.rados.0.trial178.stdout:3691: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:31:13.236 INFO:tasks.rados.rados.0.trial178.stdout:3691: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:31:13.236 INFO:tasks.rados.rados.0.trial178.stdout:3691: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:31:13.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1427 (ObjNum 1269 snap 362 seq_num 1269) dirty exists 2026-02-21T22:31:13.322 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:31:13.323 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:31:13.323 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:31:13.323 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:31:13.325 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:31:13.325 INFO:tasks.mon_thrash.mon_thrasher:freezing mon.c 2026-02-21T22:31:13.325 INFO:tasks.mon_thrash:Sending STOP to mon c 2026-02-21T22:31:13.326 INFO:tasks.ceph.mon.c:Sent signal 19 2026-02-21T22:31:13.326 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2026-02-21T22:31:13.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:13.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:18.591 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:18.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:23.540 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:31:23] 2026-02-21T22:31:23.541 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:31:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:31:23, mode upmap, max misplaced 0.050000 2026-02-21T22:31:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:31:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:31:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['unique_pool_0', '.mgr', 'rbd'] 2026-02-21T22:31:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:31:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:31:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:23.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:31:23.576 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:31:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:31:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:31:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:31:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:31:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:31:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:31:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:31:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:31:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:31:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:31:23.581 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:31:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:31:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:31:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:31:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:31:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:31:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:31:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:31:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:31:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:31:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:31:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:31:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:31:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:31:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "766", 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:31:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:31:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:31:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:31:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 766, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 382, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:31:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:31:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:31:23.605 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:31:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:31:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.00033819410536024304 of space, bias 1.0, pg target 0.10145823160807291 quantized to 32 (current 16) 2026-02-21T22:31:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:31:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:28.326 INFO:tasks.mon_thrash.mon_thrasher:unfreezing mon.c 2026-02-21T22:31:28.327 INFO:tasks.mon_thrash:Sending CONT to mon c 2026-02-21T22:31:28.327 INFO:tasks.ceph.mon.c:Sent signal 18 2026-02-21T22:31:28.327 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:31:28.328 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: get_monmap_and_config 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: monmap: 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:28.390816+0000 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:28.390816+0000 2026-02-21T22:31:28.390 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.391 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.391 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.391 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:28.391 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:28.391 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 1 Processor -- start 2026-02-21T22:31:28.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 1 -- start start 2026-02-21T22:31:28.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: init 2026-02-21T22:31:28.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-b con 0x7fa94c08f770 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-c con 0x7fa94c163d90 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-a con 0x7fa94c185890 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c0590e0 con 0x7fa94c185890 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c089960 con 0x7fa94c163d90 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c089c40 con 0x7fa94c08f770 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95345c640 10 monclient(hunting): authenticate will time out at 2097.627253s 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95245a640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa94c08f770 0x7fa94c08fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45250/0 (socket says 10.20.193.102:45250) 2026-02-21T22:31:28.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.386+0000 7fa95245a640 1 -- 10.20.193.102:0/1648988021 learned_addr learned my addr 10.20.193.102:0/1648988021 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95245a640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.102:6790/0 conn(0x7fa94c08f770 legacy=0x7fa94c08fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa952c5b640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.102:6789/0 conn(0x7fa94c185890 legacy=0x7fa94c1a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3002126149 0 0) 0x7fa94c089c40 con 0x7fa94c08f770 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): my global_id is 94256 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa934003620 con 0x7fa94c08f770 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951c59640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.178:6789/0 conn(0x7fa94c163d90 legacy=0x7fa94c184170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 735630129 0 0) 0x7fa94c0590e0 con 0x7fa94c185890 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient(hunting): my global_id is 34752 2026-02-21T22:31:28.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa94c089c40 con 0x7fa94c185890 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 664765964 0 0) 0x7fa934003620 con 0x7fa94c08f770 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.178:6789/0 conn(0x7fa94c163d90 legacy=0x7fa94c184170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.102:6789/0 conn(0x7fa94c185890 legacy=0x7fa94c1a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 monclient: found mon.noname-b 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c058db0 con 0x7fa94c08f770 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa93c003110 con 0x7fa94c08f770 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 5 monclient: authenticate success, global_id 94256 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.398 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: dump: 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa93c0033e0 con 0x7fa94c08f770 2026-02-21T22:31:28.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 1 -- 10.20.193.102:0/1648988021 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa93c004b50 con 0x7fa94c08f770 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa951458640 10 monclient: dump: 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:28.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 4 set_mon_vals no callback set 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: shutdown 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- 10.20.193.102:0/1648988021 >> v1:10.20.193.102:6790/0 conn(0x7fa94c08f770 legacy=0x7fa94c08fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- 10.20.193.102:0/1648988021 shutdown_connections 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- 10.20.193.102:0/1648988021 >> 10.20.193.102:0/1648988021 conn(0x7fa94c056da0 msgr2=0x7fa94c093ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- 10.20.193.102:0/1648988021 shutdown_connections 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- 10.20.193.102:0/1648988021 wait complete. 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: monmap: 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:28.401 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 Processor -- start 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- start start 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: init 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-a con 0x7fa94c08f770 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-b con 0x7fa94c163d90 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient(hunting): picked mon.noname-c con 0x7fa94c185890 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c180d30 con 0x7fa94c08f770 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c167ca0 con 0x7fa94c163d90 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c167f80 con 0x7fa94c185890 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.402 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95345c640 10 monclient(hunting): authenticate will time out at 2097.631310s 2026-02-21T22:31:28.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95245a640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa94c08f770 0x7fa94c180620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41138/0 (socket says 10.20.193.102:41138) 2026-02-21T22:31:28.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa95245a640 1 -- 10.20.193.102:0/2845374415 learned_addr learned my addr 10.20.193.102:0/2845374415 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.390+0000 7fa952c5b640 1 --1- 10.20.193.102:0/2845374415 >> v1:10.20.193.102:6790/0 conn(0x7fa94c185890 0x7fa94c12da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45254/0 (socket says 10.20.193.102:45254) 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa952c5b640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.102:6790/0 conn(0x7fa94c185890 legacy=0x7fa94c12da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95245a640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.102:6789/0 conn(0x7fa94c08f770 legacy=0x7fa94c180620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273939332 0 0) 0x7fa94c167f80 con 0x7fa94c185890 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): my global_id is 94259 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa91c003620 con 0x7fa94c185890 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074690863 0 0) 0x7fa94c180d30 con 0x7fa94c08f770 2026-02-21T22:31:28.405 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): my global_id is 34755 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa94c167f80 con 0x7fa94c08f770 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa951c59640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.178:6789/0 conn(0x7fa94c163d90 legacy=0x7fa94c12c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 358419588 0 0) 0x7fa91c003620 con 0x7fa94c185890 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa94c180d30 con 0x7fa94c185890 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa948004900 con 0x7fa94c185890 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): dump: 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.406 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1948730383 0 0) 0x7fa94c167f80 con 0x7fa94c08f770 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa91c003620 con 0x7fa94c08f770 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa93c0048e0 con 0x7fa94c08f770 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient(hunting): dump: 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.407 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2220766419 0 0) 0x7fa94c180d30 con 0x7fa94c185890 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.178:6789/0 conn(0x7fa94c163d90 legacy=0x7fa94c12c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.102:6789/0 conn(0x7fa94c08f770 legacy=0x7fa94c180620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 monclient: found mon.c 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c167110 con 0x7fa94c185890 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.408 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 5 monclient: authenticate success, global_id 94259 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _renew_subs 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1676e0 con 0x7fa94c185890 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _renew_subs 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa94c1835c0 con 0x7fa94c185890 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa948004ed0 con 0x7fa94c185890 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa948005230 con 0x7fa94c185890 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 10 monclient: dump: 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.409 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa950c57640 4 set_mon_vals no callback set 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c183900 con 0x7fa94c185890 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa9480054c0 con 0x7fa94c185890 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(767..767 src has 527..767) ==== 3947+0+0 (unknown 1283094712 0 0) 0x7fa94804db20 con 0x7fa94c185890 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.394+0000 7fa95245a640 10 monclient: get_auth_request con 0x7fa91c03d6a0 auth_method 0 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.398+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fa948018ab0 con 0x7fa94c185890 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.398+0000 7fa932ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.410 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.398+0000 7fa932ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:28.429 INFO:tasks.rados.rados.0.trial178.stdout:3691: done (1 left) 2026-02-21T22:31:28.429 INFO:tasks.rados.rados.0.trial178.stdout:3693: done (0 left) 2026-02-21T22:31:28.429 INFO:tasks.rados.rados.0.trial178.stdout:3694: snap_remove snap 364 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa94c092c10 con 0x7fa94c185890 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa932ffd640 1 -- 10.20.193.102:0/2845374415 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2273809189) 0x7fa94c092c10 con 0x7fa94c185890 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa932ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa932ffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:28.517 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa91c03d6a0 legacy=0x7fa91c05db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 10 monclient: shutdown 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 >> v1:10.20.193.102:6790/0 conn(0x7fa94c185890 legacy=0x7fa94c12da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 shutdown_connections 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 >> 10.20.193.102:0/2845374415 conn(0x7fa94c056da0 msgr2=0x7fa94c0573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 shutdown_connections 2026-02-21T22:31:28.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.514+0000 7fa95345c640 1 -- 10.20.193.102:0/2845374415 wait complete. 2026-02-21T22:31:28.528 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:31:28.528 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:31:28.528 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:31:28.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:28.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: get_monmap_and_config 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: monmap: 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:28.598807+0000 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:28.598807+0000 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:28.598 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 1 Processor -- start 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 1 -- start start 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: init 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-c con 0x7fd90808c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-a con 0x7fd90808cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-b con 0x7fd908183480 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90808d780 con 0x7fd90808cc80 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd908085da0 con 0x7fd90808c1a0 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd908085420 con 0x7fd908183480 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90e619640 10 monclient(hunting): authenticate will time out at 2097.834714s 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90ce16640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 0x7fd908181d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41140/0 (socket says 10.20.193.102:41140) 2026-02-21T22:31:28.599 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.594+0000 7fd90ce16640 1 -- 10.20.193.102:0/2257730480 learned_addr learned my addr 10.20.193.102:0/2257730480 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.600 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90de18640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.102:6790/0 conn(0x7fd908183480 legacy=0x7fd9081a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.601 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90ce16640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 legacy=0x7fd908181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90d617640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.178:6789/0 conn(0x7fd90808c1a0 legacy=0x7fd90808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1804978456 0 0) 0x7fd908085420 con 0x7fd908183480 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): my global_id is 94262 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8f0003620 con 0x7fd908183480 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576234014 0 0) 0x7fd8fc002b90 con 0x7fd90808cc80 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): my global_id is 34758 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd908085420 con 0x7fd90808cc80 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4181120602 0 0) 0x7fd908085da0 con 0x7fd90808c1a0 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.602 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient(hunting): my global_id is 124105 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8fc002b90 con 0x7fd90808c1a0 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 398624324 0 0) 0x7fd8f0003620 con 0x7fd908183480 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.178:6789/0 conn(0x7fd90808c1a0 legacy=0x7fd90808c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 legacy=0x7fd908181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 monclient: found mon.noname-b 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908086640 con 0x7fd908183480 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd904004670 con 0x7fd908183480 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: dump: 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.603 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 5 monclient: authenticate success, global_id 94262 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd904004940 con 0x7fd908183480 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 1 -- 10.20.193.102:0/2257730480 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd904004dd0 con 0x7fd908183480 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd8f7fff640 10 monclient: dump: 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 4 set_mon_vals no callback set 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: shutdown 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- 10.20.193.102:0/2257730480 >> v1:10.20.193.102:6790/0 conn(0x7fd908183480 legacy=0x7fd9081a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.604 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- 10.20.193.102:0/2257730480 shutdown_connections 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- 10.20.193.102:0/2257730480 >> 10.20.193.102:0/2257730480 conn(0x7fd90808f6a0 msgr2=0x7fd90808faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- 10.20.193.102:0/2257730480 shutdown_connections 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- 10.20.193.102:0/2257730480 wait complete. 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: monmap: 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:28.605 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 Processor -- start 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- start start 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: init 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-c con 0x7fd90808c1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-a con 0x7fd90808cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient(hunting): picked mon.noname-b con 0x7fd908183480 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90807edf0 con 0x7fd90808cc80 2026-02-21T22:31:28.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd908081150 con 0x7fd908183480 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9080808b0 con 0x7fd90808c1a0 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90e619640 10 monclient(hunting): authenticate will time out at 2097.839050s 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90ce16640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 0x7fd908172300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41152/0 (socket says 10.20.193.102:41152) 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.598+0000 7fd90ce16640 1 -- 10.20.193.102:0/3083501318 learned_addr learned my addr 10.20.193.102:0/3083501318 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90d617640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6790/0 conn(0x7fd90808c1a0 legacy=0x7fd908151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90ce16640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 legacy=0x7fd908172300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90de18640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.178:6789/0 conn(0x7fd908183480 legacy=0x7fd908173a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991175855 0 0) 0x7fd9080808b0 con 0x7fd90808c1a0 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): my global_id is 94265 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8d8003620 con 0x7fd90808c1a0 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890247059 0 0) 0x7fd90807edf0 con 0x7fd90808cc80 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): my global_id is 34761 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9080808b0 con 0x7fd90808cc80 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016272375 0 0) 0x7fd908081150 con 0x7fd908183480 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): my global_id is 124108 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd90807edf0 con 0x7fd908183480 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 779210120 0 0) 0x7fd9080808b0 con 0x7fd90808cc80 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd908081150 con 0x7fd90808cc80 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd8fc0048e0 con 0x7fd90808cc80 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): dump: 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2819923573 0 0) 0x7fd8d8003620 con 0x7fd90808c1a0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9080808b0 con 0x7fd90808c1a0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd8f80033c0 con 0x7fd90808c1a0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient(hunting): dump: 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.609 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1678563157 0 0) 0x7fd908081150 con 0x7fd90808cc80 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6790/0 conn(0x7fd90808c1a0 legacy=0x7fd908151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.178:6789/0 conn(0x7fd908183480 legacy=0x7fd908173a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 monclient: found mon.a 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908081ac0 con 0x7fd90808cc80 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 5 monclient: authenticate success, global_id 34761 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 10 monclient: _renew_subs 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd90807f950 con 0x7fd90808cc80 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 10 monclient: _renew_subs 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.610 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd90807e510 con 0x7fd90808cc80 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd8fc004da0 con 0x7fd90808cc80 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd8fc005170 con 0x7fd90808cc80 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 10 monclient: dump: 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f77fe640 4 set_mon_vals no callback set 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd90807e800 con 0x7fd9081517a0 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd8fc005440 con 0x7fd90808cc80 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(768..768 src has 527..768) ==== 3871+0+0 (unknown 462652577 0 0) 0x7fd8fc04c7d0 con 0x7fd90808cc80 2026-02-21T22:31:28.611 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.602+0000 7fd90de18640 10 monclient: get_auth_request con 0x7fd8d803d620 auth_method 0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd90d617640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd9081517a0 legacy=0x7fd90805c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352239489 0 0) 0x7fd90807e800 con 0x7fd9081517a0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient(hunting): my global_id is 34764 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd908081150 con 0x7fd9081517a0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 549377084 0 0) 0x7fd908081150 con 0x7fd9081517a0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd90807e800 con 0x7fd9081517a0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd8f8003af0 con 0x7fd9081517a0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.612 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient: dump: 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 518439520 0 0) 0x7fd90807e800 con 0x7fd9081517a0 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd908088990 con 0x7fd9081517a0 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fd8f80058c0 con 0x7fd9081517a0 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd9081517a0 legacy=0x7fd90805c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.606+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd9081517a0 legacy=0x7fd90805c220 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd90e619640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:28.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd908062ce0 con 0x7fd90804d280 2026-02-21T22:31:28.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd90d617640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd90804d280 legacy=0x7fd908176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575237058 0 0) 0x7fd908062ce0 con 0x7fd90804d280 2026-02-21T22:31:28.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient(hunting): my global_id is 34767 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd90807e800 con 0x7fd90804d280 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2384792958 0 0) 0x7fd90807e800 con 0x7fd90804d280 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd908062ce0 con 0x7fd90804d280 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd8f80041a0 con 0x7fd90804d280 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 10 monclient: dump: 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3598585166 0 0) 0x7fd908062ce0 con 0x7fd90804d280 2026-02-21T22:31:28.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.610+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd908151b70 con 0x7fd90804d280 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 2583781076) 0x7fd8f8005b10 con 0x7fd90804d280 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd8f5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd8f5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd90804d280 legacy=0x7fd908176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd8f5ffb640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd90804d280 legacy=0x7fd908176150 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 837483, 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:28.617 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.618 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.619 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:28.620 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.621 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.622 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.623 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.624 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:28.625 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.626 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:28.627 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:28.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd8d803d620 legacy=0x7fd8d805dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 10 monclient: shutdown 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 >> v1:10.20.193.102:6789/0 conn(0x7fd90808cc80 legacy=0x7fd908172300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90de18640 1 -- 10.20.193.102:0/3083501318 reap_dead start 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90de18640 1 -- 10.20.193.102:0/3083501318 reap_dead start 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 shutdown_connections 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 >> 10.20.193.102:0/3083501318 conn(0x7fd90808f6a0 msgr2=0x7fd908057cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 shutdown_connections 2026-02-21T22:31:28.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.614+0000 7fd90e619640 1 -- 10.20.193.102:0/3083501318 wait complete. 2026-02-21T22:31:28.629 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:28.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:31:28.630 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: get_monmap_and_config 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: monmap: 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:28.698959+0000 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:28.698959+0000 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 1 Processor -- start 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 1 -- start start 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: init 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-c con 0x7f7f4c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-a con 0x7f7f4c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-b con 0x7f7f4c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c086a10 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c08b130 con 0x7f7f4c08ce00 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c08bd60 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f51d87640 10 monclient(hunting): authenticate will time out at 2097.934867s 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f4affd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 0x7f7f4c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41176/0 (socket says 10.20.193.102:41176) 2026-02-21T22:31:28.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.694+0000 7f7f4affd640 1 -- 10.20.193.102:0/1867345962 learned_addr learned my addr 10.20.193.102:0/1867345962 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4bfff640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c08d8e0 legacy=0x7f7f4c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4affd640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 legacy=0x7f7f4c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4b7fe640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.178:6789/0 conn(0x7f7f4c08ce00 legacy=0x7f7f4c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683654778 0 0) 0x7f7f4c086a10 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): my global_id is 34770 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f30003620 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858463674 0 0) 0x7f7f4c08bd60 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): my global_id is 94268 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f4c086a10 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371985789 0 0) 0x7f7f4c08b130 con 0x7f7f4c08ce00 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient(hunting): my global_id is 124111 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f4c08bd60 con 0x7f7f4c08ce00 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2900613162 0 0) 0x7f7f4c086a10 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.178:6789/0 conn(0x7f7f4c08ce00 legacy=0x7f7f4c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 legacy=0x7f7f4c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 monclient: found mon.noname-b 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c08b370 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 5 monclient: authenticate success, global_id 94268 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f400045f0 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: dump: 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7f400048c0 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 1 -- 10.20.193.102:0/1867345962 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f40004dd0 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 4 set_mon_vals no callback set 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4a7fc640 10 monclient: dump: 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: shutdown 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- 10.20.193.102:0/1867345962 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c08d8e0 legacy=0x7f7f4c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- 10.20.193.102:0/1867345962 shutdown_connections 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- 10.20.193.102:0/1867345962 >> 10.20.193.102:0/1867345962 conn(0x7f7f4c091a50 msgr2=0x7f7f4c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- 10.20.193.102:0/1867345962 shutdown_connections 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- 10.20.193.102:0/1867345962 wait complete. 2026-02-21T22:31:28.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: monmap: 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 Processor -- start 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- start start 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: init 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-a con 0x7f7f4c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-b con 0x7f7f4c08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient(hunting): picked mon.noname-c con 0x7f7f4c08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c081090 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c05e380 con 0x7f7f4c08ce00 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c088a70 con 0x7f7f4c08d8e0 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f51d87640 10 monclient(hunting): authenticate will time out at 2097.939540s 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4b7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 0x7f7f4c153260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41188/0 (socket says 10.20.193.102:41188) 2026-02-21T22:31:28.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.698+0000 7f7f4b7fe640 1 -- 10.20.193.102:0/2447781649 learned_addr learned my addr 10.20.193.102:0/2447781649 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f4b7fe640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 legacy=0x7f7f4c153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f4bfff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c08d8e0 legacy=0x7f7f4c1955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 223094659 0 0) 0x7f7f4c081090 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): my global_id is 34773 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f1c003620 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f4affd640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.178:6789/0 conn(0x7f7f4c08ce00 legacy=0x7f7f4c084920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1307907736 0 0) 0x7f7f1c003620 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f4c081090 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f38003490 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): dump: 2026-02-21T22:31:28.706 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680796433 0 0) 0x7f7f4c05e380 con 0x7f7f4c08ce00 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient(hunting): my global_id is 124114 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f1c003620 con 0x7f7f4c08ce00 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3148883935 0 0) 0x7f7f4c081090 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c08d8e0 legacy=0x7f7f4c1955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.178:6789/0 conn(0x7f7f4c08ce00 legacy=0x7f7f4c084920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 monclient: found mon.a 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0833f0 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 5 monclient: authenticate success, global_id 34773 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 10 monclient: _renew_subs 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c082b50 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 10 monclient: _renew_subs 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7f4c085030 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7f38003790 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f38003de0 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f49ffb640 4 set_mon_vals no callback set 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 10 monclient: dump: 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.707 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f7f380040e0 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7f4c0008e0 con 0x7f7f4c07b380 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(768..768 src has 527..768) ==== 3871+0+0 (unknown 462652577 0 0) 0x7f7f3804c780 con 0x7f7f4c08b7f0 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.702+0000 7f7f4bfff640 10 monclient: get_auth_request con 0x7f7f1c03d540 auth_method 0 2026-02-21T22:31:28.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f4affd640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07b380 legacy=0x7f7f4c07b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787562669 0 0) 0x7f7f4c0008e0 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient(hunting): my global_id is 94271 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f4c081090 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1177075213 0 0) 0x7f7f4c081090 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f4c0008e0 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f3c002e70 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient: dump: 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4168438863 0 0) 0x7f7f4c0008e0 con 0x7f7f4c07b380 2026-02-21T22:31:28.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f4c090810 con 0x7f7f4c07b380 2026-02-21T22:31:28.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f7f3c0048e0 con 0x7f7f4c07b380 2026-02-21T22:31:28.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:28.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07b380 legacy=0x7f7f4c07b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.706+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07b380 legacy=0x7f7f4c07b750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f51d87640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:28.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7f4c0909c0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f4affd640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07be60 legacy=0x7f7f4c197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759967232 0 0) 0x7f7f4c0909c0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient(hunting): my global_id is 94274 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f4c0008e0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 648997389 0 0) 0x7f7f4c0008e0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f4c0909c0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7f3c004ed0 con 0x7f7f4c07be60 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient: dump: 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.715 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 93574035 0 0) 0x7f7f4c0909c0 con 0x7f7f4c07be60 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7f4c07c260 con 0x7f7f4c07be60 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 701766038) 0x7f7f3c0071b0 con 0x7f7f4c07be60 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07be60 legacy=0x7f7f4c197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.710+0000 7f7f27fff640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6790/0 conn(0x7f7f4c07be60 legacy=0x7f7f4c197d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 234985, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.716 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:28.717 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.718 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:28.719 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.178:6800/2549949439 conn(0x7f7f1c03d540 legacy=0x7f7f1c05d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 10 monclient: shutdown 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 >> v1:10.20.193.102:6789/0 conn(0x7f7f4c08b7f0 legacy=0x7f7f4c153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f4bfff640 1 -- 10.20.193.102:0/2447781649 reap_dead start 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f4bfff640 1 -- 10.20.193.102:0/2447781649 reap_dead start 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 shutdown_connections 2026-02-21T22:31:28.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 >> 10.20.193.102:0/2447781649 conn(0x7f7f4c091a50 msgr2=0x7f7f4c05c410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 shutdown_connections 2026-02-21T22:31:28.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.714+0000 7f7f51d87640 1 -- 10.20.193.102:0/2447781649 wait complete. 2026-02-21T22:31:28.724 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:28.724 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:31:28.724 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: get_monmap_and_config 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: monmap: 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:28.783559+0000 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:28.783559+0000 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.783 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 1 Processor -- start 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 1 -- start start 2026-02-21T22:31:28.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: init 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-c con 0x7f57500584c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-a con 0x7f57500588a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-b con 0x7f57501435f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57500902a0 con 0x7f57500588a0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57500921d0 con 0x7f57500584c0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57500574a0 con 0x7f57501435f0 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5757b0d640 10 monclient(hunting): authenticate will time out at 2098.019604s 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5755081640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f57500588a0 0x7f5750183d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41202/0 (socket says 10.20.193.102:41202) 2026-02-21T22:31:28.785 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.778+0000 7f5755081640 1 -- 10.20.193.102:0/657560094 learned_addr learned my addr 10.20.193.102:0/657560094 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5755081640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.102:6789/0 conn(0x7f57500588a0 legacy=0x7f5750183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5756083640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.102:6790/0 conn(0x7f57501435f0 legacy=0x7f57501a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899982108 0 0) 0x7f5744002c60 con 0x7f57500588a0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient(hunting): my global_id is 34776 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5738003620 con 0x7f57500588a0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5755882640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.178:6789/0 conn(0x7f57500584c0 legacy=0x7f5750142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2880691296 0 0) 0x7f5738003620 con 0x7f57500588a0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.102:6790/0 conn(0x7f57501435f0 legacy=0x7f57501a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.178:6789/0 conn(0x7f57500584c0 legacy=0x7f5750142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 monclient: found mon.noname-a 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5750056640 con 0x7f57500588a0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5744003360 con 0x7f57500588a0 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 5 monclient: authenticate success, global_id 34776 2026-02-21T22:31:28.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: dump: 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f57440049a0 con 0x7f57500588a0 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 4 set_mon_vals no callback set 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 1 -- 10.20.193.102:0/657560094 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5744004df0 con 0x7f57500588a0 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5754880640 10 monclient: dump: 2026-02-21T22:31:28.788 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: shutdown 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- 10.20.193.102:0/657560094 >> v1:10.20.193.102:6789/0 conn(0x7f57500588a0 legacy=0x7f5750183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- 10.20.193.102:0/657560094 shutdown_connections 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- 10.20.193.102:0/657560094 >> 10.20.193.102:0/657560094 conn(0x7f575005e700 msgr2=0x7f5750095ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- 10.20.193.102:0/657560094 shutdown_connections 2026-02-21T22:31:28.789 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- 10.20.193.102:0/657560094 wait complete. 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: monmap: 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 Processor -- start 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- start start 2026-02-21T22:31:28.792 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: init 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-b con 0x7f57500588a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-c con 0x7f57501435f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient(hunting): picked mon.noname-a con 0x7f5750130160 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5750131680 con 0x7f5750130160 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5750131960 con 0x7f57500588a0 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5750130af0 con 0x7f57501435f0 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.782+0000 7f5757b0d640 10 monclient(hunting): authenticate will time out at 2098.023979s 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5755081640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f57501435f0 0x7f575012e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45342/0 (socket says 10.20.193.102:45342) 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5755081640 1 -- 10.20.193.102:0/386759826 learned_addr learned my addr 10.20.193.102:0/386759826 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5756083640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.102:6789/0 conn(0x7f5750130160 legacy=0x7f5750195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5755081640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.102:6790/0 conn(0x7f57501435f0 legacy=0x7f575012e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5755882640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f57500588a0 legacy=0x7f575012db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765122083 0 0) 0x7f5750131680 con 0x7f5750130160 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): my global_id is 34779 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5720003620 con 0x7f5750130160 2026-02-21T22:31:28.793 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1497538688 0 0) 0x7f5750130af0 con 0x7f57501435f0 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): my global_id is 94277 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5750131680 con 0x7f57501435f0 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793671991 0 0) 0x7f5750131960 con 0x7f57500588a0 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): my global_id is 124120 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5750130af0 con 0x7f57500588a0 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1800931601 0 0) 0x7f5720003620 con 0x7f5750130160 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5750131960 con 0x7f5750130160 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f574c004890 con 0x7f5750130160 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): dump: 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.794 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3400452600 0 0) 0x7f5750131680 con 0x7f57501435f0 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5720003620 con 0x7f57501435f0 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f57440031b0 con 0x7f57501435f0 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): dump: 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1145084728 0 0) 0x7f5750130af0 con 0x7f57500588a0 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5750131680 con 0x7f57500588a0 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5740003340 con 0x7f57500588a0 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient(hunting): dump: 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4085622589 0 0) 0x7f5750131960 con 0x7f5750130160 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.102:6790/0 conn(0x7f57501435f0 legacy=0x7f575012e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f57500588a0 legacy=0x7f575012db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 monclient: found mon.a 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575012f600 con 0x7f5750130160 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 5 monclient: authenticate success, global_id 34779 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 10 monclient: _renew_subs 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f57501310c0 con 0x7f5750130160 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 10 monclient: _renew_subs 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f575012eb30 con 0x7f5750130160 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f574c004ed0 con 0x7f5750130160 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f574c005260 con 0x7f5750130160 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 10 monclient: dump: 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.797 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573ffff640 4 set_mon_vals no callback set 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f57500008e0 con 0x7f57500584c0 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f574c005440 con 0x7f5750130160 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.786+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(768..768 src has 527..768) ==== 3871+0+0 (unknown 462652577 0 0) 0x7f574c04da30 con 0x7f5750130160 2026-02-21T22:31:28.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f5755081640 10 monclient: get_auth_request con 0x7f572003d900 auth_method 0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f5755882640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f57500584c0 legacy=0x7f575008bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417512865 0 0) 0x7f57500008e0 con 0x7f57500584c0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient(hunting): my global_id is 124123 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5750131960 con 0x7f57500584c0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2040477001 0 0) 0x7f5750131960 con 0x7f57500584c0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f57500008e0 con 0x7f57500584c0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5740003ab0 con 0x7f57500584c0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:28.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient: dump: 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1206394629 0 0) 0x7f57500008e0 con 0x7f57500584c0 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5750054480 con 0x7f57500584c0 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f57400058c0 con 0x7f57500584c0 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f57500584c0 legacy=0x7f575008bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.790+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f57500584c0 legacy=0x7f575008bfc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.794+0000 7f5757b0d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:28.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.794+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5750054630 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5755882640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f5750094c10 legacy=0x7f575004d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196727972 0 0) 0x7f5750054630 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient(hunting): my global_id is 124126 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57500008e0 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4082314373 0 0) 0x7f57500008e0 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5750054630 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5740004250 con 0x7f5750094c10 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:28.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient: dump: 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 837208915 0 0) 0x7f5750054630 con 0x7f5750094c10 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f575004d990 con 0x7f5750094c10 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 784166823) 0x7f5740005b40 con 0x7f5750094c10 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:28.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f5750094c10 legacy=0x7f575004d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f573e7fc640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6789/0 conn(0x7f5750094c10 legacy=0x7f575004d280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 15439, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.803 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.804 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:28.805 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.806 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:28.807 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:28.808 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:28.809 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.178:6800/2549949439 conn(0x7f572003d900 legacy=0x7f572005ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 10 monclient: shutdown 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 >> v1:10.20.193.102:6789/0 conn(0x7f5750130160 legacy=0x7f5750195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5756083640 1 -- 10.20.193.102:0/386759826 reap_dead start 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5756083640 1 -- 10.20.193.102:0/386759826 reap_dead start 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 shutdown_connections 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 >> 10.20.193.102:0/386759826 conn(0x7f575005e700 msgr2=0x7f5750092f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 shutdown_connections 2026-02-21T22:31:28.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.798+0000 7f5757b0d640 1 -- 10.20.193.102:0/386759826 wait complete. 2026-02-21T22:31:28.811 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:28.811 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:31:28.811 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: get_monmap_and_config 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: monmap: 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:28.871041+0000 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:28.871041+0000 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.870 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 1 Processor -- start 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 1 -- start start 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: init 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:31:28.871 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-c con 0x7f84b0084860 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-b con 0x7f84b0086e90 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-a con 0x7f84b0183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b0086a70 con 0x7f84b0183350 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b008b130 con 0x7f84b0084860 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b0085340 con 0x7f84b0086e90 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84b5f7d640 10 monclient(hunting): authenticate will time out at 2098.107213s 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84aeffd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f84b0086e90 0x7f84b0181c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45346/0 (socket says 10.20.193.102:45346) 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84aeffd640 1 -- 10.20.193.102:0/3780954054 learned_addr learned my addr 10.20.193.102:0/3780954054 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.866+0000 7f84affff640 1 --1- 10.20.193.102:0/3780954054 >> v1:10.20.193.102:6789/0 conn(0x7f84b0183350 0x7f84b01a3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41224/0 (socket says 10.20.193.102:41224) 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84aeffd640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.102:6790/0 conn(0x7f84b0086e90 legacy=0x7f84b0181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84affff640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.102:6789/0 conn(0x7f84b0183350 legacy=0x7f84b01a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574984217 0 0) 0x7f84b0085340 con 0x7f84b0086e90 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): my global_id is 94280 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8494003620 con 0x7f84b0086e90 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095579618 0 0) 0x7f84b0086a70 con 0x7f84b0183350 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient(hunting): my global_id is 34782 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b0085340 con 0x7f84b0183350 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84af7fe640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.178:6789/0 conn(0x7f84b0084860 legacy=0x7f84b0084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301006934 0 0) 0x7f8494003620 con 0x7f84b0086e90 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.178:6789/0 conn(0x7f84b0084860 legacy=0x7f84b0084c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.102:6789/0 conn(0x7f84b0183350 legacy=0x7f84b01a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 monclient: found mon.noname-b 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b0086790 con 0x7f84b0086e90 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f84a0003100 con 0x7f84b0086e90 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: dump: 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.875 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 5 monclient: authenticate success, global_id 94280 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f84a00033d0 con 0x7f84b0086e90 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 1 -- 10.20.193.102:0/3780954054 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f84a0004b50 con 0x7f84b0086e90 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84ae7fc640 10 monclient: dump: 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 4 set_mon_vals no callback set 2026-02-21T22:31:28.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: shutdown 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- 10.20.193.102:0/3780954054 >> v1:10.20.193.102:6790/0 conn(0x7f84b0086e90 legacy=0x7f84b0181c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- 10.20.193.102:0/3780954054 shutdown_connections 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- 10.20.193.102:0/3780954054 >> 10.20.193.102:0/3780954054 conn(0x7f84b0091a50 msgr2=0x7f84b0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- 10.20.193.102:0/3780954054 shutdown_connections 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- 10.20.193.102:0/3780954054 wait complete. 2026-02-21T22:31:28.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: build_initial_monmap 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: monmap: 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 Processor -- start 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- start start 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: init 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-c con 0x7f84b0084860 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-b con 0x7f84b0086e90 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 10 monclient(hunting): picked mon.noname-a con 0x7f84b0183350 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b0087890 con 0x7f84b0183350 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.870+0000 7f84b5f7d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b0088a70 con 0x7f84b0086e90 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b007ea80 con 0x7f84b0084860 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient(hunting): authenticate will time out at 2098.112016s 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84af7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f84b0084860 0x7f84b01320a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45348/0 (socket says 10.20.193.102:45348) 2026-02-21T22:31:28.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84af7fe640 1 -- 10.20.193.102:0/1858923094 learned_addr learned my addr 10.20.193.102:0/1858923094 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84af7fe640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.102:6790/0 conn(0x7f84b0084860 legacy=0x7f84b01320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84affff640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.102:6789/0 conn(0x7f84b0183350 legacy=0x7f84b0174450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84aeffd640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.178:6789/0 conn(0x7f84b0086e90 legacy=0x7f84b0172d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323487327 0 0) 0x7f84b007ea80 con 0x7f84b0084860 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): my global_id is 94283 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8480003620 con 0x7f84b0084860 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956520683 0 0) 0x7f84b0087890 con 0x7f84b0183350 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): my global_id is 34785 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b007ea80 con 0x7f84b0183350 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175597070 0 0) 0x7f84b0088a70 con 0x7f84b0086e90 2026-02-21T22:31:28.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): my global_id is 124132 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b0087890 con 0x7f84b0086e90 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2733724284 0 0) 0x7f8480003620 con 0x7f84b0084860 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f84b0088a70 con 0x7f84b0084860 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f849c003420 con 0x7f84b0084860 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): dump: 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1279295963 0 0) 0x7f84b007ea80 con 0x7f84b0183350 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8480003620 con 0x7f84b0183350 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f84a40048e0 con 0x7f84b0183350 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): dump: 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4224209344 0 0) 0x7f84b0087890 con 0x7f84b0086e90 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f84b007ea80 con 0x7f84b0086e90 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f84a0004a90 con 0x7f84b0086e90 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient(hunting): dump: 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.882 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 988815614 0 0) 0x7f84b0088a70 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.178:6789/0 conn(0x7f84b0086e90 legacy=0x7f84b0172d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.102:6789/0 conn(0x7f84b0183350 legacy=0x7f84b0174450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 monclient: found mon.c 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b00842e0 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: _finish_auth 0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 5 monclient: authenticate success, global_id 94283 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _renew_subs 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f84b0080b00 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _renew_subs 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f84b007f5e0 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f849c0039c0 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f849c003d50 con 0x7f84b0084860 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 10 monclient: dump: 2026-02-21T22:31:28.883 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84adffb640 4 set_mon_vals no callback set 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b0062ce0 con 0x7f84b0084860 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.874+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f849c003fe0 con 0x7f84b0084860 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.878+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(768..768 src has 527..768) ==== 3871+0+0 (unknown 462652577 0 0) 0x7f849c04c6d0 con 0x7f84b0084860 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.878+0000 7f84aeffd640 10 monclient: get_auth_request con 0x7f848003d9a0 auth_method 0 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.878+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f849c0176b0 con 0x7f84b0084860 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.878+0000 7f848bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:28.884 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.878+0000 7f848bfff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:28.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:31:28.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:31:28.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:28.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f84b0132870 con 0x7f84b0084860 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=13,kv=12,logm=53} crc {auth=3745474519,config=1618278320,health=2279815780,kv=514132712,logm=66154420}) 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=20,config=2,health=13,kv=12,logm=53} crc {auth=1420688697,config=1618278320,health=2279815780,kv=514132712,logm=2995864597}) 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=13,kv=12,logm=53} crc {auth=3745474519,config=1618278320,health=2279815780,kv=514132712,logm=66154420}) 2026-02-21T22:31:28.998 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=2,health=12,kv=12,logm=54} crc {auth=1031063381,config=1618278320,health=3897484431,kv=514132712,logm=407537258}) 2026-02-21T22:31:28.999 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f848bfff640 1 -- 10.20.193.102:0/1858923094 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f849c016f90 con 0x7f84b0084860 2026-02-21T22:31:28.999 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f848bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:31:28.999 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f848bfff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:28.999 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.000 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346674465}) 2026-02-21T22:31:29.000 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=484528155}) 2026-02-21T22:31:29.000 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.000 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346674465}) 2026-02-21T22:31:29.000 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1387991046}) 2026-02-21T22:31:29.000 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.178:6800/2549949439 conn(0x7f848003d9a0 legacy=0x7f848005de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 10 monclient: shutdown 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 >> v1:10.20.193.102:6790/0 conn(0x7f84b0084860 legacy=0x7f84b01320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 shutdown_connections 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 >> 10.20.193.102:0/1858923094 conn(0x7f84b0091a50 msgr2=0x7f84b008cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 shutdown_connections 2026-02-21T22:31:29.001 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:28.994+0000 7f84b5f7d640 1 -- 10.20.193.102:0/1858923094 wait complete. 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292078330}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=806284841}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3292078330}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.994+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2368838722}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330627944}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1268407274}) 2026-02-21T22:31:29.003 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2330627944}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=939677922}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1987779759}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2174301526}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1987779759}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2888614550}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3305130926}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4082228692}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3305130926}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2396298428}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2525048378}) 2026-02-21T22:31:29.004 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2995585020}) 2026-02-21T22:31:29.005 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.005 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2525048378}) 2026-02-21T22:31:29.005 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1945031953}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4154495715}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3374686171}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4154495715}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:28.998+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3543427881}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=251248959}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2855320921}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=251248959}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=555212833}) 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.007 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2409854067}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3455583704}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2409854067}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=153506424}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=315377341}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=893827590}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=315377341}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2318255189}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254983693}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=562405974}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4254983693}) 2026-02-21T22:31:29.008 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=753934098}) 2026-02-21T22:31:29.010 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.010 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3252576702}) 2026-02-21T22:31:29.010 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1495719832}) 2026-02-21T22:31:29.010 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.010 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3252576702}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.002+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3360818342}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3774068392}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3622056461}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3774068392}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1291384529}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=809906698}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2958198939}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=809906698}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2069614861}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718590604}) 2026-02-21T22:31:29.011 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1303423195}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3718590604}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1657143516}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366853078}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=101702669}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366853078}) 2026-02-21T22:31:29.012 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4039884608}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1977767520}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=393828924}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1977767520}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3587192587}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1201596042}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3265618616}) 2026-02-21T22:31:29.013 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.014 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1201596042}) 2026-02-21T22:31:29.014 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.006+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2682090814}) 2026-02-21T22:31:29.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:31:29.017 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.017 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917644943}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1802278323}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917644943}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=785192175}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809396945}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=41502371}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3809396945}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3904434463}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3704614151}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3059649187}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3704614151}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=437969780}) 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.018 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1167320321}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2994790952}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1167320321}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=364301905}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3346605049}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3454458353}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3346605049}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.010+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2558722015}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065465248}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=215618374}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4065465248}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=827044591}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=198308489}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1032205827}) 2026-02-21T22:31:29.019 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=198308489}) 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3615559456}) 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1856196998}) 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3743027094}) 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1856196998}) 2026-02-21T22:31:29.020 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2342303890}) 2026-02-21T22:31:29.021 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2188605565}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3189904570}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2188605565}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2807042401}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261269504}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2926857508}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1261269504}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3081058683}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761403450}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1417736851}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2761403450}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.014+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=124739233}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=642687385}) 2026-02-21T22:31:29.022 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=241754059}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=642687385}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3430508697}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130893030}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=323233191}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130893030}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1271667836}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380906742}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3107899234}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=380906742}) 2026-02-21T22:31:29.023 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3115741850}) 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1405898178}) 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=663730402}) 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1405898178}) 2026-02-21T22:31:29.024 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1038361365}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=8} crc {logm=2626025547,mdsmap=3074481156,mgr=2421504707,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2388458575,mon_config_key=2125654824,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=1544990792,osd_snap=374658029}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=49,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=9} crc {logm=4179317374,mdsmap=3074481157,mgr=100579201,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2388458575,mon_config_key=1037519540,monmap=918322475,nvmeofgw=935146538,osd_metadata=3232105792,osd_pg_creating=1544990792,osd_snap=2025933991}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=50,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=6,osd_pg_creating=1,osd_snap=8} crc {logm=2626025547,mdsmap=3074481156,mgr=2421504707,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2388458575,mon_config_key=2125654824,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_pg_creating=1544990792,osd_snap=374658029}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.018+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=9,monmap=3,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=6} crc {logm=4061127394,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=2388458575,mon_config_key=3320192658,monmap=918322475,nvmeofgw=3360707892,osd_metadata=1493522314,osd_pg_creating=1544990792,osd_snap=94580219}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1274523980}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=4274289130}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=1274523980}) 2026-02-21T22:31:29.025 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=700835330}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=40,osdmap=60} crc {osd_snap=3517393381,osdmap=2794099271}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=4179964008,osdmap=704303923}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=40,osdmap=60} crc {osd_snap=3517393381,osdmap=2794099271}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=3761730275,osdmap=377442903}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=594760645}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1866825602}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=594760645}) 2026-02-21T22:31:29.026 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3158101617}) 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3864733328}) 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=689829945}) 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3864733328}) 2026-02-21T22:31:29.027 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3628536916}) 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3361651393}) 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2642959768}) 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.022+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3361651393}) 2026-02-21T22:31:29.028 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:31:29.026+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3339805123}) 2026-02-21T22:31:29.439 INFO:tasks.rados.rados.0.trial178.stdout:3694: done (0 left) 2026-02-21T22:31:29.440 INFO:tasks.rados.rados.0.trial178.stdout:3695: snap_create 2026-02-21T22:31:30.015 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:31:30.016 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.070+0000 7f0bc62c0640 10 monclient: get_monmap_and_config 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.070+0000 7f0bc62c0640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.070+0000 7f0bc62c0640 10 monclient: monmap: 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.076 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:30.076293+0000 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:30.076293+0000 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.070+0000 7f0bc62c0640 1 Processor -- start 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- start start 2026-02-21T22:31:30.077 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient: init 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc008d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.082 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0086a10 con 0x7f0bc008ce00 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc008b130 con 0x7f0bc008b7f0 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc008bd60 con 0x7f0bc008d8e0 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient(hunting): authenticate will time out at 2099.312467s 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbffff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0bc008ce00 0x7f0bc008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41244/0 (socket says 10.20.193.102:41244) 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbffff640 1 -- 10.20.193.102:0/966064734 learned_addr learned my addr 10.20.193.102:0/966064734 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbffff640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.102:6789/0 conn(0x7f0bc008ce00 legacy=0x7f0bc008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc4836640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.102:6790/0 conn(0x7f0bc008d8e0 legacy=0x7f0bc01a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233090905 0 0) 0x7f0bc0086a10 con 0x7f0bc008ce00 2026-02-21T22:31:30.083 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): my global_id is 34788 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ba4003620 con 0x7f0bc008ce00 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492646743 0 0) 0x7f0bc008bd60 con 0x7f0bc008d8e0 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient(hunting): my global_id is 94286 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc0086a10 con 0x7f0bc008d8e0 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbf7fe640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.178:6789/0 conn(0x7f0bc008b7f0 legacy=0x7f0bc0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.084 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1694869881 0 0) 0x7f0bc0086a10 con 0x7f0bc008d8e0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.178:6789/0 conn(0x7f0bc008b7f0 legacy=0x7f0bc0181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.102:6789/0 conn(0x7f0bc008ce00 legacy=0x7f0bc008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 monclient: found mon.noname-b 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc008b370 con 0x7f0bc008d8e0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0bb4004670 con 0x7f0bc008d8e0 2026-02-21T22:31:30.085 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: dump: 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.086 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 5 monclient: authenticate success, global_id 94286 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0bb4004940 con 0x7f0bc008d8e0 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 1 -- 10.20.193.102:0/966064734 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0bb4004dd0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bbeffd640 10 monclient: dump: 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.087 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 4 set_mon_vals no callback set 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 10 monclient: shutdown 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- 10.20.193.102:0/966064734 >> v1:10.20.193.102:6790/0 conn(0x7f0bc008d8e0 legacy=0x7f0bc01a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.088 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- 10.20.193.102:0/966064734 shutdown_connections 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- 10.20.193.102:0/966064734 >> 10.20.193.102:0/966064734 conn(0x7f0bc0091a50 msgr2=0x7f0bc0091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- 10.20.193.102:0/966064734 shutdown_connections 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.074+0000 7f0bc62c0640 1 -- 10.20.193.102:0/966064734 wait complete. 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: monmap: 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.089 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 Processor -- start 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- start start 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: init 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-c con 0x7f0bc008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-b con 0x7f0bc008ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc00832f0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0080f90 con 0x7f0bc008ce00 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc0082a50 con 0x7f0bc008b7f0 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient(hunting): authenticate will time out at 2099.317056s 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f0bc008b7f0 0x7f0bc01531e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45356/0 (socket says 10.20.193.102:45356) 2026-02-21T22:31:30.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbffff640 1 -- 10.20.193.102:0/3854477978 learned_addr learned my addr 10.20.193.102:0/3854477978 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbffff640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.102:6790/0 conn(0x7f0bc008b7f0 legacy=0x7f0bc01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc4836640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.102:6789/0 conn(0x7f0bc008d8e0 legacy=0x7f0bc01953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497862503 0 0) 0x7f0bc0082a50 con 0x7f0bc008b7f0 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): my global_id is 94289 2026-02-21T22:31:30.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbf7fe640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.178:6789/0 conn(0x7f0bc008ce00 legacy=0x7f0bc0084800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b90003620 con 0x7f0bc008b7f0 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858465150 0 0) 0x7f0bc00832f0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): my global_id is 34791 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc0082a50 con 0x7f0bc008d8e0 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3268306189 0 0) 0x7f0bc0080f90 con 0x7f0bc008ce00 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): my global_id is 124138 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc00832f0 con 0x7f0bc008ce00 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 41246839 0 0) 0x7f0bc0082a50 con 0x7f0bc008d8e0 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0bc0080f90 con 0x7f0bc008d8e0 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0bb4004c20 con 0x7f0bc008d8e0 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): dump: 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.097 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4050004504 0 0) 0x7f0b90003620 con 0x7f0bc008b7f0 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0bc0082a50 con 0x7f0bc008b7f0 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0bac0048e0 con 0x7f0bc008b7f0 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient(hunting): dump: 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.099 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3214626484 0 0) 0x7f0bc0080f90 con 0x7f0bc008d8e0 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.102:6790/0 conn(0x7f0bc008b7f0 legacy=0x7f0bc01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.178:6789/0 conn(0x7f0bc008ce00 legacy=0x7f0bc0084800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 monclient: found mon.a 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc0081af0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bbcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 5 monclient: authenticate success, global_id 34791 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _renew_subs 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc0084f10 con 0x7f0bc008d8e0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _renew_subs 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.078+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0bc0196c90 con 0x7f0bc008d8e0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0bb4004910 con 0x7f0bc008d8e0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0bb40054e0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.101 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbe7fc640 4 set_mon_vals no callback set 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: dump: 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.102 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bc62c0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bc62c0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bc62c0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0085300 con 0x7f0bc008d8e0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0bb4005780 con 0x7f0bc008d8e0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(769..769 src has 527..769) ==== 3927+0+0 (unknown 2804465678 0 0) 0x7f0bb404d9f0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbffff640 10 monclient: get_auth_request con 0x7f0b9003d850 auth_method 0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0bb40189f0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.082+0000 7f0bbcff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:30.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.194+0000 7f0bc62c0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:31:30.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.194+0000 7f0bc62c0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:30.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.194+0000 7f0bc62c0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.194+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0bc0062ce0 con 0x7f0bc008d8e0 2026-02-21T22:31:30.201 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bbcff9640 1 -- 10.20.193.102:0/3854477978 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2418404165) 0x7f0bb401dc50 con 0x7f0bc008d8e0 2026-02-21T22:31:30.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bbcff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:30.202 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bbcff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:30.202 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:31:30.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0b9003d850 legacy=0x7f0b9005dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 10 monclient: shutdown 2026-02-21T22:31:30.204 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 >> v1:10.20.193.102:6789/0 conn(0x7f0bc008d8e0 legacy=0x7f0bc01953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 shutdown_connections 2026-02-21T22:31:30.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 >> 10.20.193.102:0/3854477978 conn(0x7f0bc0091a50 msgr2=0x7f0bc005dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 shutdown_connections 2026-02-21T22:31:30.205 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.198+0000 7f0bc62c0640 1 -- 10.20.193.102:0/3854477978 wait complete. 2026-02-21T22:31:30.210 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:31:30.210 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2026-02-21T22:31:30.211 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:31:30.211 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:31:30.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.266+0000 7f2e91d74640 10 monclient: get_monmap_and_config 2026-02-21T22:31:30.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.266+0000 7f2e91d74640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.266+0000 7f2e91d74640 10 monclient: monmap: 2026-02-21T22:31:30.272 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:30.272488+0000 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:30.272488+0000 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 Processor -- start 2026-02-21T22:31:30.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- start start 2026-02-21T22:31:30.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient: init 2026-02-21T22:31:30.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:30.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-b con 0x7f2e8c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-a con 0x7f2e8c08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-c con 0x7f2e8c08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c086a10 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c08b130 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c08bd60 con 0x7f2e8c08ce00 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient(hunting): authenticate will time out at 2099.508596s 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8affd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08b7f0 0x7f2e8c181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41260/0 (socket says 10.20.193.102:41260) 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8affd640 1 -- 10.20.193.102:0/3349719895 learned_addr learned my addr 10.20.193.102:0/3349719895 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8affd640 1 -- 10.20.193.102:0/3349719895 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08b7f0 legacy=0x7f2e8c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8b7fe640 1 -- 10.20.193.102:0/3349719895 >> v1:10.20.193.102:6790/0 conn(0x7f2e8c08ce00 legacy=0x7f2e8c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039443407 0 0) 0x7f2e8c086a10 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): my global_id is 34794 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e70003620 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203562801 0 0) 0x7f2e8c08bd60 con 0x7f2e8c08ce00 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient(hunting): my global_id is 94292 2026-02-21T22:31:30.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c086a10 con 0x7f2e8c08ce00 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2348398177 0 0) 0x7f2e8c086a10 con 0x7f2e8c08ce00 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 >> v1:10.20.193.178:6789/0 conn(0x7f2e8c08d8e0 legacy=0x7f2e8c1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08b7f0 legacy=0x7f2e8c181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 monclient: found mon.noname-b 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c08b370 con 0x7f2e8c08ce00 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e780030e0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: dump: 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.282 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 5 monclient: authenticate success, global_id 94292 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2e780033b0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 1 -- 10.20.193.102:0/3349719895 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e780038c0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e8a7fc640 10 monclient: dump: 2026-02-21T22:31:30.283 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:30.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 4 set_mon_vals no callback set 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 10 monclient: shutdown 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- 10.20.193.102:0/3349719895 >> v1:10.20.193.102:6790/0 conn(0x7f2e8c08ce00 legacy=0x7f2e8c08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- 10.20.193.102:0/3349719895 shutdown_connections 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- 10.20.193.102:0/3349719895 >> 10.20.193.102:0/3349719895 conn(0x7f2e8c091a50 msgr2=0x7f2e8c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- 10.20.193.102:0/3349719895 shutdown_connections 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.270+0000 7f2e91d74640 1 -- 10.20.193.102:0/3349719895 wait complete. 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient: monmap: 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.285 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 1 Processor -- start 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 1 -- start start 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient: init 2026-02-21T22:31:30.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-b con 0x7f2e8c08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-c con 0x7f2e8c08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient(hunting): picked mon.noname-a con 0x7f2e8c08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c080e50 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c058c90 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e8c05d8d0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e91d74640 10 monclient(hunting): authenticate will time out at 2099.513143s 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e8affd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f2e8c08ce00 0x7f2e8c192290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45372/0 (socket says 10.20.193.102:45372) 2026-02-21T22:31:30.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e8affd640 1 -- 10.20.193.102:0/1078394526 learned_addr learned my addr 10.20.193.102:0/1078394526 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e8affd640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6790/0 conn(0x7f2e8c08ce00 legacy=0x7f2e8c192290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e8bfff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08d8e0 legacy=0x7f2e8c1939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e8b7fe640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.178:6789/0 conn(0x7f2e8c08b7f0 legacy=0x7f2e8c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187171068 0 0) 0x7f2e8c05d8d0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.294 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): my global_id is 94295 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e5c003620 con 0x7f2e8c08ce00 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991025487 0 0) 0x7f2e8c080e50 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): my global_id is 34797 2026-02-21T22:31:30.295 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c05d8d0 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740625629 0 0) 0x7f2e8c058c90 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): my global_id is 124141 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c080e50 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3162256567 0 0) 0x7f2e5c003620 con 0x7f2e8c08ce00 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e8c058c90 con 0x7f2e8c08ce00 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e7c0048e0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.296 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): dump: 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.297 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1566266924 0 0) 0x7f2e8c05d8d0 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e5c003620 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e800057c0 con 0x7f2e8c08d8e0 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): dump: 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.298 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2058731056 0 0) 0x7f2e8c080e50 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e8c05d8d0 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e78003710 con 0x7f2e8c08b7f0 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient(hunting): dump: 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.299 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 934857761 0 0) 0x7f2e8c058c90 con 0x7f2e8c08ce00 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.178:6789/0 conn(0x7f2e8c08b7f0 legacy=0x7f2e8c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08d8e0 legacy=0x7f2e8c1939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 1 monclient: found mon.c 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.274+0000 7f2e67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c088a70 con 0x7f2e8c08ce00 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 5 monclient: authenticate success, global_id 94295 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 10 monclient: _renew_subs 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c080b70 con 0x7f2e8c08ce00 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 10 monclient: _renew_subs 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2e8c07e040 con 0x7f2e8c08ce00 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2e7c004d30 con 0x7f2e8c08ce00 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e7c0050c0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: dump: 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.302 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e89ffb640 4 set_mon_vals no callback set 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2e8c07e380 con 0x7f2e8c151910 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2e7c005350 con 0x7f2e8c08ce00 2026-02-21T22:31:30.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(769..769 src has 527..769) ==== 3927+0+0 (unknown 2804465678 0 0) 0x7f2e7c04c7c0 con 0x7f2e8c08ce00 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e8bfff640 10 monclient: get_auth_request con 0x7f2e5c03d910 auth_method 0 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e8b7fe640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c151910 legacy=0x7f2e8c05c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895042336 0 0) 0x7f2e8c07e380 con 0x7f2e8c151910 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient(hunting): my global_id is 34800 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c058c90 con 0x7f2e8c151910 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1116008180 0 0) 0x7f2e8c058c90 con 0x7f2e8c151910 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e8c07e380 con 0x7f2e8c151910 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e78003de0 con 0x7f2e8c151910 2026-02-21T22:31:30.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: dump: 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3741244801 0 0) 0x7f2e8c07e380 con 0x7f2e8c151910 2026-02-21T22:31:30.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e8c08ada0 con 0x7f2e8c151910 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2e780058c0 con 0x7f2e8c151910 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c151910 legacy=0x7f2e8c05c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.278+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c151910 legacy=0x7f2e8c05c860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.282+0000 7f2e91d74640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.282+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2e8c062ce0 con 0x7f2e8c08a550 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e8b7fe640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08a550 legacy=0x7f2e8c05b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465136232 0 0) 0x7f2e8c062ce0 con 0x7f2e8c08a550 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient(hunting): my global_id is 34803 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c07e380 con 0x7f2e8c08a550 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1629593479 0 0) 0x7f2e8c07e380 con 0x7f2e8c08a550 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e8c062ce0 con 0x7f2e8c08a550 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2e78004200 con 0x7f2e8c08a550 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient: dump: 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.307 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2756596426 0 0) 0x7f2e8c062ce0 con 0x7f2e8c08a550 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2e8c151ce0 con 0x7f2e8c08a550 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4683 (unknown 0 0 3631116804) 0x7f2e78004570 con 0x7f2e8c08a550 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08a550 legacy=0x7f2e8c05b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.308 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e67fff640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6789/0 conn(0x7f2e8c08a550 legacy=0x7f2e8c05b6e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 839155, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 1, 2026-02-21T22:31:30.309 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.310 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.311 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:30.312 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.313 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.314 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.315 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.316 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.317 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.318 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:30.319 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.178:6800/2549949439 conn(0x7f2e5c03d910 legacy=0x7f2e5c05ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 10 monclient: shutdown 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 >> v1:10.20.193.102:6790/0 conn(0x7f2e8c08ce00 legacy=0x7f2e8c192290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e8bfff640 1 -- 10.20.193.102:0/1078394526 reap_dead start 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e8bfff640 1 -- 10.20.193.102:0/1078394526 reap_dead start 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 shutdown_connections 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 >> 10.20.193.102:0/1078394526 conn(0x7f2e8c091a50 msgr2=0x7f2e8c059fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 shutdown_connections 2026-02-21T22:31:30.320 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.286+0000 7f2e91d74640 1 -- 10.20.193.102:0/1078394526 wait complete. 2026-02-21T22:31:30.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:30.321 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:31:30.321 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: get_monmap_and_config 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: monmap: 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:30.383086+0000 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:30.383086+0000 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:30.382 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 1 Processor -- start 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 1 -- start start 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: init 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-b con 0x7f8e5808cd00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-a con 0x7f8e5808d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-c con 0x7f8e5808b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e580890e0 con 0x7f8e5808d7e0 2026-02-21T22:31:30.383 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e58088ea0 con 0x7f8e5808b8d0 2026-02-21T22:31:30.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5808aea0 con 0x7f8e5808cd00 2026-02-21T22:31:30.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e5dacb640 10 monclient(hunting): authenticate will time out at 2099.618978s 2026-02-21T22:31:30.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e56ffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f8e5808d7e0 0x7f8e581840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41298/0 (socket says 10.20.193.102:41298) 2026-02-21T22:31:30.384 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.378+0000 7f8e56ffd640 1 -- 10.20.193.102:0/3992781155 learned_addr learned my addr 10.20.193.102:0/3992781155 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e56ffd640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.102:6789/0 conn(0x7f8e5808d7e0 legacy=0x7f8e581840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e577fe640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.102:6790/0 conn(0x7f8e5808cd00 legacy=0x7f8e5808d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e57fff640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.178:6789/0 conn(0x7f8e5808b8d0 legacy=0x7f8e581a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238298489 0 0) 0x7f8e580890e0 con 0x7f8e5808d7e0 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): my global_id is 34806 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e3c003620 con 0x7f8e5808d7e0 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945663242 0 0) 0x7f8e5808aea0 con 0x7f8e5808cd00 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.386 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): my global_id is 94298 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e580890e0 con 0x7f8e5808cd00 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127535123 0 0) 0x7f8e58088ea0 con 0x7f8e5808b8d0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient(hunting): my global_id is 124144 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e5808aea0 con 0x7f8e5808b8d0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3662393211 0 0) 0x7f8e3c003620 con 0x7f8e5808d7e0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.102:6790/0 conn(0x7f8e5808cd00 legacy=0x7f8e5808d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.178:6789/0 conn(0x7f8e5808b8d0 legacy=0x7f8e581a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 monclient: found mon.noname-a 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5808a600 con 0x7f8e5808d7e0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e48003110 con 0x7f8e5808d7e0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: dump: 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.387 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 5 monclient: authenticate success, global_id 34806 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e480033e0 con 0x7f8e5808d7e0 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 1 -- 10.20.193.102:0/3992781155 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e48004b50 con 0x7f8e5808d7e0 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e567fc640 10 monclient: dump: 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:30.388 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 4 set_mon_vals no callback set 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: shutdown 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3992781155 >> v1:10.20.193.102:6789/0 conn(0x7f8e5808d7e0 legacy=0x7f8e581840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3992781155 shutdown_connections 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3992781155 >> 10.20.193.102:0/3992781155 conn(0x7f8e58091a50 msgr2=0x7f8e58091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3992781155 shutdown_connections 2026-02-21T22:31:30.389 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3992781155 wait complete. 2026-02-21T22:31:30.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.390 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: monmap: 2026-02-21T22:31:30.390 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.390 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:30.390 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 Processor -- start 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- start start 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: init 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-a con 0x7f8e5808b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-b con 0x7f8e5808cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient(hunting): picked mon.noname-c con 0x7f8e5808d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e58082200 con 0x7f8e5808b8d0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e58081f20 con 0x7f8e5808cd00 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e58080a00 con 0x7f8e5808d7e0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e577fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f8e5808b8d0 0x7f8e58083730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41310/0 (socket says 10.20.193.102:41310) 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e577fe640 1 -- 10.20.193.102:0/3841407263 learned_addr learned my addr 10.20.193.102:0/3841407263 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.382+0000 7f8e5dacb640 10 monclient(hunting): authenticate will time out at 2099.623623s 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e577fe640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6789/0 conn(0x7f8e5808b8d0 legacy=0x7f8e58083730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e57fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5808d7e0 legacy=0x7f8e58195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063379532 0 0) 0x7f8e58080a00 con 0x7f8e5808d7e0 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.391 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): my global_id is 94301 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e28003620 con 0x7f8e5808d7e0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982543908 0 0) 0x7f8e58082200 con 0x7f8e5808b8d0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): my global_id is 34809 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e58080a00 con 0x7f8e5808b8d0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e56ffd640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.178:6789/0 conn(0x7f8e5808cd00 legacy=0x7f8e58194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2120146441 0 0) 0x7f8e28003620 con 0x7f8e5808d7e0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e58082200 con 0x7f8e5808d7e0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e4c004830 con 0x7f8e5808d7e0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): dump: 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 465517635 0 0) 0x7f8e58080a00 con 0x7f8e5808b8d0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e28003620 con 0x7f8e5808b8d0 2026-02-21T22:31:30.392 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e44003450 con 0x7f8e5808b8d0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient(hunting): dump: 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2057269236 0 0) 0x7f8e58082200 con 0x7f8e5808d7e0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.178:6789/0 conn(0x7f8e5808cd00 legacy=0x7f8e58194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6789/0 conn(0x7f8e5808b8d0 legacy=0x7f8e58083730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 monclient: found mon.c 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e58083e40 con 0x7f8e5808d7e0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 5 monclient: authenticate success, global_id 94301 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 10 monclient: _renew_subs 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581532f0 con 0x7f8e5808d7e0 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 10 monclient: _renew_subs 2026-02-21T22:31:30.393 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f8e58197490 con 0x7f8e5808d7e0 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e4c004a10 con 0x7f8e5808d7e0 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e4c005080 con 0x7f8e5808d7e0 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 10 monclient: dump: 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e55ffb640 4 set_mon_vals no callback set 2026-02-21T22:31:30.394 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e580889f0 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8e4c005380 con 0x7f8e5808d7e0 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(769..769 src has 527..769) ==== 3927+0+0 (unknown 2804465678 0 0) 0x7f8e4c04da30 con 0x7f8e5808d7e0 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.386+0000 7f8e56ffd640 10 monclient: get_auth_request con 0x7f8e2803d690 auth_method 0 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e577fe640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807a370 legacy=0x7f8e5807a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805148655 0 0) 0x7f8e580889f0 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient(hunting): my global_id is 94304 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e58082200 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 525232720 0 0) 0x7f8e58082200 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e580889f0 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e44003b70 con 0x7f8e5807a370 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient: dump: 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.395 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4060713966 0 0) 0x7f8e580889f0 con 0x7f8e5807a370 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e58088580 con 0x7f8e5807a370 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f8e440058c0 con 0x7f8e5807a370 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807a370 legacy=0x7f8e5807a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.396 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.390+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807a370 legacy=0x7f8e5807a740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e5dacb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:30.397 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e58062ce0 con 0x7f8e5807ae50 2026-02-21T22:31:30.399 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e577fe640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807ae50 legacy=0x7f8e58198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274634018 0 0) 0x7f8e58062ce0 con 0x7f8e5807ae50 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 10 monclient(hunting): my global_id is 94307 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.394+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e580889f0 con 0x7f8e5807ae50 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3386888368 0 0) 0x7f8e580889f0 con 0x7f8e5807ae50 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e58062ce0 con 0x7f8e5807ae50 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8e44004280 con 0x7f8e5807ae50 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 10 monclient: dump: 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.400 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 294695505 0 0) 0x7f8e58062ce0 con 0x7f8e5807ae50 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e5807b250 con 0x7f8e5807ae50 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 2499641174) 0x7f8e44005b20 con 0x7f8e5807ae50 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807ae50 legacy=0x7f8e58198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e33fff640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5807ae50 legacy=0x7f8e58198790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 236673, 2026-02-21T22:31:30.401 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.402 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:30.403 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.404 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.405 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.406 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.178:6800/2549949439 conn(0x7f8e2803d690 legacy=0x7f8e2805db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 10 monclient: shutdown 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 >> v1:10.20.193.102:6790/0 conn(0x7f8e5808d7e0 legacy=0x7f8e58195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e57fff640 1 -- 10.20.193.102:0/3841407263 reap_dead start 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e57fff640 1 -- 10.20.193.102:0/3841407263 reap_dead start 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 shutdown_connections 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 >> 10.20.193.102:0/3841407263 conn(0x7f8e58091a50 msgr2=0x7f8e5805bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 shutdown_connections 2026-02-21T22:31:30.407 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.398+0000 7f8e5dacb640 1 -- 10.20.193.102:0/3841407263 wait complete. 2026-02-21T22:31:30.411 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:30.411 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:31:30.411 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:31:30.443 INFO:tasks.rados.rados.0.trial178.stdout:3695: done (0 left) 2026-02-21T22:31:30.443 INFO:tasks.rados.rados.0.trial178.stdout:3696: read oid 32 snap -1 2026-02-21T22:31:30.443 INFO:tasks.rados.rados.0.trial178.stdout:3696: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:31:30.443 INFO:tasks.rados.rados.0.trial178.stdout:3697: snap_remove snap 368 2026-02-21T22:31:30.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: get_monmap_and_config 2026-02-21T22:31:30.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: monmap: 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:30.471643+0000 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:30.471643+0000 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:30.471 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 1 Processor -- start 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 1 -- start start 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: init 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:30.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-c con 0x7fdd6408cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-a con 0x7fdd6408d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-b con 0x7fdd6408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640890e0 con 0x7fdd6408d7e0 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd64088ea0 con 0x7fdd6408cd00 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6408aea0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6b209640 10 monclient(hunting): authenticate will time out at 2099.707564s 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd5bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdd6408d7e0 0x7fdd641840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41318/0 (socket says 10.20.193.102:41318) 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd5bfff640 1 -- 10.20.193.102:0/2068660068 learned_addr learned my addr 10.20.193.102:0/2068660068 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.473 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.466+0000 7fdd6977f640 1 --1- 10.20.193.102:0/2068660068 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 0x7fdd641a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:45418/0 (socket says 10.20.193.102:45418) 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6977f640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd641a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5bfff640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd641840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd68f7e640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd6408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933968416 0 0) 0x7fdd640890e0 con 0x7fdd6408d7e0 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): my global_id is 34812 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd50003620 con 0x7fdd6408d7e0 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923635245 0 0) 0x7fdd6408aea0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): my global_id is 94310 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd640890e0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913515653 0 0) 0x7fdd64088ea0 con 0x7fdd6408cd00 2026-02-21T22:31:30.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient(hunting): my global_id is 124150 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd6408aea0 con 0x7fdd6408cd00 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4209551378 0 0) 0x7fdd50003620 con 0x7fdd6408d7e0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd641a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd6408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 monclient: found mon.noname-a 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6408a600 con 0x7fdd6408d7e0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd4c003040 con 0x7fdd6408d7e0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: dump: 2026-02-21T22:31:30.476 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 5 monclient: authenticate success, global_id 34812 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd4c003310 con 0x7fdd6408d7e0 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 1 -- 10.20.193.102:0/2068660068 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd4c0049a0 con 0x7fdd6408d7e0 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd5b7fe640 10 monclient: dump: 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 4 set_mon_vals no callback set 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 10 monclient: shutdown 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- 10.20.193.102:0/2068660068 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd641840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- 10.20.193.102:0/2068660068 shutdown_connections 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- 10.20.193.102:0/2068660068 >> 10.20.193.102:0/2068660068 conn(0x7fdd64091a50 msgr2=0x7fdd64091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- 10.20.193.102:0/2068660068 shutdown_connections 2026-02-21T22:31:30.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- 10.20.193.102:0/2068660068 wait complete. 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 10 monclient: monmap: 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 Processor -- start 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 1 -- start start 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.470+0000 7fdd6b209640 10 monclient: init 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:30.481 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-c con 0x7fdd6408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-a con 0x7fdd6408cd00 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): picked mon.noname-b con 0x7fdd6408d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640821e0 con 0x7fdd6408cd00 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd64088a70 con 0x7fdd6408d7e0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6405d220 con 0x7fdd6408b8d0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): authenticate will time out at 2099.712191s 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 0x7fdd64194650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41326/0 (socket says 10.20.193.102:41326) 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 -- 10.20.193.102:0/1941114959 learned_addr learned my addr 10.20.193.102:0/1941114959 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd64194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd68f7e640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd64083710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397340068 0 0) 0x7fdd640821e0 con 0x7fdd6408cd00 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): my global_id is 34815 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd34003620 con 0x7fdd6408cd00 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355084069 0 0) 0x7fdd6405d220 con 0x7fdd6408b8d0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): my global_id is 94313 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd640821e0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6977f640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd64195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.482 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3516503858 0 0) 0x7fdd34003620 con 0x7fdd6408cd00 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd6405d220 con 0x7fdd6408cd00 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd4c0048e0 con 0x7fdd6408cd00 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): dump: 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277341131 0 0) 0x7fdd64088a70 con 0x7fdd6408d7e0 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): my global_id is 124153 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd34003620 con 0x7fdd6408d7e0 2026-02-21T22:31:30.483 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 197798258 0 0) 0x7fdd640821e0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd64088a70 con 0x7fdd6408b8d0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd540033c0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): dump: 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 816816159 0 0) 0x7fdd6405d220 con 0x7fdd6408cd00 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd64083710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd64195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 monclient: found mon.a 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6407fe80 con 0x7fdd6408cd00 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.484 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 5 monclient: authenticate success, global_id 34815 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient: _renew_subs 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdd64081f00 con 0x7fdd6408cd00 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd64194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd64194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 --1- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 0x7fdd64194650 :-1 s=OPENED pgs=612 gs=2 cs=1 l=1).handle_message read tag failed 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd5bfff640 1 --1- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 0x7fdd64194650 :-1 s=OPENED pgs=612 gs=2 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6789/0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd6408cd00 legacy=0x7fdd64194650 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): picked mon.b con 0x7fdd6408d7e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): picked mon.a con 0x7fdd34026a80 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): picked mon.c con 0x7fdd6408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdd6405d220 con 0x7fdd34026a80 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdd640821e0 con 0x7fdd6408d7e0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdd3406c4b0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd597fa640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.485 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.474+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdd640008e0 con 0x7fdd64153580 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd5bfff640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd34026a80 legacy=0x7fdd34046e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd6977f640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd34068850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997697689 0 0) 0x7fdd6405d220 con 0x7fdd34026a80 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fdd3406c680 con 0x7fdd34026a80 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd68f7e640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd34004590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394553436 0 0) 0x7fdd3406c4b0 con 0x7fdd6408b8d0 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fdd6405d220 con 0x7fdd6408b8d0 2026-02-21T22:31:30.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 546170397 0 0) 0x7fdd3406c680 con 0x7fdd34026a80 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6790/0 conn(0x7fdd6408b8d0 legacy=0x7fdd34068850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd6408d7e0 legacy=0x7fdd34004590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 monclient: found mon.a 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) -- 0x7fdd3406d870 con 0x7fdd34026a80 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdd641532f0 con 0x7fdd34026a80 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd4c0052c0 con 0x7fdd34026a80 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: dump: 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd68f7e640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64153580 legacy=0x7fdd64083710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd4c005460 con 0x7fdd34026a80 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd5affd640 4 set_mon_vals no callback set 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026015569 0 0) 0x7fdd640008e0 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient(hunting): my global_id is 124156 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd3406c680 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1469961364 0 0) 0x7fdd3406c680 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd640008e0 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd54003b40 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: dump: 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdd4c011b00 con 0x7fdd34026a80 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.0 v1:10.20.193.102:6789/0 6 ==== osd_map(770..770 src has 527..770) ==== 3899+0+0 (unknown 2209956645 0 0) 0x7fdd4c04c8e0 con 0x7fdd34026a80 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3536990390 0 0) 0x7fdd640008e0 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd64090810 con 0x7fdd64153580 2026-02-21T22:31:30.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd6977f640 10 monclient: get_auth_request con 0x7fdd6408cd00 auth_method 0 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7fdd540058c0 con 0x7fdd64153580 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64153580 legacy=0x7fdd64083710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.478+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64153580 legacy=0x7fdd64083710 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.482+0000 7fdd6b209640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.482+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdd640909c0 con 0x7fdd64083e20 2026-02-21T22:31:30.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd68f7e640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64083e20 legacy=0x7fdd6407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797317000 0 0) 0x7fdd640909c0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient(hunting): my global_id is 124159 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd640008e0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3415994070 0 0) 0x7fdd640008e0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd640909c0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdd54004440 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient: dump: 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 818654955 0 0) 0x7fdd640909c0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdd641777e0 con 0x7fdd64083e20 2026-02-21T22:31:30.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4346 (unknown 0 0 2761309710) 0x7fdd54005b60 con 0x7fdd64083e20 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64083e20 legacy=0x7fdd6407a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd597fa640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6789/0 conn(0x7fdd64083e20 legacy=0x7fdd6407a370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 58, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 17127, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 2, 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.492 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:30.493 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:30.494 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:30.495 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.496 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.178:6800/2549949439 conn(0x7fdd6408cd00 legacy=0x7fdd340c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 10 monclient: shutdown 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 >> v1:10.20.193.102:6789/0 conn(0x7fdd34026a80 legacy=0x7fdd34046e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6977f640 1 -- 10.20.193.102:0/1941114959 reap_dead start 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6977f640 1 -- 10.20.193.102:0/1941114959 reap_dead start 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 shutdown_connections 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 >> 10.20.193.102:0/1941114959 conn(0x7fdd64091a50 msgr2=0x7fdd6405bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 shutdown_connections 2026-02-21T22:31:30.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.486+0000 7fdd6b209640 1 -- 10.20.193.102:0/1941114959 wait complete. 2026-02-21T22:31:30.498 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:31:30.499 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2026-02-21T22:31:30.499 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2026-02-21T22:31:30.499 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2026-02-21T22:31:30.499 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2026-02-21T22:31:30.499 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:31:30.499 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:31:30.510 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:31:30.510 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2026-02-21T22:31:30.510 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:31:30.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: get_monmap_and_config 2026-02-21T22:31:30.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: monmap: 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:30.579478+0000 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:30.579478+0000 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:30.579 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 1 Processor -- start 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 1 -- start start 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: init 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-c con 0x7f457808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-a con 0x7f457808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-b con 0x7f457808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4578086a10 con 0x7f457808b7f0 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457808b130 con 0x7f457808ce00 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457808bd60 con 0x7f457808d8e0 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457f5b6640 10 monclient(hunting): authenticate will time out at 2099.815351s 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457cb2a640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 0x7f4578181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41346/0 (socket says 10.20.193.102:41346) 2026-02-21T22:31:30.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457d32b640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f457808ce00 legacy=0x7f457808d1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:30.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.574+0000 7f457cb2a640 1 -- 10.20.193.102:0/4291194772 learned_addr learned my addr 10.20.193.102:0/4291194772 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457cb2a640 1 -- 10.20.193.102:0/4291194772 >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 legacy=0x7f4578181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457db2c640 1 -- 10.20.193.102:0/4291194772 >> v1:10.20.193.102:6790/0 conn(0x7f457808d8e0 legacy=0x7f45781a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342427884 0 0) 0x7f4578086a10 con 0x7f457808b7f0 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): my global_id is 34818 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4560003620 con 0x7f457808b7f0 2026-02-21T22:31:30.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1295170921 0 0) 0x7f457808bd60 con 0x7f457808d8e0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient(hunting): my global_id is 94316 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4578086a10 con 0x7f457808d8e0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1156777205 0 0) 0x7f4560003620 con 0x7f457808b7f0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 >> v1:10.20.193.102:6790/0 conn(0x7f457808d8e0 legacy=0x7f45781a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 >> v1:10.20.193.178:6789/0 conn(0x7f457808ce00 legacy=0x7f457808d1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 monclient: found mon.noname-a 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457808b370 con 0x7f457808b7f0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f456c003110 con 0x7f457808b7f0 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 5 monclient: authenticate success, global_id 34818 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: dump: 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f456c0033e0 con 0x7f457808b7f0 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 1 -- 10.20.193.102:0/4291194772 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f456c004b50 con 0x7f457808b7f0 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f4567fff640 10 monclient: dump: 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.586 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 4 set_mon_vals no callback set 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: shutdown 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- 10.20.193.102:0/4291194772 >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 legacy=0x7f4578181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- 10.20.193.102:0/4291194772 shutdown_connections 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- 10.20.193.102:0/4291194772 >> 10.20.193.102:0/4291194772 conn(0x7f4578091a50 msgr2=0x7f4578091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- 10.20.193.102:0/4291194772 shutdown_connections 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- 10.20.193.102:0/4291194772 wait complete. 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: build_initial_monmap 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: monmap: 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:30.587 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:30.588 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 Processor -- start 2026-02-21T22:31:30.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- start start 2026-02-21T22:31:30.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: init 2026-02-21T22:31:30.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:30.591 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-a con 0x7f457808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-b con 0x7f457808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient(hunting): picked mon.noname-c con 0x7f457808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4578080ef0 con 0x7f457808b7f0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4578088a70 con 0x7f457808ce00 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4578083250 con 0x7f457808d8e0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.578+0000 7f457f5b6640 10 monclient(hunting): authenticate will time out at 2099.819972s 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457cb2a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f457808ce00 legacy=0x7f4578084780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457d32b640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 0x7f4578153030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:41354/0 (socket says 10.20.193.102:41354) 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457d32b640 1 -- 10.20.193.102:0/593308634 learned_addr learned my addr 10.20.193.102:0/593308634 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457d32b640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 legacy=0x7f4578153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457db2c640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.102:6790/0 conn(0x7f457808d8e0 legacy=0x7f45781953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784812143 0 0) 0x7f4578080ef0 con 0x7f457808b7f0 2026-02-21T22:31:30.592 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): my global_id is 34821 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4548003620 con 0x7f457808b7f0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401315535 0 0) 0x7f4578083250 con 0x7f457808d8e0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): my global_id is 94319 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4578080ef0 con 0x7f457808d8e0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 381970038 0 0) 0x7f4548003620 con 0x7f457808b7f0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4578083250 con 0x7f457808b7f0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4568004590 con 0x7f457808b7f0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): dump: 2026-02-21T22:31:30.593 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 919742485 0 0) 0x7f4578080ef0 con 0x7f457808d8e0 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4548003620 con 0x7f457808d8e0 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4574004900 con 0x7f457808d8e0 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:30.594 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient(hunting): dump: 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1758898818 0 0) 0x7f4578083250 con 0x7f457808b7f0 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.102:6790/0 conn(0x7f457808d8e0 legacy=0x7f45781953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.178:6789/0 conn(0x7f457808ce00 legacy=0x7f4578084780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 monclient: found mon.a 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:30.595 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45780829b0 con 0x7f457808b7f0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: _finish_auth 0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 5 monclient: authenticate success, global_id 34821 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _renew_subs 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4578081a50 con 0x7f457808b7f0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _renew_subs 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4578085060 con 0x7f457808b7f0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4568004860 con 0x7f457808b7f0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f45677fe640 4 set_mon_vals no callback set 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4568004c40 con 0x7f457808b7f0 2026-02-21T22:31:30.596 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 10 monclient: dump: 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45780008e0 con 0x7f457808b7f0 2026-02-21T22:31:30.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4568004e60 con 0x7f457808b7f0 2026-02-21T22:31:30.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(770..770 src has 527..770) ==== 3899+0+0 (unknown 2209956645 0 0) 0x7f456804c7c0 con 0x7f457808b7f0 2026-02-21T22:31:30.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.582+0000 7f457cb2a640 10 monclient: get_auth_request con 0x7f454803d610 auth_method 0 2026-02-21T22:31:30.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.586+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f4568017720 con 0x7f457808b7f0 2026-02-21T22:31:30.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.586+0000 7f4565ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:30.598 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.586+0000 7f4565ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:30.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4578090810 con 0x7f457808b7f0 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f4565ffb640 1 -- 10.20.193.102:0/593308634 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 2831377381) 0x7f456801ca40 con 0x7f457808b7f0 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f4565ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f4565ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:30.706 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:31:30.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.178:6800/2549949439 conn(0x7f454803d610 legacy=0x7f454805dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 10 monclient: shutdown 2026-02-21T22:31:30.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:30.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 >> v1:10.20.193.102:6789/0 conn(0x7f457808b7f0 legacy=0x7f4578153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:30.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 shutdown_connections 2026-02-21T22:31:30.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 >> 10.20.193.102:0/593308634 conn(0x7f4578091a50 msgr2=0x7f4578057ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:30.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 shutdown_connections 2026-02-21T22:31:30.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:30.702+0000 7f457f5b6640 1 -- 10.20.193.102:0/593308634 wait complete. 2026-02-21T22:31:33.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:33.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:33.716 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:31:33.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.781+0000 7f421b913640 10 monclient: get_monmap_and_config 2026-02-21T22:31:33.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.781+0000 7f421b913640 10 monclient: build_initial_monmap 2026-02-21T22:31:33.788 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.781+0000 7f421b913640 10 monclient: monmap: 2026-02-21T22:31:33.788 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:33.788316+0000 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:33.788316+0000 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:33.789 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.781+0000 7f421b913640 1 Processor -- start 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- start start 2026-02-21T22:31:33.790 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient: init 2026-02-21T22:31:33.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:33.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:31:33.794 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient(hunting): picked mon.noname-a con 0x7f4214088d80 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient(hunting): picked mon.noname-b con 0x7f421408a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient(hunting): picked mon.noname-c con 0x7f4214089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4214085920 con 0x7f4214088d80 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421405e780 con 0x7f4214089860 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4214086330 con 0x7f421408a1d0 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient(hunting): authenticate will time out at 2103.024474s 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4219688640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f4214088d80 0x7f4214089150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:55764/0 (socket says 10.20.193.102:55764) 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4219688640 1 -- 10.20.193.102:0/2806767096 learned_addr learned my addr 10.20.193.102:0/2806767096 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4219e89640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f4214089860 legacy=0x7f42141a2370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:33.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4219688640 1 -- 10.20.193.102:0/2806767096 >> v1:10.20.193.102:6789/0 conn(0x7f4214088d80 legacy=0x7f4214089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4218e87640 1 -- 10.20.193.102:0/2806767096 >> v1:10.20.193.102:6790/0 conn(0x7f421408a1d0 legacy=0x7f4214180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921901830 0 0) 0x7f4214085920 con 0x7f4214088d80 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient(hunting): _negotiate 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient(hunting): my global_id is 34824 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41fc003620 con 0x7f4214088d80 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2482961184 0 0) 0x7f41fc003620 con 0x7f4214088d80 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 >> v1:10.20.193.102:6790/0 conn(0x7f421408a1d0 legacy=0x7f4214180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 >> v1:10.20.193.178:6789/0 conn(0x7f4214089860 legacy=0x7f42141a2370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:33.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: _finish_hunting 0 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 monclient: found mon.noname-a 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421408c6b0 con 0x7f4214088d80 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: _finish_auth 0 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: _check_auth_tickets 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 5 monclient: authenticate success, global_id 34824 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4204003110 con 0x7f4214088d80 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:33.797 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: dump: 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4204003450 con 0x7f4214088d80 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 1 -- 10.20.193.102:0/2806767096 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4204004b50 con 0x7f4214088d80 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f4203fff640 10 monclient: dump: 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:33.799 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 4 set_mon_vals no callback set 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 10 monclient: shutdown 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- 10.20.193.102:0/2806767096 >> v1:10.20.193.102:6789/0 conn(0x7f4214088d80 legacy=0x7f4214089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- 10.20.193.102:0/2806767096 shutdown_connections 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- 10.20.193.102:0/2806767096 >> 10.20.193.102:0/2806767096 conn(0x7f4214091a50 msgr2=0x7f4214091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- 10.20.193.102:0/2806767096 shutdown_connections 2026-02-21T22:31:33.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.785+0000 7f421b913640 1 -- 10.20.193.102:0/2806767096 wait complete. 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient: build_initial_monmap 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient: monmap: 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:33.801 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 1 Processor -- start 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 1 -- start start 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient: init 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient(hunting): picked mon.noname-c con 0x7f4214088d80 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient(hunting): picked mon.noname-b con 0x7f4214089860 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient(hunting): picked mon.noname-a con 0x7f421408a1d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421413fcd0 con 0x7f421408a1d0 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421413cd60 con 0x7f4214089860 2026-02-21T22:31:33.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4214088190 con 0x7f4214088d80 2026-02-21T22:31:33.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:33.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f421b913640 10 monclient(hunting): authenticate will time out at 2103.029118s 2026-02-21T22:31:33.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4219688640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4214088d80 0x7f421413bf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:57230/0 (socket says 10.20.193.102:57230) 2026-02-21T22:31:33.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4219688640 1 -- 10.20.193.102:0/2836875830 learned_addr learned my addr 10.20.193.102:0/2836875830 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:33.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4218e87640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.178:6789/0 conn(0x7f4214089860 legacy=0x7f421413c650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:33.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4219688640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.102:6790/0 conn(0x7f4214088d80 legacy=0x7f421413bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:33.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4219e89640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.102:6789/0 conn(0x7f421408a1d0 legacy=0x7f421423c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:33.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2936397212 0 0) 0x7f421413fcd0 con 0x7f421408a1d0 2026-02-21T22:31:33.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): my global_id is 34827 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e4003620 con 0x7f421408a1d0 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2704342450 0 0) 0x7f41e4003620 con 0x7f421408a1d0 2026-02-21T22:31:33.810 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f421413fcd0 con 0x7f421408a1d0 2026-02-21T22:31:33.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f42100058f0 con 0x7f421408a1d0 2026-02-21T22:31:33.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:33.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:33.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): dump: 2026-02-21T22:31:33.811 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:33.812 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1174332796 0 0) 0x7f421413fcd0 con 0x7f421408a1d0 2026-02-21T22:31:33.813 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.102:6790/0 conn(0x7f4214088d80 legacy=0x7f421413bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.178:6789/0 conn(0x7f4214089860 legacy=0x7f421413c650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 monclient: found mon.a 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.814 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4214088a70 con 0x7f421408a1d0 2026-02-21T22:31:33.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient: _finish_auth 0 2026-02-21T22:31:33.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:31:33.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.789+0000 7f4201ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:33.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 5 monclient: authenticate success, global_id 34827 2026-02-21T22:31:33.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _renew_subs 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f421413d970 con 0x7f421408a1d0 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _renew_subs 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4214084f50 con 0x7f421408a1d0 2026-02-21T22:31:33.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4210005bc0 con 0x7f421408a1d0 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4210005f60 con 0x7f421408a1d0 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f42037fe640 4 set_mon_vals no callback set 2026-02-21T22:31:33.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: dump: 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:33.818 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:33.819 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421413f530 con 0x7f421408a1d0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4210006230 con 0x7f421408a1d0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(770..770 src has 527..770) ==== 3899+0+0 (unknown 2209956645 0 0) 0x7f421004da50 con 0x7f421408a1d0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4219688640 10 monclient: get_auth_request con 0x7f41e403d340 auth_method 0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f4210018a70 con 0x7f421408a1d0 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:33.820 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.793+0000 7f4201ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:33.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:31:33.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:33.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:33.918 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4214090810 con 0x7f421408a1d0 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f4201ffb640 1 -- 10.20.193.102:0/2836875830 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3467381445) 0x7f4210018350 con 0x7f421408a1d0 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f4201ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f4201ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.178:6800/2549949439 conn(0x7f41e403d340 legacy=0x7f41e405d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 10 monclient: shutdown 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 >> v1:10.20.193.102:6789/0 conn(0x7f421408a1d0 legacy=0x7f421423c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 shutdown_connections 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 >> 10.20.193.102:0/2836875830 conn(0x7f4214091a50 msgr2=0x7f4214086500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:33.919 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 shutdown_connections 2026-02-21T22:31:33.920 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:33.913+0000 7f421b913640 1 -- 10.20.193.102:0/2836875830 wait complete. 2026-02-21T22:31:36.929 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:31:36.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: get_monmap_and_config 2026-02-21T22:31:36.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: build_initial_monmap 2026-02-21T22:31:36.992 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: monmap: 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:36.991864+0000 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:36.991864+0000 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:36.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 1 Processor -- start 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 1 -- start start 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: init 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-b con 0x7f948008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-c con 0x7f948008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-a con 0x7f948008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9480086a10 con 0x7f948008d8e0 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948008b130 con 0x7f948008b7f0 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948008bd60 con 0x7f948008ce00 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f9487ae6640 10 monclient(hunting): authenticate will time out at 2106.227738s 2026-02-21T22:31:36.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f948505a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:36.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f948585b640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 0x7f948008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:57252/0 (socket says 10.20.193.102:57252) 2026-02-21T22:31:36.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.985+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 learned_addr learned my addr 10.20.193.102:0/3615139503 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:36.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.989+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f948008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:36.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:36.989+0000 7f948605c640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f948008d8e0 legacy=0x7f94801a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:37.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:37.189+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:37.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:37.593+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:38.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:38.397+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:38.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:39.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): tick 2026-02-21T22:31:39.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:31:39.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 monclient(hunting): continuing hunt 2026-02-21T22:31:39.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:31:39.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f948008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f948008d8e0 legacy=0x7f94801a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f948008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f948008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f948008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948008b370 con 0x7f948008d8e0 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948008b130 con 0x7f948008b7f0 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f946406a0c0 con 0x7f948008ce00 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f946f7fe640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:39.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.985+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:39.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.989+0000 7f948605c640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f94640668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:39.996 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:39.989+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f948008d8e0 legacy=0x7f94640450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:40.192 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:40.189+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:40.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:40.593+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:41.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:41.397+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:42.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): tick 2026-02-21T22:31:42.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): _check_auth_tickets 2026-02-21T22:31:42.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 monclient(hunting): continuing hunt 2026-02-21T22:31:42.993 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): _reopen_session rank -1 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f94640668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f948008d8e0 legacy=0x7f94640450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-a con 0x7f9464068150 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-c con 0x7f948008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): picked mon.noname-b con 0x7f948008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f946406b3c0 con 0x7f9464068150 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948008b130 con 0x7f948008b7f0 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f946406ac50 con 0x7f948008ce00 2026-02-21T22:31:42.994 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f946f7fe640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:42.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:42.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f948585b640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f9464068150 legacy=0x7f946406a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:42.995 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:42.989+0000 7f948605c640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f94640668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:43.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:43.193+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:43.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:43.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:43.597 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:43.597+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:44.404 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:44.401+0000 7f948505a640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398606769 0 0) 0x7f946406ac50 con 0x7f948008ce00 2026-02-21T22:31:45.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient(hunting): my global_id is 94322 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9468003620 con 0x7f948008ce00 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2357999549 0 0) 0x7f9468003620 con 0x7f948008ce00 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.178:6789/0 conn(0x7f948008b7f0 legacy=0x7f94640238d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6789/0 conn(0x7f9464068150 legacy=0x7f946406a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 monclient: found mon.noname-b 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946406aee0 con 0x7f948008ce00 2026-02-21T22:31:45.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 5 monclient: authenticate success, global_id 94322 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f947c004600 con 0x7f948008ce00 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: dump: 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.494 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f947c004850 con 0x7f948008ce00 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 1 -- 10.20.193.102:0/3615139503 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f947c004c20 con 0x7f948008ce00 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9484859640 10 monclient: dump: 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 4 set_mon_vals no callback set 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: shutdown 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- 10.20.193.102:0/3615139503 >> v1:10.20.193.102:6790/0 conn(0x7f948008ce00 legacy=0x7f94640668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- 10.20.193.102:0/3615139503 shutdown_connections 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- 10.20.193.102:0/3615139503 >> 10.20.193.102:0/3615139503 conn(0x7f9480091a50 msgr2=0x7f9480091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- 10.20.193.102:0/3615139503 shutdown_connections 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- 10.20.193.102:0/3615139503 wait complete. 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: build_initial_monmap 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: monmap: 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:45.497 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 Processor -- start 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- start start 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: init 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:45.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-c con 0x7f9464068150 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-a con 0x7f948008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient(hunting): picked mon.noname-b con 0x7f948008ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948005e1f0 con 0x7f948008b7f0 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9480088a70 con 0x7f948008ce00 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94801d8490 con 0x7f9464068150 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f9487ae6640 10 monclient(hunting): authenticate will time out at 2114.727372s 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f948605c640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f948008ce00 legacy=0x7f948023a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f948585b640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9464068150 0x7f9480084a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37630/0 (socket says 10.20.193.102:37630) 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f948585b640 1 -- 10.20.193.102:0/3457937143 learned_addr learned my addr 10.20.193.102:0/3457937143 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:45.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.485+0000 7f948505a640 1 --1- 10.20.193.102:0/3457937143 >> v1:10.20.193.102:6789/0 conn(0x7f948008b7f0 0x7f9480219100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60278/0 (socket says 10.20.193.102:60278) 2026-02-21T22:31:45.501 INFO:tasks.rados.rados.0.trial178.stdout:3697: done (1 left) 2026-02-21T22:31:45.501 INFO:tasks.rados.rados.0.trial178.stdout:3698: read oid 44 snap -1 2026-02-21T22:31:45.501 INFO:tasks.rados.rados.0.trial178.stdout:3698: expect (ObjNum 1260 snap 359 seq_num 1260) 2026-02-21T22:31:45.501 INFO:tasks.rados.rados.0.trial178.stdout:3699: read oid 45 snap -1 2026-02-21T22:31:45.502 INFO:tasks.rados.rados.0.trial178.stdout:3699: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:31:45.502 INFO:tasks.rados.rados.0.trial178.stdout:3700: copy_from oid 24 from oid 2 current snap is 375 2026-02-21T22:31:45.502 INFO:tasks.rados.rados.0.trial178.stdout:3696: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f948585b640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.102:6790/0 conn(0x7f9464068150 legacy=0x7f9480084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f948505a640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.102:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480219100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197244196 0 0) 0x7f948005e1f0 con 0x7f948008b7f0 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): my global_id is 34830 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9450003620 con 0x7f948008b7f0 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3035106858 0 0) 0x7f9450003620 con 0x7f948008b7f0 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f948005e1f0 con 0x7f948008b7f0 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f94740046c0 con 0x7f948008b7f0 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): dump: 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.505 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 629880241 0 0) 0x7f948005e1f0 con 0x7f948008b7f0 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.102:6790/0 conn(0x7f9464068150 legacy=0x7f9480084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.178:6789/0 conn(0x7f948008ce00 legacy=0x7f948023a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:31:45.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 monclient: found mon.a 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948007f190 con 0x7f948008b7f0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 5 monclient: authenticate success, global_id 34830 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _renew_subs 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9480081210 con 0x7f948008b7f0 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _renew_subs 2026-02-21T22:31:45.507 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9480085150 con 0x7f948008b7f0 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9474004960 con 0x7f948008b7f0 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9474004d00 con 0x7f948008b7f0 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 10 monclient: dump: 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.508 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946ffff640 4 set_mon_vals no callback set 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94800008e0 con 0x7f948008b7f0 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9474004fd0 con 0x7f948008b7f0 2026-02-21T22:31:45.509 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(771..771 src has 527..771) ==== 3943+0+0 (unknown 2512407312 0 0) 0x7f947404c8a0 con 0x7f948008b7f0 2026-02-21T22:31:45.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.489+0000 7f948585b640 10 monclient: get_auth_request con 0x7f945003d340 auth_method 0 2026-02-21T22:31:45.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.493+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9474017830 con 0x7f948008b7f0 2026-02-21T22:31:45.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.493+0000 7f946e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.510 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.493+0000 7f946e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:45.513 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1601283 2026-02-21T22:31:45.605 INFO:tasks.rados.rados.0.trial178.stdout:3700: finishing copy_from to trial17818531-24 2026-02-21T22:31:45.605 INFO:tasks.rados.rados.0.trial178.stdout:3700: got expected ENOENT (src dne) 2026-02-21T22:31:45.605 INFO:tasks.rados.rados.0.trial178.stdout:3700: finishing copy_from racing read to trial17818531-24 2026-02-21T22:31:45.605 INFO:tasks.rados.rados.0.trial178.stdout:3698: expect (ObjNum 1260 snap 359 seq_num 1260) 2026-02-21T22:31:45.605 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:45.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f948008d1d0 con 0x7f948008b7f0 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f946e7fc640 1 -- 10.20.193.102:0/3457937143 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1604 (unknown 254746657 0 3178365276) 0x7f9474017110 con 0x7f948008b7f0 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f946e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f946e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:45.615 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":62,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:31:45.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.178:6800/2549949439 conn(0x7f945003d340 legacy=0x7f945005d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 10 monclient: shutdown 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 >> v1:10.20.193.102:6789/0 conn(0x7f948008b7f0 legacy=0x7f9480219100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 shutdown_connections 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 >> 10.20.193.102:0/3457937143 conn(0x7f9480091a50 msgr2=0x7f94800575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 shutdown_connections 2026-02-21T22:31:45.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.609+0000 7f9487ae6640 1 -- 10.20.193.102:0/3457937143 wait complete. 2026-02-21T22:31:45.626 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2026-02-21T22:31:45.626 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2026-02-21T22:31:45.626 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:31:45.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.681+0000 7f2ab6855640 10 monclient: get_monmap_and_config 2026-02-21T22:31:45.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.681+0000 7f2ab6855640 10 monclient: build_initial_monmap 2026-02-21T22:31:45.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.681+0000 7f2ab6855640 10 monclient: monmap: 2026-02-21T22:31:45.689 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:45.688456+0000 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:45.688456+0000 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 Processor -- start 2026-02-21T22:31:45.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- start start 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: init 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-c con 0x7f2ab008d290 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-b con 0x7f2ab008c500 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-a con 0x7f2ab008dd70 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab008b0b0 con 0x7f2ab008dd70 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab008add0 con 0x7f2ab008d290 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab008f010 con 0x7f2ab008c500 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient(hunting): authenticate will time out at 2114.924379s 2026-02-21T22:31:45.691 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaffff640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f2ab008d290 legacy=0x7f2ab008bdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaf7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f2ab008c500 0x7f2ab008d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37642/0 (socket says 10.20.193.102:37642) 2026-02-21T22:31:45.692 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaf7fe640 1 -- 10.20.193.102:0/2003626073 learned_addr learned my addr 10.20.193.102:0/2003626073 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaf7fe640 1 -- 10.20.193.102:0/2003626073 >> v1:10.20.193.102:6790/0 conn(0x7f2ab008c500 legacy=0x7f2ab008d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab4dcb640 1 -- 10.20.193.102:0/2003626073 >> v1:10.20.193.102:6789/0 conn(0x7f2ab008dd70 legacy=0x7f2ab01a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981645241 0 0) 0x7f2ab008f010 con 0x7f2ab008c500 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): my global_id is 94325 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a940058b0 con 0x7f2ab008c500 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178611691 0 0) 0x7f2aa40041c0 con 0x7f2ab008dd70 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient(hunting): my global_id is 34833 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab008f010 con 0x7f2ab008dd70 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3568819059 0 0) 0x7f2a940058b0 con 0x7f2ab008c500 2026-02-21T22:31:45.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 >> v1:10.20.193.178:6789/0 conn(0x7f2ab008d290 legacy=0x7f2ab008bdf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 >> v1:10.20.193.102:6789/0 conn(0x7f2ab008dd70 legacy=0x7f2ab01a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 monclient: found mon.noname-b 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab01a6400 con 0x7f2ab008c500 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 5 monclient: authenticate success, global_id 94325 2026-02-21T22:31:45.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2aa0003190 con 0x7f2ab008c500 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: dump: 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.699 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2aa0003490 con 0x7f2ab008c500 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 1 -- 10.20.193.102:0/2003626073 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2aa0004b50 con 0x7f2ab008c500 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2aaeffd640 10 monclient: dump: 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.700 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 4 set_mon_vals no callback set 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: shutdown 2026-02-21T22:31:45.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- 10.20.193.102:0/2003626073 >> v1:10.20.193.102:6790/0 conn(0x7f2ab008c500 legacy=0x7f2ab008d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- 10.20.193.102:0/2003626073 shutdown_connections 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- 10.20.193.102:0/2003626073 >> 10.20.193.102:0/2003626073 conn(0x7f2ab0054e40 msgr2=0x7f2ab0092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- 10.20.193.102:0/2003626073 shutdown_connections 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 1 -- 10.20.193.102:0/2003626073 wait complete. 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: build_initial_monmap 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.685+0000 7f2ab6855640 10 monclient: monmap: 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:45.702 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 1 Processor -- start 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 1 -- start start 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient: init 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-a con 0x7f2ab008c500 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-b con 0x7f2ab008d290 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:45.703 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient(hunting): picked mon.noname-c con 0x7f2ab008dd70 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab00894c0 con 0x7f2ab008c500 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab005ec00 con 0x7f2ab008d290 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab01405c0 con 0x7f2ab008dd70 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab6855640 10 monclient(hunting): authenticate will time out at 2114.928333s 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab4dcb640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f2ab008dd70 0x7f2ab0195e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37644/0 (socket says 10.20.193.102:37644) 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab4dcb640 1 -- 10.20.193.102:0/1866291410 learned_addr learned my addr 10.20.193.102:0/1866291410 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2aaf7fe640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.178:6789/0 conn(0x7f2ab008d290 legacy=0x7f2ab0194770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2ab4dcb640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6790/0 conn(0x7f2ab008dd70 legacy=0x7f2ab0195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:31:45.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.689+0000 7f2aaffff640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab008c500 legacy=0x7f2ab0085a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430002749 0 0) 0x7f2ab01405c0 con 0x7f2ab008dd70 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): my global_id is 94328 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a80003620 con 0x7f2ab008dd70 2026-02-21T22:31:45.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962610471 0 0) 0x7f2ab00894c0 con 0x7f2ab008c500 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): my global_id is 34836 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab01405c0 con 0x7f2ab008c500 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2068244137 0 0) 0x7f2a80003620 con 0x7f2ab008dd70 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ab00894c0 con 0x7f2ab008dd70 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2aa40041d0 con 0x7f2ab008dd70 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:31:45.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): dump: 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3166616245 0 0) 0x7f2ab01405c0 con 0x7f2ab008c500 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a80003620 con 0x7f2ab008c500 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a9c004400 con 0x7f2ab008c500 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): dump: 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.713 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2105144678 0 0) 0x7f2ab00894c0 con 0x7f2ab008dd70 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.178:6789/0 conn(0x7f2ab008d290 legacy=0x7f2ab0194770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab008c500 legacy=0x7f2ab0085a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 monclient: found mon.c 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:45.714 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab007a4a0 con 0x7f2ab008dd70 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 5 monclient: authenticate success, global_id 94328 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 10 monclient: _renew_subs 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab013f380 con 0x7f2ab008dd70 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 10 monclient: _renew_subs 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2ab0153430 con 0x7f2ab008dd70 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2aa4004ed0 con 0x7f2ab008dd70 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.715 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2aa4005260 con 0x7f2ab008dd70 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aae7fc640 4 set_mon_vals no callback set 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient: dump: 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.716 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2ab00008e0 con 0x7f2ab0056480 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f2aa4005440 con 0x7f2ab008dd70 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(771..771 src has 527..771) ==== 3943+0+0 (unknown 2512407312 0 0) 0x7f2aa404db80 con 0x7f2ab008dd70 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aaf7fe640 10 monclient: get_auth_request con 0x7f2a8003d640 auth_method 0 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aaffff640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab0056480 legacy=0x7f2ab0138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.717 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796972581 0 0) 0x7f2ab00008e0 con 0x7f2ab0056480 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 10 monclient(hunting): my global_id is 34839 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.693+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab00894c0 con 0x7f2ab0056480 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 376493531 0 0) 0x7f2ab00894c0 con 0x7f2ab0056480 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ab00008e0 con 0x7f2ab0056480 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a9c004b00 con 0x7f2ab0056480 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.718 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 10 monclient: dump: 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2503452065 0 0) 0x7f2ab00008e0 con 0x7f2ab0056480 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ab0153720 con 0x7f2ab0056480 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f2a9c004de0 con 0x7f2ab0056480 2026-02-21T22:31:45.719 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab0056480 legacy=0x7f2ab0138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab0056480 legacy=0x7f2ab0138ae0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2ab6855640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.697+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2ab0091810 con 0x7f2ab004d280 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aaffff640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab004d280 legacy=0x7f2ab01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1128176606 0 0) 0x7f2ab0091810 con 0x7f2ab004d280 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient(hunting): my global_id is 34842 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab00008e0 con 0x7f2ab004d280 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1902791660 0 0) 0x7f2ab00008e0 con 0x7f2ab004d280 2026-02-21T22:31:45.720 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ab0091810 con 0x7f2ab004d280 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f2a9c0054b0 con 0x7f2ab004d280 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient: dump: 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.721 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2600379678 0 0) 0x7f2ab0091810 con 0x7f2ab004d280 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2ab00919e0 con 0x7f2ab004d280 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4672 (unknown 0 0 201700217) 0x7f2a9c006930 con 0x7f2ab004d280 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab004d280 legacy=0x7f2ab01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2aacff9640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6789/0 conn(0x7f2ab004d280 legacy=0x7f2ab01985c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:45.722 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 62, 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 854570, 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:45.723 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.724 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.725 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:45.726 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.727 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.728 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:45.729 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.730 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.731 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.732 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 4 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.733 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:45.734 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:45.734 INFO:tasks.rados.rados.0.trial178.stdout:3699: expect (ObjNum 1167 snap 334 seq_num 1167) 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.178:6800/2549949439 conn(0x7f2a8003d640 legacy=0x7f2a8005daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 10 monclient: shutdown 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 >> v1:10.20.193.102:6790/0 conn(0x7f2ab008dd70 legacy=0x7f2ab0195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab4dcb640 1 -- 10.20.193.102:0/1866291410 reap_dead start 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab4dcb640 1 -- 10.20.193.102:0/1866291410 reap_dead start 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 shutdown_connections 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 >> 10.20.193.102:0/1866291410 conn(0x7f2ab0054e40 msgr2=0x7f2ab005dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.735 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 shutdown_connections 2026-02-21T22:31:45.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.701+0000 7f2ab6855640 1 -- 10.20.193.102:0/1866291410 wait complete. 2026-02-21T22:31:45.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:31:45.736 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2026-02-21T22:31:45.736 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:31:45.757 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: get_monmap_and_config 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: build_initial_monmap 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: monmap: 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:31:45.795854+0000 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:31:45.795854+0000 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:31:45.795 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 1 Processor -- start 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 1 -- start start 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: init 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-b con 0x7f1e4008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-a con 0x7f1e4008b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-c con 0x7f1e4008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e40086a10 con 0x7f1e4008b7f0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4008b130 con 0x7f1e4008d8e0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4008bd60 con 0x7f1e4008ce00 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e46904640 10 monclient(hunting): authenticate will time out at 2115.031933s 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e44e7a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f1e4008d8e0 legacy=0x7f1e401a3570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e3ffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f1e4008ce00 0x7f1e4008d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:37646/0 (socket says 10.20.193.102:37646) 2026-02-21T22:31:45.796 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.789+0000 7f1e3ffff640 1 -- 10.20.193.102:0/3540503227 learned_addr learned my addr 10.20.193.102:0/3540503227 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:45.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3f7fe640 1 -- 10.20.193.102:0/3540503227 >> v1:10.20.193.102:6789/0 conn(0x7f1e4008b7f0 legacy=0x7f1e40181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.798 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3ffff640 1 -- 10.20.193.102:0/3540503227 >> v1:10.20.193.102:6790/0 conn(0x7f1e4008ce00 legacy=0x7f1e4008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040552314 0 0) 0x7f1e40086a10 con 0x7f1e4008b7f0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): my global_id is 34845 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e24003620 con 0x7f1e4008b7f0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613609159 0 0) 0x7f1e4008bd60 con 0x7f1e4008ce00 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient(hunting): my global_id is 94331 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e40086a10 con 0x7f1e4008ce00 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2776612014 0 0) 0x7f1e24003620 con 0x7f1e4008b7f0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 >> v1:10.20.193.102:6790/0 conn(0x7f1e4008ce00 legacy=0x7f1e4008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 >> v1:10.20.193.178:6789/0 conn(0x7f1e4008d8e0 legacy=0x7f1e401a3570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 monclient: found mon.noname-a 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4008b370 con 0x7f1e4008b7f0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.799 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 5 monclient: authenticate success, global_id 34845 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e30003160 con 0x7f1e4008b7f0 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: dump: 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1e30003430 con 0x7f1e4008b7f0 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 10 monclient: get_monmap_and_config success 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 4 set_mon_vals no callback set 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 1 -- 10.20.193.102:0/3540503227 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e300038c0 con 0x7f1e4008b7f0 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e3effd640 10 monclient: dump: 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.800 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 10 monclient: shutdown 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- 10.20.193.102:0/3540503227 >> v1:10.20.193.102:6789/0 conn(0x7f1e4008b7f0 legacy=0x7f1e40181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- 10.20.193.102:0/3540503227 shutdown_connections 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- 10.20.193.102:0/3540503227 >> 10.20.193.102:0/3540503227 conn(0x7f1e40091a50 msgr2=0x7f1e40091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- 10.20.193.102:0/3540503227 shutdown_connections 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- 10.20.193.102:0/3540503227 wait complete. 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 10 monclient: build_initial_monmap 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 10 monclient: monmap: 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:31:45.801 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 Processor -- start 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 1 -- start start 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.793+0000 7f1e46904640 10 monclient: init 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient: _reopen_session rank -1 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-c con 0x7f1e4008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-a con 0x7f1e4008ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient(hunting): picked mon.noname-b con 0x7f1e4008d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e40081140 con 0x7f1e4008ce00 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4007ede0 con 0x7f1e4008d8e0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e400808a0 con 0x7f1e4008b7f0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient(hunting): _renew_subs 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e46904640 10 monclient(hunting): authenticate will time out at 2115.036065s 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e44e7a640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7f1e4008d8e0 legacy=0x7f1e40193c50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e3f7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f1e4008ce00 0x7f1e40192530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60324/0 (socket says 10.20.193.102:60324) 2026-02-21T22:31:45.802 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e3f7fe640 1 -- 10.20.193.102:0/4276531913 learned_addr learned my addr 10.20.193.102:0/4276531913 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:31:45.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e3f7fe640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6789/0 conn(0x7f1e4008ce00 legacy=0x7f1e40192530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:31:45.803 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.797+0000 7f1e3ffff640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4008b7f0 legacy=0x7f1e40151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681645174 0 0) 0x7f1e40081140 con 0x7f1e4008ce00 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): my global_id is 34848 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e10003620 con 0x7f1e4008ce00 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265748152 0 0) 0x7f1e400808a0 con 0x7f1e4008b7f0 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): my global_id is 94334 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e40081140 con 0x7f1e4008b7f0 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4127753191 0 0) 0x7f1e10003620 con 0x7f1e4008ce00 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e400808a0 con 0x7f1e4008ce00 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e30002bc0 con 0x7f1e4008ce00 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): dump: 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.805 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3ffff640 1 --1- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4008b7f0 0x7f1e40151270 :-1 s=OPENED pgs=528 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f1e2c0032d0 mon_map magic: 0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2903395160 0 0) 0x7f1e40081140 con 0x7f1e4008b7f0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e10003620 con 0x7f1e4008b7f0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1541130465 0 0) 0x7f1e400808a0 con 0x7f1e4008ce00 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4008b7f0 legacy=0x7f1e40151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.178:6789/0 conn(0x7f1e4008d8e0 legacy=0x7f1e40193c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: _finish_hunting 0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 monclient: found mon.a 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e40081b40 con 0x7f1e4008ce00 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: _finish_auth 0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: _check_auth_tickets 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e2c0032d0 con 0x7f1e4008b7f0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 5 monclient: authenticate success, global_id 34848 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 10 monclient: _renew_subs 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4007f940 con 0x7f1e4008ce00 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 10 monclient: _renew_subs 2026-02-21T22:31:45.806 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1e4007e500 con 0x7f1e4008ce00 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1e30003430 con 0x7f1e4008ce00 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e30003fc0 con 0x7f1e4008ce00 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: dump: 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3e7fc640 4 set_mon_vals no callback set 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.807 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1e400008e0 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1e30004290 con 0x7f1e4008ce00 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(771..771 src has 527..771) ==== 3943+0+0 (unknown 2512407312 0 0) 0x7f1e3004c720 con 0x7f1e4008ce00 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e44e7a640 10 monclient: get_auth_request con 0x7f1e1003d570 auth_method 0 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3ffff640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e40151980 legacy=0x7f1e4005c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689650190 0 0) 0x7f1e400008e0 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient(hunting): my global_id is 94337 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e400808a0 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1939481699 0 0) 0x7f1e400808a0 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e400008e0 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e2c004c80 con 0x7f1e40151980 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.801+0000 7f1e3cff9640 10 monclient: dump: 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.808 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1994043660 0 0) 0x7f1e400008e0 con 0x7f1e40151980 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e40088a70 con 0x7f1e40151980 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f1e2c007a20 con 0x7f1e40151980 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e40151980 legacy=0x7f1e4005c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.809 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e40151980 legacy=0x7f1e4005c9c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:45.810 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:45.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e46904640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:31:45.811 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.805+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1e40151d50 con 0x7f1e4004d280 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3ffff640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4004d280 legacy=0x7f1e40196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502969367 0 0) 0x7f1e40151d50 con 0x7f1e4004d280 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient(hunting): _negotiate 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient(hunting): my global_id is 94340 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e400008e0 con 0x7f1e4004d280 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 771812685 0 0) 0x7f1e400008e0 con 0x7f1e4004d280 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1e40151d50 con 0x7f1e4004d280 2026-02-21T22:31:45.815 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1e2c004e60 con 0x7f1e4004d280 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient: dump: 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:31:45.816 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3766244156 0 0) 0x7f1e40151d50 con 0x7f1e4004d280 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1e40080d10 con 0x7f1e4004d280 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4503 (unknown 0 0 2510265801) 0x7f1e2c0071a0 con 0x7f1e4004d280 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4004d280 legacy=0x7f1e40196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e3cff9640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6790/0 conn(0x7f1e4004d280 legacy=0x7f1e40196380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:45.817 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 62, 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 252084, 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:31:45.818 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.819 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.820 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:31:45.821 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:31:45.822 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.823 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.824 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:31:45.825 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:45.826 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "num": 3 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:31:45.827 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.178:6800/2549949439 conn(0x7f1e1003d570 legacy=0x7f1e1005da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 10 monclient: shutdown 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 >> v1:10.20.193.102:6789/0 conn(0x7f1e4008ce00 legacy=0x7f1e40192530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e44e7a640 1 -- 10.20.193.102:0/4276531913 reap_dead start 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e44e7a640 1 -- 10.20.193.102:0/4276531913 reap_dead start 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 shutdown_connections 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 >> 10.20.193.102:0/4276531913 conn(0x7f1e40091a50 msgr2=0x7f1e4005a120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 shutdown_connections 2026-02-21T22:31:45.829 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:31:45.809+0000 7f1e46904640 1 -- 10.20.193.102:0/4276531913 wait complete. 2026-02-21T22:31:45.830 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2026-02-21T22:31:45.830 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2026-02-21T22:31:45.863 INFO:tasks.rados.rados.0.trial178.stdout:3696: done (3 left) 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3698: done (2 left) 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3699: done (1 left) 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3700: done (0 left) 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3701: read oid 8 snap 366 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3701: expect deleted 2026-02-21T22:31:45.864 INFO:tasks.rados.rados.0.trial178.stdout:3702: snap_create 2026-02-21T22:31:46.493 INFO:tasks.rados.rados.0.trial178.stdout:3701: done (1 left) 2026-02-21T22:31:46.493 INFO:tasks.rados.rados.0.trial178.stdout:3702: done (0 left) 2026-02-21T22:31:46.494 INFO:tasks.rados.rados.0.trial178.stdout:3703: delete oid 28 current snap is 376 2026-02-21T22:31:46.498 INFO:tasks.rados.rados.0.trial178.stdout:3703: done (0 left) 2026-02-21T22:31:46.498 INFO:tasks.rados.rados.0.trial178.stdout:3704: write oid 49 current snap is 376 2026-02-21T22:31:46.498 INFO:tasks.rados.rados.0.trial178.stdout:3704: seq_num 1294 ranges {548258=752216,1771817=490781,2985235=713867} 2026-02-21T22:31:46.507 INFO:tasks.rados.rados.0.trial178.stdout:3704: writing trial17818531-49 from 548258 to 1300474 tid 1 2026-02-21T22:31:46.513 INFO:tasks.rados.rados.0.trial178.stdout:3704: writing trial17818531-49 from 1771817 to 2262598 tid 2 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3704: writing trial17818531-49 from 2985235 to 3699102 tid 3 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3705: read oid 22 snap -1 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3705: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3706: rollback oid 21 current snap is 376 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3704: finishing write tid 1 to trial17818531-49 2026-02-21T22:31:46.523 INFO:tasks.rados.rados.0.trial178.stdout:3704: finishing write tid 2 to trial17818531-49 2026-02-21T22:31:46.524 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 21 to 366 2026-02-21T22:31:46.524 INFO:tasks.rados.rados.0.trial178.stdout:3707: read oid 18 snap 366 2026-02-21T22:31:46.524 INFO:tasks.rados.rados.0.trial178.stdout:3707: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:31:46.524 INFO:tasks.rados.rados.0.trial178.stdout:3708: snap_create 2026-02-21T22:31:46.527 INFO:tasks.rados.rados.0.trial178.stdout:3704: finishing write tid 3 to trial17818531-49 2026-02-21T22:31:46.527 INFO:tasks.rados.rados.0.trial178.stdout:3704: finishing write tid 4 to trial17818531-49 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: finishing write tid 5 to trial17818531-49 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: oid 49 updating version 0 to 1646 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: oid 49 updating version 1646 to 1648 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: oid 49 version 1648 is already newer than 1647 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: oid 49 updating version 1648 to 1649 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1649 (ObjNum 1294 snap 376 seq_num 1294) dirty exists 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3704: left oid 49 (ObjNum 1294 snap 376 seq_num 1294) 2026-02-21T22:31:46.528 INFO:tasks.rados.rados.0.trial178.stdout:3705: expect (ObjNum 1232 snap 350 seq_num 1232) 2026-02-21T22:31:46.578 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 19734 2026-02-21T22:31:46.599 INFO:tasks.rados.rados.0.trial178.stdout:3706: finishing rollback tid 0 to trial17818531-21 2026-02-21T22:31:46.599 INFO:tasks.rados.rados.0.trial178.stdout:3706: finishing rollback tid 1 to trial17818531-21 2026-02-21T22:31:46.599 INFO:tasks.rados.rados.0.trial178.stdout:3706: finishing rollback tid 2 to trial17818531-21 2026-02-21T22:31:46.599 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 1985 (ObjNum 1175 snap 336 seq_num 1175) dirty exists 2026-02-21T22:31:46.599 INFO:tasks.rados.rados.0.trial178.stdout:3707: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:31:46.643 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 623121 2026-02-21T22:31:47.220 INFO:tasks.rados.rados.0.trial178.stdout:3704: done (4 left) 2026-02-21T22:31:47.221 INFO:tasks.rados.rados.0.trial178.stdout:3705: done (3 left) 2026-02-21T22:31:47.221 INFO:tasks.rados.rados.0.trial178.stdout:3706: done (2 left) 2026-02-21T22:31:47.221 INFO:tasks.rados.rados.0.trial178.stdout:3707: done (1 left) 2026-02-21T22:31:47.221 INFO:tasks.rados.rados.0.trial178.stdout:3708: done (0 left) 2026-02-21T22:31:47.221 INFO:tasks.rados.rados.0.trial178.stdout:3709: delete oid 31 current snap is 377 2026-02-21T22:31:47.226 INFO:tasks.rados.rados.0.trial178.stdout:3709: done (0 left) 2026-02-21T22:31:47.226 INFO:tasks.rados.rados.0.trial178.stdout:3710: write oid 31 current snap is 377 2026-02-21T22:31:47.226 INFO:tasks.rados.rados.0.trial178.stdout:3710: seq_num 1296 ranges {581872=502714,1488781=494450,2640081=736411} 2026-02-21T22:31:47.233 INFO:tasks.rados.rados.0.trial178.stdout:3710: writing trial17818531-31 from 581872 to 1084586 tid 1 2026-02-21T22:31:47.239 INFO:tasks.rados.rados.0.trial178.stdout:3710: writing trial17818531-31 from 1488781 to 1983231 tid 2 2026-02-21T22:31:47.249 INFO:tasks.rados.rados.0.trial178.stdout:3710: writing trial17818531-31 from 2640081 to 3376492 tid 3 2026-02-21T22:31:47.249 INFO:tasks.rados.rados.0.trial178.stdout:3711: write (excl) oid 1 current snap is 377 2026-02-21T22:31:47.249 INFO:tasks.rados.rados.0.trial178.stdout:3711: seq_num 1297 ranges {598679=577963,1947263=638108,3351151=447528} 2026-02-21T22:31:47.256 INFO:tasks.rados.rados.0.trial178.stdout:3711: writing trial17818531-1 from 598679 to 1176642 tid 1 2026-02-21T22:31:47.266 INFO:tasks.rados.rados.0.trial178.stdout:3711: writing trial17818531-1 from 1947263 to 2585371 tid 2 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3711: writing trial17818531-1 from 3351151 to 3798679 tid 3 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3712: rollback oid 45 current snap is 377 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 45 to 373 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: finishing write tid 1 to trial17818531-31 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3713: delete oid 44 current snap is 377 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: finishing write tid 2 to trial17818531-31 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: finishing write tid 3 to trial17818531-31 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: finishing write tid 4 to trial17818531-31 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: finishing write tid 5 to trial17818531-31 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: oid 31 updating version 0 to 1653 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: oid 31 version 1653 is already newer than 1651 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: oid 31 updating version 1653 to 1654 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: oid 31 version 1654 is already newer than 1652 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1654 (ObjNum 1296 snap 377 seq_num 1296) dirty exists 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3710: left oid 31 (ObjNum 1296 snap 377 seq_num 1296) 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3711: finishing write tid 1 to trial17818531-1 2026-02-21T22:31:47.273 INFO:tasks.rados.rados.0.trial178.stdout:3711: finishing write tid 2 to trial17818531-1 2026-02-21T22:31:47.276 INFO:tasks.rados.rados.0.trial178.stdout:3711: finishing write tid 3 to trial17818531-1 2026-02-21T22:31:47.278 INFO:tasks.rados.rados.0.trial178.stdout:3711: finishing write tid 4 to trial17818531-1 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: finishing write tid 5 to trial17818531-1 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: oid 1 updating version 0 to 1649 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: oid 1 updating version 1649 to 1651 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: oid 1 version 1651 is already newer than 1650 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: oid 1 updating version 1651 to 1652 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1652 (ObjNum 1297 snap 377 seq_num 1297) dirty exists 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3711: left oid 1 (ObjNum 1297 snap 377 seq_num 1297) 2026-02-21T22:31:47.279 INFO:tasks.rados.rados.0.trial178.stdout:3712: finishing rollback tid 0 to trial17818531-45 2026-02-21T22:31:47.280 INFO:tasks.rados.rados.0.trial178.stdout:3712: finishing rollback tid 1 to trial17818531-45 2026-02-21T22:31:47.280 INFO:tasks.rados.rados.0.trial178.stdout:3712: finishing rollback tid 2 to trial17818531-45 2026-02-21T22:31:47.280 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1655 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:31:47.281 INFO:tasks.rados.rados.0.trial178.stdout:3710: done (3 left) 2026-02-21T22:31:47.281 INFO:tasks.rados.rados.0.trial178.stdout:3711: done (2 left) 2026-02-21T22:31:47.281 INFO:tasks.rados.rados.0.trial178.stdout:3712: done (1 left) 2026-02-21T22:31:47.281 INFO:tasks.rados.rados.0.trial178.stdout:3713: done (0 left) 2026-02-21T22:31:47.281 INFO:tasks.rados.rados.0.trial178.stdout:3714: snap_create 2026-02-21T22:31:48.224 INFO:tasks.rados.rados.0.trial178.stdout:3714: done (0 left) 2026-02-21T22:31:48.224 INFO:tasks.rados.rados.0.trial178.stdout:3715: snap_create 2026-02-21T22:31:48.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:48.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:49.225 INFO:tasks.rados.rados.0.trial178.stdout:3715: done (0 left) 2026-02-21T22:31:49.225 INFO:tasks.rados.rados.0.trial178.stdout:3716: write (excl) oid 49 current snap is 379 2026-02-21T22:31:49.225 INFO:tasks.rados.rados.0.trial178.stdout:3716: seq_num 1299 ranges {632293=728461,1947874=641777,3005997=686580} 2026-02-21T22:31:49.234 INFO:tasks.rados.rados.0.trial178.stdout:3716: writing trial17818531-49 from 632293 to 1360754 tid 1 2026-02-21T22:31:49.242 INFO:tasks.rados.rados.0.trial178.stdout:3716: writing trial17818531-49 from 1947874 to 2589651 tid 2 2026-02-21T22:31:49.250 INFO:tasks.rados.rados.0.trial178.stdout:3716: writing trial17818531-49 from 3005997 to 3692577 tid 3 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:3717: rollback oid 23 current snap is 379 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 23 to 377 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:3716: finishing write tid 1 to trial17818531-49 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:3716: finishing write tid 2 to trial17818531-49 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:3718: write oid 26 current snap is 379 2026-02-21T22:31:49.251 INFO:tasks.rados.rados.0.trial178.stdout:3718: seq_num 1300 ranges {649100=403710,1606356=501788,2633420=497852,3636005=213095} 2026-02-21T22:31:49.257 INFO:tasks.rados.rados.0.trial178.stdout:3718: writing trial17818531-26 from 649100 to 1052810 tid 1 2026-02-21T22:31:49.264 INFO:tasks.rados.rados.0.trial178.stdout:3718: writing trial17818531-26 from 1606356 to 2108144 tid 2 2026-02-21T22:31:49.271 INFO:tasks.rados.rados.0.trial178.stdout:3718: writing trial17818531-26 from 2633420 to 3131272 tid 3 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3718: writing trial17818531-26 from 3636005 to 3849100 tid 4 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3719: rollback oid 44 current snap is 379 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 371 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3716: finishing write tid 3 to trial17818531-49 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3716: finishing write tid 4 to trial17818531-49 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3716: finishing write tid 5 to trial17818531-49 2026-02-21T22:31:49.275 INFO:tasks.rados.rados.0.trial178.stdout:3716: oid 49 updating version 0 to 1658 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3716: oid 49 version 1658 is already newer than 1657 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3716: oid 49 version 1658 is already newer than 1655 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3716: oid 49 version 1658 is already newer than 1656 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1658 (ObjNum 1299 snap 379 seq_num 1299) dirty exists 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3716: left oid 49 (ObjNum 1299 snap 379 seq_num 1299) 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3717: finishing rollback tid 0 to trial17818531-23 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3716: done (3 left) 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3720: copy_from oid 16 from oid 17 current snap is 379 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3717: finishing rollback tid 1 to trial17818531-23 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3717: finishing rollback tid 2 to trial17818531-23 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1430 (ObjNum 1269 snap 362 seq_num 1269) dirty exists 2026-02-21T22:31:49.276 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 1 to trial17818531-26 2026-02-21T22:31:49.277 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 2 to trial17818531-26 2026-02-21T22:31:49.277 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 3 to trial17818531-26 2026-02-21T22:31:49.277 INFO:tasks.rados.rados.0.trial178.stdout:3717: done (3 left) 2026-02-21T22:31:49.277 INFO:tasks.rados.rados.0.trial178.stdout:3721: delete oid 29 current snap is 379 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 4 to trial17818531-26 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 5 to trial17818531-26 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: finishing write tid 6 to trial17818531-26 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: oid 26 updating version 0 to 1660 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: oid 26 version 1660 is already newer than 1659 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: oid 26 updating version 1660 to 1661 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: oid 26 version 1661 is already newer than 1657 2026-02-21T22:31:49.278 INFO:tasks.rados.rados.0.trial178.stdout:3718: oid 26 version 1661 is already newer than 1658 2026-02-21T22:31:49.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1661 (ObjNum 1300 snap 379 seq_num 1300) dirty exists 2026-02-21T22:31:49.279 INFO:tasks.rados.rados.0.trial178.stdout:3718: left oid 26 (ObjNum 1300 snap 379 seq_num 1300) 2026-02-21T22:31:49.279 INFO:tasks.rados.rados.0.trial178.stdout:3719: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:31:49.279 INFO:tasks.rados.rados.0.trial178.stdout:3719: finishing rollback tid 2 to trial17818531-44 2026-02-21T22:31:49.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1856 (ObjNum 1260 snap 359 seq_num 1260) dirty exists 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3718: done (3 left) 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3719: done (2 left) 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3721: done (1 left) 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3722: copy_from oid 11 from oid 10 current snap is 379 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3723: rollback oid 38 current snap is 379 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 38 to 370 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3724: read oid 15 snap -1 2026-02-21T22:31:49.280 INFO:tasks.rados.rados.0.trial178.stdout:3724: expect deleted 2026-02-21T22:31:49.281 INFO:tasks.rados.rados.0.trial178.stdout:3725: snap_remove snap 377 2026-02-21T22:31:50.231 INFO:tasks.rados.rados.0.trial178.stdout:3725: done (4 left) 2026-02-21T22:31:50.231 INFO:tasks.rados.rados.0.trial178.stdout:3726: snap_remove snap 365 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3726: done (4 left) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3727: read oid 29 snap -1 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3727: expect deleted 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3728: copy_from oid 49 from oid 21 current snap is 379 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3722: finishing copy_from to trial17818531-11 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3722: got expected ENOENT (src dne) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3722: finishing copy_from racing read to trial17818531-11 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3723: finishing rollback tid 1 to trial17818531-38 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3723: finishing rollback tid 2 to trial17818531-38 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 38 v 1818 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3720: finishing copy_from racing read to trial17818531-16 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3720: finishing copy_from to trial17818531-16 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1739 (ObjNum 1244 snap 356 seq_num 1244) dirty exists 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3720: done (5 left) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3722: done (4 left) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3723: done (3 left) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3724: done (2 left) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3729: read oid 47 snap -1 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3729: expect (ObjNum 1246 snap 357 seq_num 1246) 2026-02-21T22:31:51.232 INFO:tasks.rados.rados.0.trial178.stdout:3730: rollback oid 32 current snap is 379 2026-02-21T22:31:51.233 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 378 2026-02-21T22:31:51.233 INFO:tasks.rados.rados.0.trial178.stdout:3731: copy_from oid 5 from oid 45 current snap is 379 2026-02-21T22:31:51.233 INFO:tasks.rados.rados.0.trial178.stdout:3732: read oid 17 snap 371 2026-02-21T22:31:51.233 INFO:tasks.rados.rados.0.trial178.stdout:3732: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:31:51.233 INFO:tasks.rados.rados.0.trial178.stdout:3733: snap_create 2026-02-21T22:31:51.239 INFO:tasks.rados.rados.0.trial178.stdout:3730: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:31:51.239 INFO:tasks.rados.rados.0.trial178.stdout:3730: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:31:51.239 INFO:tasks.rados.rados.0.trial178.stdout:3730: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:31:51.239 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1803 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:31:51.240 INFO:tasks.rados.rados.0.trial178.stdout:3732: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:31:51.319 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2493681 2026-02-21T22:31:51.358 INFO:tasks.rados.rados.0.trial178.stdout:3729: expect (ObjNum 1246 snap 357 seq_num 1246) 2026-02-21T22:31:51.358 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:51.398 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:51.438 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:51.477 INFO:tasks.rados.rados.0.trial178.stdout:3731: finishing copy_from racing read to trial17818531-5 2026-02-21T22:31:51.477 INFO:tasks.rados.rados.0.trial178.stdout:3731: finishing copy_from to trial17818531-5 2026-02-21T22:31:51.477 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1655 (ObjNum 1167 snap 334 seq_num 1167) dirty exists 2026-02-21T22:31:51.477 INFO:tasks.rados.rados.0.trial178.stdout:3728: finishing copy_from racing read to trial17818531-49 2026-02-21T22:31:51.478 INFO:tasks.rados.rados.0.trial178.stdout:3728: finishing copy_from to trial17818531-49 2026-02-21T22:31:51.478 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1985 (ObjNum 1175 snap 336 seq_num 1175) dirty exists 2026-02-21T22:31:52.222 INFO:tasks.rados.rados.0.trial178.stdout:3727: done (6 left) 2026-02-21T22:31:52.222 INFO:tasks.rados.rados.0.trial178.stdout:3728: done (5 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3729: done (4 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3730: done (3 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3731: done (2 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3732: done (1 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3733: done (0 left) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3734: read oid 4 snap -1 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3734: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:31:52.223 INFO:tasks.rados.rados.0.trial178.stdout:3735: delete oid 14 current snap is 380 2026-02-21T22:31:52.225 INFO:tasks.rados.rados.0.trial178.stdout:3735: done (1 left) 2026-02-21T22:31:52.225 INFO:tasks.rados.rados.0.trial178.stdout:3736: snap_create 2026-02-21T22:31:52.229 INFO:tasks.rados.rados.0.trial178.stdout:3734: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:31:52.245 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2521464 2026-02-21T22:31:53.225 INFO:tasks.rados.rados.0.trial178.stdout:3734: done (1 left) 2026-02-21T22:31:53.225 INFO:tasks.rados.rados.0.trial178.stdout:3736: done (0 left) 2026-02-21T22:31:53.225 INFO:tasks.rados.rados.0.trial178.stdout:3737: snap_remove snap 375 2026-02-21T22:31:53.542 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:53.542 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:53.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:53.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:31:53.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:53.555 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:31:53.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:53.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:54.226 INFO:tasks.rados.rados.0.trial178.stdout:3737: done (0 left) 2026-02-21T22:31:54.226 INFO:tasks.rados.rados.0.trial178.stdout:3738: delete oid 6 current snap is 381 2026-02-21T22:31:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3738: done (0 left) 2026-02-21T22:31:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3739: write oid 47 current snap is 381 2026-02-21T22:31:54.230 INFO:tasks.rados.rados.0.trial178.stdout:3739: seq_num 1304 ranges {716328=421059,1840284=509126,3143112=659293} 2026-02-21T22:31:54.234 INFO:tasks.rados.rados.0.trial178.stdout:3739: writing trial17818531-47 from 716328 to 1137387 tid 1 2026-02-21T22:31:54.238 INFO:tasks.rados.rados.0.trial178.stdout:3739: writing trial17818531-47 from 1840284 to 2349410 tid 2 2026-02-21T22:31:54.244 INFO:tasks.rados.rados.0.trial178.stdout:3739: writing trial17818531-47 from 3143112 to 3802405 tid 3 2026-02-21T22:31:54.244 INFO:tasks.rados.rados.0.trial178.stdout:3740: write (excl) oid 43 current snap is 381 2026-02-21T22:31:54.244 INFO:tasks.rados.rados.0.trial178.stdout:3740: seq_num 1305 ranges {733135=496308,1898766=652784,3170535=470565} 2026-02-21T22:31:54.248 INFO:tasks.rados.rados.0.trial178.stdout:3740: writing trial17818531-43 from 733135 to 1229443 tid 1 2026-02-21T22:31:54.253 INFO:tasks.rados.rados.0.trial178.stdout:3740: writing trial17818531-43 from 1898766 to 2551550 tid 2 2026-02-21T22:31:54.258 INFO:tasks.rados.rados.0.trial178.stdout:3740: writing trial17818531-43 from 3170535 to 3641100 tid 3 2026-02-21T22:31:54.258 INFO:tasks.rados.rados.0.trial178.stdout:3741: rollback oid 11 current snap is 381 2026-02-21T22:31:54.258 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 376 2026-02-21T22:31:54.258 INFO:tasks.rados.rados.0.trial178.stdout:3739: finishing write tid 1 to trial17818531-47 2026-02-21T22:31:54.258 INFO:tasks.rados.rados.0.trial178.stdout:3739: finishing write tid 2 to trial17818531-47 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: finishing write tid 3 to trial17818531-47 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: finishing write tid 4 to trial17818531-47 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: finishing write tid 5 to trial17818531-47 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: oid 47 updating version 0 to 1995 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: oid 47 version 1995 is already newer than 1993 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: oid 47 version 1995 is already newer than 1994 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: oid 47 updating version 1995 to 1996 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 1996 (ObjNum 1304 snap 381 seq_num 1304) dirty exists 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: left oid 47 (ObjNum 1304 snap 381 seq_num 1304) 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3740: finishing write tid 1 to trial17818531-43 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3740: finishing write tid 2 to trial17818531-43 2026-02-21T22:31:54.259 INFO:tasks.rados.rados.0.trial178.stdout:3739: done (2 left) 2026-02-21T22:31:54.260 INFO:tasks.rados.rados.0.trial178.stdout:3742: read oid 28 snap -1 2026-02-21T22:31:54.260 INFO:tasks.rados.rados.0.trial178.stdout:3742: expect deleted 2026-02-21T22:31:54.260 INFO:tasks.rados.rados.0.trial178.stdout:3743: delete oid 25 current snap is 381 2026-02-21T22:31:54.260 INFO:tasks.rados.rados.0.trial178.stdout:3740: finishing write tid 3 to trial17818531-43 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: finishing write tid 4 to trial17818531-43 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: finishing write tid 5 to trial17818531-43 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: oid 43 updating version 0 to 1614 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: oid 43 version 1614 is already newer than 1613 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: oid 43 version 1614 is already newer than 1612 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: oid 43 updating version 1614 to 1615 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1615 (ObjNum 1305 snap 381 seq_num 1305) dirty exists 2026-02-21T22:31:54.261 INFO:tasks.rados.rados.0.trial178.stdout:3740: left oid 43 (ObjNum 1305 snap 381 seq_num 1305) 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3741: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3740: done (3 left) 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3742: done (2 left) 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3743: done (1 left) 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3744: write oid 17 current snap is 381 2026-02-21T22:31:54.262 INFO:tasks.rados.rados.0.trial178.stdout:3744: seq_num 1307 ranges {766749=646806,1899377=656453,3109028=609462} 2026-02-21T22:31:54.269 INFO:tasks.rados.rados.0.trial178.stdout:3744: writing trial17818531-17 from 766749 to 1413555 tid 1 2026-02-21T22:31:54.273 INFO:tasks.rados.rados.0.trial178.stdout:3744: writing trial17818531-17 from 1899377 to 2555830 tid 2 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3744: writing trial17818531-17 from 3109028 to 3718490 tid 3 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3745: delete oid 22 current snap is 381 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3741: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3741: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1809 (ObjNum 1265 snap 362 seq_num 1265) dirty exists 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3744: finishing write tid 1 to trial17818531-17 2026-02-21T22:31:54.278 INFO:tasks.rados.rados.0.trial178.stdout:3744: finishing write tid 2 to trial17818531-17 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: finishing write tid 3 to trial17818531-17 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: finishing write tid 4 to trial17818531-17 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: finishing write tid 5 to trial17818531-17 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: oid 17 updating version 0 to 1746 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: oid 17 updating version 1746 to 1748 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: oid 17 version 1748 is already newer than 1745 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:3744: oid 17 version 1748 is already newer than 1747 2026-02-21T22:31:54.280 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1748 (ObjNum 1307 snap 381 seq_num 1307) dirty exists 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3744: left oid 17 (ObjNum 1307 snap 381 seq_num 1307) 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3741: done (2 left) 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3744: done (1 left) 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3745: done (0 left) 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3746: rollback oid 29 current snap is 381 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 370 2026-02-21T22:31:54.281 INFO:tasks.rados.rados.0.trial178.stdout:3747: delete oid 44 current snap is 381 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3746: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3747: done (1 left) 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3748: read oid 18 snap -1 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3748: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3749: snap_create 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:3746: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:31:54.283 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1636 (ObjNum 1261 snap 359 seq_num 1261) dirty exists 2026-02-21T22:31:54.285 INFO:tasks.rados.rados.0.trial178.stdout:3748: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:31:54.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2813318 2026-02-21T22:31:54.397 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1115258 2026-02-21T22:31:55.239 INFO:tasks.rados.rados.0.trial178.stdout:3746: done (2 left) 2026-02-21T22:31:55.239 INFO:tasks.rados.rados.0.trial178.stdout:3748: done (1 left) 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3749: done (0 left) 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3750: read oid 47 snap -1 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3750: expect (ObjNum 1304 snap 381 seq_num 1304) 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3751: copy_from oid 12 from oid 22 current snap is 382 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3752: rollback oid 34 current snap is 382 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 34 to 378 2026-02-21T22:31:55.240 INFO:tasks.rados.rados.0.trial178.stdout:3753: delete oid 36 current snap is 382 2026-02-21T22:31:55.241 INFO:tasks.rados.rados.0.trial178.stdout:3752: finishing rollback tid 1 to trial17818531-34 2026-02-21T22:31:55.241 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 99686288) dirty dne 2026-02-21T22:31:55.241 INFO:tasks.rados.rados.0.trial178.stdout:3751: finishing copy_from to trial17818531-12 2026-02-21T22:31:55.241 INFO:tasks.rados.rados.0.trial178.stdout:3751: got expected ENOENT (src dne) 2026-02-21T22:31:55.241 INFO:tasks.rados.rados.0.trial178.stdout:3751: finishing copy_from racing read to trial17818531-12 2026-02-21T22:31:55.248 INFO:tasks.rados.rados.0.trial178.stdout:3751: done (3 left) 2026-02-21T22:31:55.249 INFO:tasks.rados.rados.0.trial178.stdout:3752: done (2 left) 2026-02-21T22:31:55.249 INFO:tasks.rados.rados.0.trial178.stdout:3753: done (1 left) 2026-02-21T22:31:55.249 INFO:tasks.rados.rados.0.trial178.stdout:3754: read oid 26 snap -1 2026-02-21T22:31:55.249 INFO:tasks.rados.rados.0.trial178.stdout:3754: expect (ObjNum 1300 snap 379 seq_num 1300) 2026-02-21T22:31:55.249 INFO:tasks.rados.rados.0.trial178.stdout:3755: snap_create 2026-02-21T22:31:55.250 INFO:tasks.rados.rados.0.trial178.stdout:3750: expect (ObjNum 1304 snap 381 seq_num 1304) 2026-02-21T22:31:55.330 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:55.428 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3127404 2026-02-21T22:31:55.440 INFO:tasks.rados.rados.0.trial178.stdout:3754: expect (ObjNum 1300 snap 379 seq_num 1300) 2026-02-21T22:31:55.440 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:31:55.502 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2576802 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3750: done (2 left) 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3754: done (1 left) 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3755: done (0 left) 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3756: copy_from oid 7 from oid 38 current snap is 383 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3757: write oid 49 current snap is 383 2026-02-21T22:31:56.243 INFO:tasks.rados.rados.0.trial178.stdout:3757: seq_num 1311 ranges {433977=664155,1733305=300672} 2026-02-21T22:31:56.250 INFO:tasks.rados.rados.0.trial178.stdout:3757: writing trial17818531-49 from 433977 to 1098132 tid 1 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3757: writing trial17818531-49 from 1733305 to 2033977 tid 2 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3758: copy_from oid 8 from oid 32 current snap is 383 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3759: read oid 33 snap 376 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3759: expect (ObjNum 1275 snap 364 seq_num 1275) 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3760: write (excl) oid 31 current snap is 383 2026-02-21T22:31:56.257 INFO:tasks.rados.rados.0.trial178.stdout:3760: seq_num 1312 ranges {450784=739404,1791787=258997} 2026-02-21T22:31:56.265 INFO:tasks.rados.rados.0.trial178.stdout:3760: writing trial17818531-31 from 450784 to 1190188 tid 1 2026-02-21T22:31:56.271 INFO:tasks.rados.rados.0.trial178.stdout:3760: writing trial17818531-31 from 1791787 to 2050784 tid 2 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3761: rollback oid 42 current snap is 383 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 42 to 371 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: finishing write tid 1 to trial17818531-49 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: finishing write tid 2 to trial17818531-49 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: finishing write tid 3 to trial17818531-49 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: finishing write tid 4 to trial17818531-49 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: oid 49 updating version 0 to 1987 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: oid 49 updating version 1987 to 1989 2026-02-21T22:31:56.272 INFO:tasks.rados.rados.0.trial178.stdout:3757: oid 49 version 1989 is already newer than 1988 2026-02-21T22:31:56.273 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 1989 (ObjNum 1311 snap 383 seq_num 1311) dirty exists 2026-02-21T22:31:56.273 INFO:tasks.rados.rados.0.trial178.stdout:3757: left oid 49 (ObjNum 1311 snap 383 seq_num 1311) 2026-02-21T22:31:56.273 INFO:tasks.rados.rados.0.trial178.stdout:3759: expect (ObjNum 1275 snap 364 seq_num 1275) 2026-02-21T22:31:56.290 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1160758 2026-02-21T22:31:56.312 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2608032 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3756: finishing copy_from racing read to trial17818531-7 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3756: finishing copy_from to trial17818531-7 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 7 v 1818 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3756: done (5 left) 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3757: done (4 left) 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3759: done (3 left) 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3762: snap_create 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: finishing write tid 1 to trial17818531-31 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: finishing write tid 2 to trial17818531-31 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: finishing write tid 3 to trial17818531-31 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: finishing write tid 4 to trial17818531-31 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: oid 31 updating version 0 to 1990 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: oid 31 updating version 1990 to 1992 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: oid 31 version 1992 is already newer than 1991 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1992 (ObjNum 1312 snap 383 seq_num 1312) dirty exists 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3760: left oid 31 (ObjNum 1312 snap 383 seq_num 1312) 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3758: finishing copy_from racing read to trial17818531-8 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3758: finishing copy_from to trial17818531-8 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 1803 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3761: finishing rollback tid 0 to trial17818531-42 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3761: finishing rollback tid 1 to trial17818531-42 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:3761: finishing rollback tid 2 to trial17818531-42 2026-02-21T22:31:56.322 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 2003 (ObjNum 1259 snap 359 seq_num 1259) dirty exists 2026-02-21T22:31:57.223 INFO:tasks.rados.rados.0.trial178.stdout:3758: done (3 left) 2026-02-21T22:31:57.223 INFO:tasks.rados.rados.0.trial178.stdout:3760: done (2 left) 2026-02-21T22:31:57.223 INFO:tasks.rados.rados.0.trial178.stdout:3761: done (1 left) 2026-02-21T22:31:57.223 INFO:tasks.rados.rados.0.trial178.stdout:3762: done (0 left) 2026-02-21T22:31:57.223 INFO:tasks.rados.rados.0.trial178.stdout:3763: delete oid 14 current snap is 384 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3763: done (0 left) 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3764: read oid 3 snap -1 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3764: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3765: read oid 36 snap -1 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3765: expect deleted 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3766: rollback oid 10 current snap is 384 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 366 2026-02-21T22:31:57.227 INFO:tasks.rados.rados.0.trial178.stdout:3767: snap_remove snap 378 2026-02-21T22:31:58.227 INFO:tasks.rados.rados.0.trial178.stdout:3767: done (3 left) 2026-02-21T22:31:58.227 INFO:tasks.rados.rados.0.trial178.stdout:3768: delete oid 50 current snap is 384 2026-02-21T22:31:58.227 INFO:tasks.rados.rados.0.trial178.stdout:3766: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:31:58.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 0 (ObjNum 33 snap 0 seq_num 4122730944) dirty dne 2026-02-21T22:31:58.227 INFO:tasks.rados.rados.0.trial178.stdout:3764: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:31:58.279 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 790294 2026-02-21T22:31:58.326 INFO:tasks.rados.rados.0.trial178.stdout:3764: done (3 left) 2026-02-21T22:31:58.327 INFO:tasks.rados.rados.0.trial178.stdout:3765: done (2 left) 2026-02-21T22:31:58.327 INFO:tasks.rados.rados.0.trial178.stdout:3766: done (1 left) 2026-02-21T22:31:58.327 INFO:tasks.rados.rados.0.trial178.stdout:3768: done (0 left) 2026-02-21T22:31:58.327 INFO:tasks.rados.rados.0.trial178.stdout:3769: snap_create 2026-02-21T22:31:58.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:31:58.593 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:31:59.246 INFO:tasks.rados.rados.0.trial178.stdout:3769: done (0 left) 2026-02-21T22:31:59.246 INFO:tasks.rados.rados.0.trial178.stdout:3770: write oid 15 current snap is 385 2026-02-21T22:31:59.246 INFO:tasks.rados.rados.0.trial178.stdout:3770: seq_num 1315 ranges {501205=565151,1850880=250325} 2026-02-21T22:31:59.252 INFO:tasks.rados.rados.0.trial178.stdout:3770: writing trial17818531-15 from 501205 to 1066356 tid 1 2026-02-21T22:31:59.260 INFO:tasks.rados.rados.0.trial178.stdout:3770: writing trial17818531-15 from 1850880 to 2101205 tid 2 2026-02-21T22:31:59.260 INFO:tasks.rados.rados.0.trial178.stdout:3771: snap_remove snap 384 2026-02-21T22:32:00.246 INFO:tasks.rados.rados.0.trial178.stdout:3771: done (1 left) 2026-02-21T22:32:00.246 INFO:tasks.rados.rados.0.trial178.stdout:3772: rollback oid 31 current snap is 385 2026-02-21T22:32:00.246 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 31 to 371 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: finishing write tid 1 to trial17818531-15 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: finishing write tid 2 to trial17818531-15 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: finishing write tid 3 to trial17818531-15 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: finishing write tid 4 to trial17818531-15 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: oid 15 updating version 0 to 1749 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: oid 15 updating version 1749 to 1751 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: oid 15 version 1751 is already newer than 1750 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1751 (ObjNum 1315 snap 385 seq_num 1315) dirty exists 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: left oid 15 (ObjNum 1315 snap 385 seq_num 1315) 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3770: done (1 left) 2026-02-21T22:32:00.247 INFO:tasks.rados.rados.0.trial178.stdout:3773: snap_create 2026-02-21T22:32:00.251 INFO:tasks.rados.rados.0.trial178.stdout:3772: finishing rollback tid 0 to trial17818531-31 2026-02-21T22:32:00.251 INFO:tasks.rados.rados.0.trial178.stdout:3772: finishing rollback tid 1 to trial17818531-31 2026-02-21T22:32:00.251 INFO:tasks.rados.rados.0.trial178.stdout:3772: finishing rollback tid 2 to trial17818531-31 2026-02-21T22:32:00.251 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 31 v 1995 (ObjNum 1253 snap 358 seq_num 1253) dirty exists 2026-02-21T22:32:01.252 INFO:tasks.rados.rados.0.trial178.stdout:3772: done (1 left) 2026-02-21T22:32:01.252 INFO:tasks.rados.rados.0.trial178.stdout:3773: done (0 left) 2026-02-21T22:32:01.252 INFO:tasks.rados.rados.0.trial178.stdout:3774: write oid 45 current snap is 386 2026-02-21T22:32:01.252 INFO:tasks.rados.rados.0.trial178.stdout:3774: seq_num 1316 ranges {518012=756753,2025715=92297} 2026-02-21T22:32:01.260 INFO:tasks.rados.rados.0.trial178.stdout:3774: writing trial17818531-45 from 518012 to 1274765 tid 1 2026-02-21T22:32:01.266 INFO:tasks.rados.rados.0.trial178.stdout:3774: writing trial17818531-45 from 2025715 to 2118012 tid 2 2026-02-21T22:32:01.266 INFO:tasks.rados.rados.0.trial178.stdout:3775: write (excl) oid 48 current snap is 386 2026-02-21T22:32:01.266 INFO:tasks.rados.rados.0.trial178.stdout:3775: seq_num 1317 ranges {534819=432002,1684197=450622} 2026-02-21T22:32:01.272 INFO:tasks.rados.rados.0.trial178.stdout:3775: writing trial17818531-48 from 534819 to 966821 tid 1 2026-02-21T22:32:01.280 INFO:tasks.rados.rados.0.trial178.stdout:3775: writing trial17818531-48 from 1684197 to 2134819 tid 2 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3776: snap_create 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: finishing write tid 1 to trial17818531-45 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: finishing write tid 2 to trial17818531-45 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: finishing write tid 3 to trial17818531-45 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: finishing write tid 4 to trial17818531-45 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: oid 45 updating version 0 to 1657 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: oid 45 version 1657 is already newer than 1656 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: oid 45 updating version 1657 to 1658 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1658 (ObjNum 1316 snap 386 seq_num 1316) dirty exists 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3774: left oid 45 (ObjNum 1316 snap 386 seq_num 1316) 2026-02-21T22:32:01.281 INFO:tasks.rados.rados.0.trial178.stdout:3775: finishing write tid 1 to trial17818531-48 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: finishing write tid 2 to trial17818531-48 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: finishing write tid 3 to trial17818531-48 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: finishing write tid 4 to trial17818531-48 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: oid 48 updating version 0 to 1659 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: oid 48 updating version 1659 to 1660 2026-02-21T22:32:01.283 INFO:tasks.rados.rados.0.trial178.stdout:3775: oid 48 updating version 1660 to 1661 2026-02-21T22:32:01.284 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1661 (ObjNum 1317 snap 386 seq_num 1317) dirty exists 2026-02-21T22:32:01.284 INFO:tasks.rados.rados.0.trial178.stdout:3775: left oid 48 (ObjNum 1317 snap 386 seq_num 1317) 2026-02-21T22:32:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3774: done (2 left) 2026-02-21T22:32:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3775: done (1 left) 2026-02-21T22:32:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3776: done (0 left) 2026-02-21T22:32:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3777: write oid 39 current snap is 387 2026-02-21T22:32:02.223 INFO:tasks.rados.rados.0.trial178.stdout:3777: seq_num 1318 ranges {551626=507251,1626326=418456} 2026-02-21T22:32:02.230 INFO:tasks.rados.rados.0.trial178.stdout:3777: writing trial17818531-39 from 551626 to 1058877 tid 1 2026-02-21T22:32:02.237 INFO:tasks.rados.rados.0.trial178.stdout:3777: writing trial17818531-39 from 1626326 to 2044782 tid 2 2026-02-21T22:32:02.237 INFO:tasks.rados.rados.0.trial178.stdout:3778: write oid 35 current snap is 387 2026-02-21T22:32:02.237 INFO:tasks.rados.rados.0.trial178.stdout:3778: seq_num 1319 ranges {568433=582500,1684808=483625} 2026-02-21T22:32:02.244 INFO:tasks.rados.rados.0.trial178.stdout:3778: writing trial17818531-35 from 568433 to 1150933 tid 1 2026-02-21T22:32:02.252 INFO:tasks.rados.rados.0.trial178.stdout:3778: writing trial17818531-35 from 1684808 to 2168433 tid 2 2026-02-21T22:32:02.252 INFO:tasks.rados.rados.0.trial178.stdout:3779: snap_remove snap 381 2026-02-21T22:32:03.257 INFO:tasks.rados.rados.0.trial178.stdout:3779: done (2 left) 2026-02-21T22:32:03.257 INFO:tasks.rados.rados.0.trial178.stdout:3780: snap_remove snap 383 2026-02-21T22:32:03.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:03.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:04.260 INFO:tasks.rados.rados.0.trial178.stdout:3780: done (2 left) 2026-02-21T22:32:04.260 INFO:tasks.rados.rados.0.trial178.stdout:3781: snap_create 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: finishing write tid 1 to trial17818531-39 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: finishing write tid 2 to trial17818531-39 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: finishing write tid 3 to trial17818531-39 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: finishing write tid 4 to trial17818531-39 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: oid 39 updating version 0 to 1652 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: oid 39 version 1652 is already newer than 1651 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: oid 39 updating version 1652 to 1653 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 1653 (ObjNum 1318 snap 387 seq_num 1318) dirty exists 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3777: left oid 39 (ObjNum 1318 snap 387 seq_num 1318) 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3778: finishing write tid 1 to trial17818531-35 2026-02-21T22:32:04.261 INFO:tasks.rados.rados.0.trial178.stdout:3778: finishing write tid 2 to trial17818531-35 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: finishing write tid 3 to trial17818531-35 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: finishing write tid 4 to trial17818531-35 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: oid 35 updating version 0 to 1815 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: oid 35 updating version 1815 to 1816 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: oid 35 updating version 1816 to 1817 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 35 v 1817 (ObjNum 1319 snap 387 seq_num 1319) dirty exists 2026-02-21T22:32:04.262 INFO:tasks.rados.rados.0.trial178.stdout:3778: left oid 35 (ObjNum 1319 snap 387 seq_num 1319) 2026-02-21T22:32:05.264 INFO:tasks.rados.rados.0.trial178.stdout:3777: done (2 left) 2026-02-21T22:32:05.265 INFO:tasks.rados.rados.0.trial178.stdout:3778: done (1 left) 2026-02-21T22:32:05.265 INFO:tasks.rados.rados.0.trial178.stdout:3781: done (0 left) 2026-02-21T22:32:05.265 INFO:tasks.rados.rados.0.trial178.stdout:3782: snap_remove snap 386 2026-02-21T22:32:05.272 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:32:05.273 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:32:05.274 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:32:05.275 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:32:05.276 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:32:05.277 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:32:06.264 INFO:tasks.rados.rados.0.trial178.stdout:3782: done (0 left) 2026-02-21T22:32:06.265 INFO:tasks.rados.rados.0.trial178.stdout:3783: write oid 9 current snap is 388 2026-02-21T22:32:06.265 INFO:tasks.rados.rados.0.trial178.stdout:3783: seq_num 1320 ranges {585240=657749,1743290=422125} 2026-02-21T22:32:06.273 INFO:tasks.rados.rados.0.trial178.stdout:3783: writing trial17818531-9 from 585240 to 1242989 tid 1 2026-02-21T22:32:06.279 INFO:tasks.rados.rados.0.trial178.stdout:3783: writing trial17818531-9 from 1743290 to 2165415 tid 2 2026-02-21T22:32:06.279 INFO:tasks.rados.rados.0.trial178.stdout:3784: write (excl) oid 33 current snap is 388 2026-02-21T22:32:06.280 INFO:tasks.rados.rados.0.trial178.stdout:3784: seq_num 1321 ranges {602047=732998,1801772=400275} 2026-02-21T22:32:06.288 INFO:tasks.rados.rados.0.trial178.stdout:3784: writing trial17818531-33 from 602047 to 1335045 tid 1 2026-02-21T22:32:06.294 INFO:tasks.rados.rados.0.trial178.stdout:3784: writing trial17818531-33 from 1801772 to 2202047 tid 2 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3785: delete oid 22 current snap is 388 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: finishing write tid 1 to trial17818531-9 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: finishing write tid 2 to trial17818531-9 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: finishing write tid 3 to trial17818531-9 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: finishing write tid 4 to trial17818531-9 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: oid 9 updating version 0 to 1655 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: oid 9 version 1655 is already newer than 1654 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: oid 9 updating version 1655 to 1656 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 1656 (ObjNum 1320 snap 388 seq_num 1320) dirty exists 2026-02-21T22:32:06.295 INFO:tasks.rados.rados.0.trial178.stdout:3783: left oid 9 (ObjNum 1320 snap 388 seq_num 1320) 2026-02-21T22:32:06.296 INFO:tasks.rados.rados.0.trial178.stdout:3784: finishing write tid 1 to trial17818531-33 2026-02-21T22:32:06.296 INFO:tasks.rados.rados.0.trial178.stdout:3783: done (2 left) 2026-02-21T22:32:06.297 INFO:tasks.rados.rados.0.trial178.stdout:3785: done (1 left) 2026-02-21T22:32:06.297 INFO:tasks.rados.rados.0.trial178.stdout:3786: read oid 40 snap 379 2026-02-21T22:32:06.297 INFO:tasks.rados.rados.0.trial178.stdout:3786: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:32:06.297 INFO:tasks.rados.rados.0.trial178.stdout:3787: write (excl) oid 22 current snap is 388 2026-02-21T22:32:06.297 INFO:tasks.rados.rados.0.trial178.stdout:3787: seq_num 1323 ranges {635661=599849,1918736=316925} 2026-02-21T22:32:06.305 INFO:tasks.rados.rados.0.trial178.stdout:3787: writing trial17818531-22 from 635661 to 1235510 tid 1 2026-02-21T22:32:06.312 INFO:tasks.rados.rados.0.trial178.stdout:3787: writing trial17818531-22 from 1918736 to 2235661 tid 2 2026-02-21T22:32:06.312 INFO:tasks.rados.rados.0.trial178.stdout:3788: rollback oid 5 current snap is 388 2026-02-21T22:32:06.312 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 5 to 363 2026-02-21T22:32:06.312 INFO:tasks.rados.rados.0.trial178.stdout:3784: finishing write tid 2 to trial17818531-33 2026-02-21T22:32:06.312 INFO:tasks.rados.rados.0.trial178.stdout:3784: finishing write tid 3 to trial17818531-33 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3784: finishing write tid 4 to trial17818531-33 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3784: oid 33 updating version 0 to 1821 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3784: oid 33 version 1821 is already newer than 1820 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3784: oid 33 version 1821 is already newer than 1819 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1821 (ObjNum 1321 snap 388 seq_num 1321) dirty exists 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3784: left oid 33 (ObjNum 1321 snap 388 seq_num 1321) 2026-02-21T22:32:06.313 INFO:tasks.rados.rados.0.trial178.stdout:3786: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:32:06.381 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2105925 2026-02-21T22:32:06.401 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:06.442 INFO:tasks.rados.rados.0.trial178.stdout:3787: finishing write tid 1 to trial17818531-22 2026-02-21T22:32:06.442 INFO:tasks.rados.rados.0.trial178.stdout:3787: finishing write tid 2 to trial17818531-22 2026-02-21T22:32:06.442 INFO:tasks.rados.rados.0.trial178.stdout:3787: finishing write tid 3 to trial17818531-22 2026-02-21T22:32:06.442 INFO:tasks.rados.rados.0.trial178.stdout:3787: finishing write tid 4 to trial17818531-22 2026-02-21T22:32:06.442 INFO:tasks.rados.rados.0.trial178.stdout:3787: oid 22 updating version 0 to 1657 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3787: oid 22 updating version 1657 to 1658 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3787: oid 22 updating version 1658 to 1659 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1659 (ObjNum 1323 snap 388 seq_num 1323) dirty exists 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3787: left oid 22 (ObjNum 1323 snap 388 seq_num 1323) 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3788: finishing rollback tid 0 to trial17818531-5 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3788: finishing rollback tid 1 to trial17818531-5 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3788: finishing rollback tid 2 to trial17818531-5 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1658 (ObjNum 997 snap 282 seq_num 997) dirty exists 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3784: done (3 left) 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3786: done (2 left) 2026-02-21T22:32:06.443 INFO:tasks.rados.rados.0.trial178.stdout:3787: done (1 left) 2026-02-21T22:32:06.444 INFO:tasks.rados.rados.0.trial178.stdout:3788: done (0 left) 2026-02-21T22:32:06.444 INFO:tasks.rados.rados.0.trial178.stdout:3789: write oid 6 current snap is 388 2026-02-21T22:32:06.444 INFO:tasks.rados.rados.0.trial178.stdout:3789: seq_num 1324 ranges {652468=675098,1977218=275250} 2026-02-21T22:32:06.451 INFO:tasks.rados.rados.0.trial178.stdout:3789: writing trial17818531-6 from 652468 to 1327566 tid 1 2026-02-21T22:32:06.458 INFO:tasks.rados.rados.0.trial178.stdout:3789: writing trial17818531-6 from 1977218 to 2252468 tid 2 2026-02-21T22:32:06.458 INFO:tasks.rados.rados.0.trial178.stdout:3790: read oid 49 snap -1 2026-02-21T22:32:06.458 INFO:tasks.rados.rados.0.trial178.stdout:3790: expect (ObjNum 1311 snap 383 seq_num 1311) 2026-02-21T22:32:06.458 INFO:tasks.rados.rados.0.trial178.stdout:3791: delete oid 28 current snap is 388 2026-02-21T22:32:06.458 INFO:tasks.rados.rados.0.trial178.stdout:3789: finishing write tid 1 to trial17818531-6 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: finishing write tid 2 to trial17818531-6 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: finishing write tid 3 to trial17818531-6 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: finishing write tid 4 to trial17818531-6 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: oid 6 updating version 0 to 1664 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: oid 6 version 1664 is already newer than 1663 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: oid 6 version 1664 is already newer than 1662 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1664 (ObjNum 1324 snap 388 seq_num 1324) dirty exists 2026-02-21T22:32:06.460 INFO:tasks.rados.rados.0.trial178.stdout:3789: left oid 6 (ObjNum 1324 snap 388 seq_num 1324) 2026-02-21T22:32:06.461 INFO:tasks.rados.rados.0.trial178.stdout:3789: done (2 left) 2026-02-21T22:32:06.461 INFO:tasks.rados.rados.0.trial178.stdout:3791: done (1 left) 2026-02-21T22:32:06.461 INFO:tasks.rados.rados.0.trial178.stdout:3792: snap_create 2026-02-21T22:32:06.463 INFO:tasks.rados.rados.0.trial178.stdout:3790: expect (ObjNum 1311 snap 383 seq_num 1311) 2026-02-21T22:32:06.510 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:07.225 INFO:tasks.rados.rados.0.trial178.stdout:3790: done (1 left) 2026-02-21T22:32:07.225 INFO:tasks.rados.rados.0.trial178.stdout:3792: done (0 left) 2026-02-21T22:32:07.225 INFO:tasks.rados.rados.0.trial178.stdout:3793: read oid 47 snap -1 2026-02-21T22:32:07.225 INFO:tasks.rados.rados.0.trial178.stdout:3793: expect (ObjNum 1304 snap 381 seq_num 1304) 2026-02-21T22:32:07.225 INFO:tasks.rados.rados.0.trial178.stdout:3794: snap_create 2026-02-21T22:32:07.242 INFO:tasks.rados.rados.0.trial178.stdout:3793: expect (ObjNum 1304 snap 381 seq_num 1304) 2026-02-21T22:32:07.242 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:07.302 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:08.229 INFO:tasks.rados.rados.0.trial178.stdout:3793: done (1 left) 2026-02-21T22:32:08.229 INFO:tasks.rados.rados.0.trial178.stdout:3794: done (0 left) 2026-02-21T22:32:08.229 INFO:tasks.rados.rados.0.trial178.stdout:3795: write oid 42 current snap is 390 2026-02-21T22:32:08.229 INFO:tasks.rados.rados.0.trial178.stdout:3795: seq_num 1326 ranges {686082=425596,1577829=433132} 2026-02-21T22:32:08.236 INFO:tasks.rados.rados.0.trial178.stdout:3795: writing trial17818531-42 from 686082 to 1111678 tid 1 2026-02-21T22:32:08.242 INFO:tasks.rados.rados.0.trial178.stdout:3795: writing trial17818531-42 from 1577829 to 2010961 tid 2 2026-02-21T22:32:08.242 INFO:tasks.rados.rados.0.trial178.stdout:3796: delete oid 21 current snap is 390 2026-02-21T22:32:08.242 INFO:tasks.rados.rados.0.trial178.stdout:3795: finishing write tid 1 to trial17818531-42 2026-02-21T22:32:08.247 INFO:tasks.rados.rados.0.trial178.stdout:3795: finishing write tid 2 to trial17818531-42 2026-02-21T22:32:08.247 INFO:tasks.rados.rados.0.trial178.stdout:3795: finishing write tid 3 to trial17818531-42 2026-02-21T22:32:08.248 INFO:tasks.rados.rados.0.trial178.stdout:3795: finishing write tid 4 to trial17818531-42 2026-02-21T22:32:08.248 INFO:tasks.rados.rados.0.trial178.stdout:3795: oid 42 updating version 0 to 2015 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3795: oid 42 updating version 2015 to 2017 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3795: oid 42 version 2017 is already newer than 2016 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 42 v 2017 (ObjNum 1326 snap 390 seq_num 1326) dirty exists 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3795: left oid 42 (ObjNum 1326 snap 390 seq_num 1326) 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3795: done (1 left) 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3796: done (0 left) 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3797: write oid 43 current snap is 390 2026-02-21T22:32:08.249 INFO:tasks.rados.rados.0.trial178.stdout:3797: seq_num 1328 ranges {719696=576094,2094793=224903} 2026-02-21T22:32:08.257 INFO:tasks.rados.rados.0.trial178.stdout:3797: writing trial17818531-43 from 719696 to 1295790 tid 1 2026-02-21T22:32:08.263 INFO:tasks.rados.rados.0.trial178.stdout:3797: writing trial17818531-43 from 2094793 to 2319696 tid 2 2026-02-21T22:32:08.263 INFO:tasks.rados.rados.0.trial178.stdout:3798: delete oid 19 current snap is 390 2026-02-21T22:32:08.263 INFO:tasks.rados.rados.0.trial178.stdout:3797: finishing write tid 1 to trial17818531-43 2026-02-21T22:32:08.265 INFO:tasks.rados.rados.0.trial178.stdout:3797: finishing write tid 2 to trial17818531-43 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: finishing write tid 3 to trial17818531-43 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: finishing write tid 4 to trial17818531-43 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: oid 43 updating version 0 to 1618 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: oid 43 version 1618 is already newer than 1617 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: oid 43 version 1618 is already newer than 1616 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1618 (ObjNum 1328 snap 390 seq_num 1328) dirty exists 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: left oid 43 (ObjNum 1328 snap 390 seq_num 1328) 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3797: done (1 left) 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3798: done (0 left) 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3799: copy_from oid 20 from oid 32 current snap is 390 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3800: write (excl) oid 27 current snap is 390 2026-02-21T22:32:08.266 INFO:tasks.rados.rados.0.trial178.stdout:3800: seq_num 1330 ranges {753310=726592,2095404=257906} 2026-02-21T22:32:08.277 INFO:tasks.rados.rados.0.trial178.stdout:3800: writing trial17818531-27 from 753310 to 1479902 tid 1 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:3800: writing trial17818531-27 from 2095404 to 2353310 tid 2 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:3801: snap_create 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:3799: finishing copy_from racing read to trial17818531-20 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:3799: finishing copy_from to trial17818531-20 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1803 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:32:08.284 INFO:tasks.rados.rados.0.trial178.stdout:3800: finishing write tid 1 to trial17818531-27 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: finishing write tid 2 to trial17818531-27 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: finishing write tid 3 to trial17818531-27 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: finishing write tid 4 to trial17818531-27 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: oid 27 updating version 0 to 1658 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: oid 27 updating version 1658 to 1659 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: oid 27 version 1659 is already newer than 1657 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 1659 (ObjNum 1330 snap 390 seq_num 1330) dirty exists 2026-02-21T22:32:08.287 INFO:tasks.rados.rados.0.trial178.stdout:3800: left oid 27 (ObjNum 1330 snap 390 seq_num 1330) 2026-02-21T22:32:08.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:08.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:09.269 INFO:tasks.rados.rados.0.trial178.stdout:3799: done (2 left) 2026-02-21T22:32:09.269 INFO:tasks.rados.rados.0.trial178.stdout:3800: done (1 left) 2026-02-21T22:32:09.269 INFO:tasks.rados.rados.0.trial178.stdout:3801: done (0 left) 2026-02-21T22:32:09.269 INFO:tasks.rados.rados.0.trial178.stdout:3802: write oid 10 current snap is 391 2026-02-21T22:32:09.269 INFO:tasks.rados.rados.0.trial178.stdout:3802: seq_num 1331 ranges {770117=518194,1870239=499878} 2026-02-21T22:32:09.278 INFO:tasks.rados.rados.0.trial178.stdout:3802: writing trial17818531-10 from 770117 to 1288311 tid 1 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:3802: writing trial17818531-10 from 1870239 to 2370117 tid 2 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:3803: rollback oid 28 current snap is 391 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 376 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:3802: finishing write tid 1 to trial17818531-10 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:3804: copy_from oid 17 from oid 7 current snap is 391 2026-02-21T22:32:09.286 INFO:tasks.rados.rados.0.trial178.stdout:3805: snap_create 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3802: finishing write tid 2 to trial17818531-10 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3802: finishing write tid 3 to trial17818531-10 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3803: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1816 (ObjNum 273 snap 0 seq_num 0) dirty dne 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3802: finishing write tid 4 to trial17818531-10 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3802: oid 10 updating version 0 to 1666 2026-02-21T22:32:09.288 INFO:tasks.rados.rados.0.trial178.stdout:3802: oid 10 updating version 1666 to 1667 2026-02-21T22:32:09.289 INFO:tasks.rados.rados.0.trial178.stdout:3802: oid 10 version 1667 is already newer than 1665 2026-02-21T22:32:09.289 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1667 (ObjNum 1331 snap 391 seq_num 1331) dirty exists 2026-02-21T22:32:09.289 INFO:tasks.rados.rados.0.trial178.stdout:3802: left oid 10 (ObjNum 1331 snap 391 seq_num 1331) 2026-02-21T22:32:09.302 INFO:tasks.rados.rados.0.trial178.stdout:3804: finishing copy_from racing read to trial17818531-17 2026-02-21T22:32:09.304 INFO:tasks.rados.rados.0.trial178.stdout:3804: finishing copy_from to trial17818531-17 2026-02-21T22:32:09.304 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 17 v 1818 (ObjNum 1179 snap 336 seq_num 1179) dirty exists 2026-02-21T22:32:10.274 INFO:tasks.rados.rados.0.trial178.stdout:3802: done (3 left) 2026-02-21T22:32:10.275 INFO:tasks.rados.rados.0.trial178.stdout:3803: done (2 left) 2026-02-21T22:32:10.275 INFO:tasks.rados.rados.0.trial178.stdout:3804: done (1 left) 2026-02-21T22:32:10.275 INFO:tasks.rados.rados.0.trial178.stdout:3805: done (0 left) 2026-02-21T22:32:10.275 INFO:tasks.rados.rados.0.trial178.stdout:3806: delete oid 40 current snap is 392 2026-02-21T22:32:10.279 INFO:tasks.rados.rados.0.trial178.stdout:3806: done (0 left) 2026-02-21T22:32:10.279 INFO:tasks.rados.rados.0.trial178.stdout:3807: write oid 22 current snap is 392 2026-02-21T22:32:10.279 INFO:tasks.rados.rados.0.trial178.stdout:3807: seq_num 1333 ranges {403731=668692,1587203=587797} 2026-02-21T22:32:10.286 INFO:tasks.rados.rados.0.trial178.stdout:3807: writing trial17818531-22 from 403731 to 1072423 tid 1 2026-02-21T22:32:10.293 INFO:tasks.rados.rados.0.trial178.stdout:3807: writing trial17818531-22 from 1587203 to 2175000 tid 2 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3808: read oid 36 snap -1 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3808: expect deleted 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3809: read oid 10 snap -1 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3809: expect (ObjNum 1331 snap 391 seq_num 1331) 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3810: write (excl) oid 15 current snap is 392 2026-02-21T22:32:10.294 INFO:tasks.rados.rados.0.trial178.stdout:3810: seq_num 1334 ranges {420538=743941,1929332=491206} 2026-02-21T22:32:10.302 INFO:tasks.rados.rados.0.trial178.stdout:3810: writing trial17818531-15 from 420538 to 1164479 tid 1 2026-02-21T22:32:10.312 INFO:tasks.rados.rados.0.trial178.stdout:3810: writing trial17818531-15 from 1929332 to 2420538 tid 2 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3811: rollback oid 32 current snap is 392 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 32 to 387 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: finishing write tid 1 to trial17818531-22 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: finishing write tid 2 to trial17818531-22 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: finishing write tid 3 to trial17818531-22 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: finishing write tid 4 to trial17818531-22 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: oid 22 updating version 0 to 1661 2026-02-21T22:32:10.313 INFO:tasks.rados.rados.0.trial178.stdout:3807: oid 22 updating version 1661 to 1662 2026-02-21T22:32:10.314 INFO:tasks.rados.rados.0.trial178.stdout:3807: oid 22 updating version 1662 to 1663 2026-02-21T22:32:10.314 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1663 (ObjNum 1333 snap 392 seq_num 1333) dirty exists 2026-02-21T22:32:10.314 INFO:tasks.rados.rados.0.trial178.stdout:3807: left oid 22 (ObjNum 1333 snap 392 seq_num 1333) 2026-02-21T22:32:10.314 INFO:tasks.rados.rados.0.trial178.stdout:3809: expect (ObjNum 1331 snap 391 seq_num 1331) 2026-02-21T22:32:10.342 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:10.374 INFO:tasks.rados.rados.0.trial178.stdout:3810: finishing write tid 1 to trial17818531-15 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: finishing write tid 2 to trial17818531-15 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: finishing write tid 3 to trial17818531-15 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3811: finishing rollback tid 0 to trial17818531-32 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3811: finishing rollback tid 1 to trial17818531-32 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: finishing write tid 4 to trial17818531-15 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: oid 15 updating version 0 to 1821 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: oid 15 version 1821 is already newer than 1819 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: oid 15 version 1821 is already newer than 1820 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1821 (ObjNum 1334 snap 392 seq_num 1334) dirty exists 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3810: left oid 15 (ObjNum 1334 snap 392 seq_num 1334) 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3811: finishing rollback tid 2 to trial17818531-32 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1827 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3807: done (4 left) 2026-02-21T22:32:10.375 INFO:tasks.rados.rados.0.trial178.stdout:3808: done (3 left) 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3809: done (2 left) 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3810: done (1 left) 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3811: done (0 left) 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3812: rollback oid 50 current snap is 392 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 50 to 390 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3813: write (excl) oid 44 current snap is 392 2026-02-21T22:32:10.376 INFO:tasks.rados.rados.0.trial178.stdout:3813: seq_num 1335 ranges {437345=419190,1587814=591466} 2026-02-21T22:32:10.380 INFO:tasks.rados.rados.0.trial178.stdout:3813: writing trial17818531-44 from 437345 to 856535 tid 1 2026-02-21T22:32:10.389 INFO:tasks.rados.rados.0.trial178.stdout:3813: writing trial17818531-44 from 1587814 to 2179280 tid 2 2026-02-21T22:32:10.389 INFO:tasks.rados.rados.0.trial178.stdout:3814: write oid 21 current snap is 392 2026-02-21T22:32:10.390 INFO:tasks.rados.rados.0.trial178.stdout:3812: finishing rollback tid 1 to trial17818531-50 2026-02-21T22:32:10.390 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1656 (ObjNum 257 snap 0 seq_num 100575248) dirty dne 2026-02-21T22:32:10.390 INFO:tasks.rados.rados.0.trial178.stdout:3813: finishing write tid 1 to trial17818531-44 2026-02-21T22:32:10.390 INFO:tasks.rados.rados.0.trial178.stdout:3814: seq_num 1336 ranges {454152=494439,1646296=735124} 2026-02-21T22:32:10.396 INFO:tasks.rados.rados.0.trial178.stdout:3814: writing trial17818531-21 from 454152 to 948591 tid 1 2026-02-21T22:32:10.405 INFO:tasks.rados.rados.0.trial178.stdout:3814: writing trial17818531-21 from 1646296 to 2381420 tid 2 2026-02-21T22:32:10.406 INFO:tasks.rados.rados.0.trial178.stdout:3812: done (2 left) 2026-02-21T22:32:10.406 INFO:tasks.rados.rados.0.trial178.stdout:3815: write oid 4 current snap is 392 2026-02-21T22:32:10.406 INFO:tasks.rados.rados.0.trial178.stdout:3815: seq_num 1337 ranges {470959=569688,1704778=595135} 2026-02-21T22:32:10.413 INFO:tasks.rados.rados.0.trial178.stdout:3815: writing trial17818531-4 from 470959 to 1040647 tid 1 2026-02-21T22:32:10.419 INFO:tasks.rados.rados.0.trial178.stdout:3815: writing trial17818531-4 from 1704778 to 2299913 tid 2 2026-02-21T22:32:10.419 INFO:tasks.rados.rados.0.trial178.stdout:3816: snap_create 2026-02-21T22:32:10.419 INFO:tasks.rados.rados.0.trial178.stdout:3813: finishing write tid 2 to trial17818531-44 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: finishing write tid 3 to trial17818531-44 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: finishing write tid 4 to trial17818531-44 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: oid 44 updating version 0 to 1860 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: oid 44 version 1860 is already newer than 1859 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: oid 44 version 1860 is already newer than 1858 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1860 (ObjNum 1335 snap 392 seq_num 1335) dirty exists 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3813: left oid 44 (ObjNum 1335 snap 392 seq_num 1335) 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3814: finishing write tid 1 to trial17818531-21 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3814: finishing write tid 2 to trial17818531-21 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3814: finishing write tid 3 to trial17818531-21 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3814: finishing write tid 4 to trial17818531-21 2026-02-21T22:32:10.420 INFO:tasks.rados.rados.0.trial178.stdout:3814: oid 21 updating version 0 to 2023 2026-02-21T22:32:10.421 INFO:tasks.rados.rados.0.trial178.stdout:3814: oid 21 updating version 2023 to 2024 2026-02-21T22:32:10.421 INFO:tasks.rados.rados.0.trial178.stdout:3814: oid 21 version 2024 is already newer than 2022 2026-02-21T22:32:10.421 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 2024 (ObjNum 1336 snap 392 seq_num 1336) dirty exists 2026-02-21T22:32:10.421 INFO:tasks.rados.rados.0.trial178.stdout:3814: left oid 21 (ObjNum 1336 snap 392 seq_num 1336) 2026-02-21T22:32:10.421 INFO:tasks.rados.rados.0.trial178.stdout:3815: finishing write tid 1 to trial17818531-4 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: finishing write tid 2 to trial17818531-4 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: finishing write tid 3 to trial17818531-4 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: finishing write tid 4 to trial17818531-4 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: oid 4 updating version 0 to 1861 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: oid 4 updating version 1861 to 1863 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: oid 4 version 1863 is already newer than 1862 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 4 v 1863 (ObjNum 1337 snap 392 seq_num 1337) dirty exists 2026-02-21T22:32:10.422 INFO:tasks.rados.rados.0.trial178.stdout:3815: left oid 4 (ObjNum 1337 snap 392 seq_num 1337) 2026-02-21T22:32:11.276 INFO:tasks.rados.rados.0.trial178.stdout:3813: done (3 left) 2026-02-21T22:32:11.276 INFO:tasks.rados.rados.0.trial178.stdout:3814: done (2 left) 2026-02-21T22:32:11.276 INFO:tasks.rados.rados.0.trial178.stdout:3815: done (1 left) 2026-02-21T22:32:11.277 INFO:tasks.rados.rados.0.trial178.stdout:3816: done (0 left) 2026-02-21T22:32:11.277 INFO:tasks.rados.rados.0.trial178.stdout:3817: write (excl) oid 43 current snap is 393 2026-02-21T22:32:11.277 INFO:tasks.rados.rados.0.trial178.stdout:3817: seq_num 1338 ranges {487766=644937,1646907=738793} 2026-02-21T22:32:11.281 INFO:tasks.rados.rados.0.trial178.stdout:3817: writing trial17818531-43 from 487766 to 1132703 tid 1 2026-02-21T22:32:11.287 INFO:tasks.rados.rados.0.trial178.stdout:3817: writing trial17818531-43 from 1646907 to 2385700 tid 2 2026-02-21T22:32:11.287 INFO:tasks.rados.rados.0.trial178.stdout:3818: read oid 18 snap -1 2026-02-21T22:32:11.287 INFO:tasks.rados.rados.0.trial178.stdout:3818: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:32:11.287 INFO:tasks.rados.rados.0.trial178.stdout:3819: snap_create 2026-02-21T22:32:11.287 INFO:tasks.rados.rados.0.trial178.stdout:3817: finishing write tid 1 to trial17818531-43 2026-02-21T22:32:11.289 INFO:tasks.rados.rados.0.trial178.stdout:3817: finishing write tid 2 to trial17818531-43 2026-02-21T22:32:11.289 INFO:tasks.rados.rados.0.trial178.stdout:3817: finishing write tid 3 to trial17818531-43 2026-02-21T22:32:11.289 INFO:tasks.rados.rados.0.trial178.stdout:3817: finishing write tid 4 to trial17818531-43 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:3817: oid 43 updating version 0 to 1620 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:3817: oid 43 updating version 1620 to 1621 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:3817: oid 43 version 1621 is already newer than 1619 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1621 (ObjNum 1338 snap 393 seq_num 1338) dirty exists 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:3817: left oid 43 (ObjNum 1338 snap 393 seq_num 1338) 2026-02-21T22:32:11.290 INFO:tasks.rados.rados.0.trial178.stdout:3818: expect (ObjNum 1247 snap 357 seq_num 1247) 2026-02-21T22:32:11.291 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:11.380 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2134956 2026-02-21T22:32:12.632 INFO:tasks.rados.rados.0.trial178.stdout:3817: done (2 left) 2026-02-21T22:32:12.633 INFO:tasks.rados.rados.0.trial178.stdout:3818: done (1 left) 2026-02-21T22:32:12.633 INFO:tasks.rados.rados.0.trial178.stdout:3819: done (0 left) 2026-02-21T22:32:12.633 INFO:tasks.rados.rados.0.trial178.stdout:3820: write (excl) oid 5 current snap is 394 2026-02-21T22:32:12.633 INFO:tasks.rados.rados.0.trial178.stdout:3820: seq_num 1339 ranges {504573=436539,1421742=598804,2437505=67068} 2026-02-21T22:32:12.639 INFO:tasks.rados.rados.0.trial178.stdout:3820: writing trial17818531-5 from 504573 to 941112 tid 1 2026-02-21T22:32:12.646 INFO:tasks.rados.rados.0.trial178.stdout:3820: writing trial17818531-5 from 1421742 to 2020546 tid 2 2026-02-21T22:32:12.649 INFO:tasks.rados.rados.0.trial178.stdout:3820: writing trial17818531-5 from 2437505 to 2504573 tid 3 2026-02-21T22:32:12.649 INFO:tasks.rados.rados.0.trial178.stdout:3821: snap_remove snap 389 2026-02-21T22:32:13.282 INFO:tasks.rados.rados.0.trial178.stdout:3821: done (1 left) 2026-02-21T22:32:13.282 INFO:tasks.rados.rados.0.trial178.stdout:3822: snap_remove snap 390 2026-02-21T22:32:13.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:13.594 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3822: done (1 left) 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3823: copy_from oid 26 from oid 23 current snap is 394 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: finishing write tid 1 to trial17818531-5 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: finishing write tid 2 to trial17818531-5 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: finishing write tid 3 to trial17818531-5 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: finishing write tid 4 to trial17818531-5 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: finishing write tid 5 to trial17818531-5 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: oid 5 updating version 0 to 1662 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: oid 5 version 1662 is already newer than 1661 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: oid 5 version 1662 is already newer than 1660 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: oid 5 version 1662 is already newer than 1659 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 5 v 1662 (ObjNum 1339 snap 394 seq_num 1339) dirty exists 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: left oid 5 (ObjNum 1339 snap 394 seq_num 1339) 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3820: done (1 left) 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3824: read oid 16 snap -1 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3824: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3825: read oid 17 snap -1 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3825: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:14.288 INFO:tasks.rados.rados.0.trial178.stdout:3826: delete oid 25 current snap is 394 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3826: done (3 left) 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3827: copy_from oid 10 from oid 41 current snap is 394 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3828: read oid 28 snap -1 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3828: expect deleted 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3829: write oid 34 current snap is 394 2026-02-21T22:32:14.290 INFO:tasks.rados.rados.0.trial178.stdout:3829: seq_num 1341 ranges {538187=587037,1538706=602473} 2026-02-21T22:32:14.297 INFO:tasks.rados.rados.0.trial178.stdout:3829: writing trial17818531-34 from 538187 to 1125224 tid 1 2026-02-21T22:32:14.304 INFO:tasks.rados.rados.0.trial178.stdout:3829: writing trial17818531-34 from 1538706 to 2141179 tid 2 2026-02-21T22:32:14.304 INFO:tasks.rados.rados.0.trial178.stdout:3830: delete oid 5 current snap is 394 2026-02-21T22:32:14.304 INFO:tasks.rados.rados.0.trial178.stdout:3824: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:14.381 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2373065 2026-02-21T22:32:14.388 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:14.424 INFO:tasks.rados.rados.0.trial178.stdout:3825: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:14.424 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:14.460 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:14.533 INFO:tasks.rados.rados.0.trial178.stdout:3829: finishing write tid 1 to trial17818531-34 2026-02-21T22:32:14.533 INFO:tasks.rados.rados.0.trial178.stdout:3829: finishing write tid 2 to trial17818531-34 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: finishing write tid 3 to trial17818531-34 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: finishing write tid 4 to trial17818531-34 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: oid 34 updating version 0 to 1637 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: oid 34 updating version 1637 to 1639 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: oid 34 version 1639 is already newer than 1638 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 34 v 1639 (ObjNum 1341 snap 394 seq_num 1341) dirty exists 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3829: left oid 34 (ObjNum 1341 snap 394 seq_num 1341) 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3827: finishing copy_from racing read to trial17818531-10 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3823: finishing copy_from racing read to trial17818531-26 2026-02-21T22:32:14.534 INFO:tasks.rados.rados.0.trial178.stdout:3823: finishing copy_from to trial17818531-26 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1430 (ObjNum 1269 snap 362 seq_num 1269) dirty exists 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3827: finishing copy_from to trial17818531-10 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1625 (ObjNum 1236 snap 350 seq_num 1236) dirty exists 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3823: done (6 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3824: done (5 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3825: done (4 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3827: done (3 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3828: done (2 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3829: done (1 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3830: done (0 left) 2026-02-21T22:32:14.535 INFO:tasks.rados.rados.0.trial178.stdout:3831: delete oid 2 current snap is 394 2026-02-21T22:32:14.536 INFO:tasks.rados.rados.0.trial178.stdout:3831: done (0 left) 2026-02-21T22:32:14.536 INFO:tasks.rados.rados.0.trial178.stdout:3832: rollback oid 46 current snap is 394 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 388 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:3833: read oid 13 snap -1 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:3833: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:3834: snap_create 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:3832: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:32:14.537 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 0 (ObjNum -1409409824 snap 32564 seq_num 304) dirty dne 2026-02-21T22:32:14.540 INFO:tasks.rados.rados.0.trial178.stdout:3833: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:14.540 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:14.576 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:15.292 INFO:tasks.rados.rados.0.trial178.stdout:3832: done (2 left) 2026-02-21T22:32:15.293 INFO:tasks.rados.rados.0.trial178.stdout:3833: done (1 left) 2026-02-21T22:32:15.293 INFO:tasks.rados.rados.0.trial178.stdout:3834: done (0 left) 2026-02-21T22:32:15.293 INFO:tasks.rados.rados.0.trial178.stdout:3835: write (excl) oid 41 current snap is 395 2026-02-21T22:32:15.293 INFO:tasks.rados.rados.0.trial178.stdout:3835: seq_num 1344 ranges {588608=412784,1597799=749800} 2026-02-21T22:32:15.299 INFO:tasks.rados.rados.0.trial178.stdout:3835: writing trial17818531-41 from 588608 to 1001392 tid 1 2026-02-21T22:32:15.309 INFO:tasks.rados.rados.0.trial178.stdout:3835: writing trial17818531-41 from 1597799 to 2347599 tid 2 2026-02-21T22:32:15.309 INFO:tasks.rados.rados.0.trial178.stdout:3836: write (excl) oid 47 current snap is 395 2026-02-21T22:32:15.309 INFO:tasks.rados.rados.0.trial178.stdout:3836: seq_num 1345 ranges {605415=488033,1656281=493458} 2026-02-21T22:32:15.316 INFO:tasks.rados.rados.0.trial178.stdout:3836: writing trial17818531-47 from 605415 to 1093448 tid 1 2026-02-21T22:32:15.324 INFO:tasks.rados.rados.0.trial178.stdout:3836: writing trial17818531-47 from 1656281 to 2149739 tid 2 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3837: snap_create 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: finishing write tid 1 to trial17818531-41 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: finishing write tid 2 to trial17818531-41 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: finishing write tid 3 to trial17818531-41 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: finishing write tid 4 to trial17818531-41 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: oid 41 updating version 0 to 1822 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: oid 41 updating version 1822 to 1824 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:3835: oid 41 version 1824 is already newer than 1823 2026-02-21T22:32:15.325 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 1824 (ObjNum 1344 snap 395 seq_num 1344) dirty exists 2026-02-21T22:32:15.326 INFO:tasks.rados.rados.0.trial178.stdout:3835: left oid 41 (ObjNum 1344 snap 395 seq_num 1344) 2026-02-21T22:32:15.326 INFO:tasks.rados.rados.0.trial178.stdout:3836: finishing write tid 1 to trial17818531-47 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: finishing write tid 2 to trial17818531-47 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: finishing write tid 3 to trial17818531-47 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: finishing write tid 4 to trial17818531-47 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: oid 47 updating version 0 to 2033 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: oid 47 updating version 2033 to 2034 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:3836: oid 47 version 2034 is already newer than 2032 2026-02-21T22:32:15.328 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 47 v 2034 (ObjNum 1345 snap 395 seq_num 1345) dirty exists 2026-02-21T22:32:15.329 INFO:tasks.rados.rados.0.trial178.stdout:3836: left oid 47 (ObjNum 1345 snap 395 seq_num 1345) 2026-02-21T22:32:16.296 INFO:tasks.rados.rados.0.trial178.stdout:3835: done (2 left) 2026-02-21T22:32:16.296 INFO:tasks.rados.rados.0.trial178.stdout:3836: done (1 left) 2026-02-21T22:32:16.297 INFO:tasks.rados.rados.0.trial178.stdout:3837: done (0 left) 2026-02-21T22:32:16.297 INFO:tasks.rados.rados.0.trial178.stdout:3838: write oid 20 current snap is 396 2026-02-21T22:32:16.297 INFO:tasks.rados.rados.0.trial178.stdout:3838: seq_num 1346 ranges {622222=679635,1831116=753469} 2026-02-21T22:32:16.306 INFO:tasks.rados.rados.0.trial178.stdout:3838: writing trial17818531-20 from 622222 to 1301857 tid 1 2026-02-21T22:32:16.315 INFO:tasks.rados.rados.0.trial178.stdout:3838: writing trial17818531-20 from 1831116 to 2584585 tid 2 2026-02-21T22:32:16.315 INFO:tasks.rados.rados.0.trial178.stdout:3839: delete oid 9 current snap is 396 2026-02-21T22:32:16.316 INFO:tasks.rados.rados.0.trial178.stdout:3838: finishing write tid 1 to trial17818531-20 2026-02-21T22:32:16.318 INFO:tasks.rados.rados.0.trial178.stdout:3838: finishing write tid 2 to trial17818531-20 2026-02-21T22:32:16.319 INFO:tasks.rados.rados.0.trial178.stdout:3839: done (1 left) 2026-02-21T22:32:16.319 INFO:tasks.rados.rados.0.trial178.stdout:3840: write (excl) oid 15 current snap is 396 2026-02-21T22:32:16.319 INFO:tasks.rados.rados.0.trial178.stdout:3840: seq_num 1348 ranges {655836=430133,1831727=757138} 2026-02-21T22:32:16.326 INFO:tasks.rados.rados.0.trial178.stdout:3840: writing trial17818531-15 from 655836 to 1085969 tid 1 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3840: writing trial17818531-15 from 1831727 to 2588865 tid 2 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3841: rollback oid 16 current snap is 396 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 16 to 385 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3838: finishing write tid 3 to trial17818531-20 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3838: finishing write tid 4 to trial17818531-20 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3838: oid 20 updating version 0 to 1826 2026-02-21T22:32:16.338 INFO:tasks.rados.rados.0.trial178.stdout:3838: oid 20 version 1826 is already newer than 1825 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3838: oid 20 updating version 1826 to 1827 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1827 (ObjNum 1346 snap 396 seq_num 1346) dirty exists 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3838: left oid 20 (ObjNum 1346 snap 396 seq_num 1346) 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3840: finishing write tid 1 to trial17818531-15 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3838: done (2 left) 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3842: rollback oid 44 current snap is 396 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 44 to 393 2026-02-21T22:32:16.339 INFO:tasks.rados.rados.0.trial178.stdout:3843: snap_remove snap 388 2026-02-21T22:32:17.226 INFO:tasks.rados.rados.0.trial178.stdout:3843: done (3 left) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3844: read oid 48 snap -1 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3844: expect (ObjNum 1317 snap 386 seq_num 1317) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3845: copy_from oid 9 from oid 42 current snap is 396 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3841: finishing rollback tid 0 to trial17818531-16 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: finishing write tid 2 to trial17818531-15 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3841: finishing rollback tid 1 to trial17818531-16 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: finishing write tid 3 to trial17818531-15 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: finishing write tid 4 to trial17818531-15 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: oid 15 updating version 0 to 1830 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: oid 15 version 1830 is already newer than 1829 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: oid 15 version 1830 is already newer than 1828 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1830 (ObjNum 1348 snap 396 seq_num 1348) dirty exists 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: left oid 15 (ObjNum 1348 snap 396 seq_num 1348) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3841: finishing rollback tid 2 to trial17818531-16 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1866 (ObjNum 1244 snap 356 seq_num 1244) dirty exists 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3842: finishing rollback tid 0 to trial17818531-44 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3842: finishing rollback tid 1 to trial17818531-44 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3842: finishing rollback tid 2 to trial17818531-44 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 44 v 1869 (ObjNum 1335 snap 392 seq_num 1335) dirty exists 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3840: done (4 left) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3841: done (3 left) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3842: done (2 left) 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3846: copy_from oid 44 from oid 50 current snap is 396 2026-02-21T22:32:17.227 INFO:tasks.rados.rados.0.trial178.stdout:3847: snap_create 2026-02-21T22:32:17.233 INFO:tasks.rados.rados.0.trial178.stdout:3844: expect (ObjNum 1317 snap 386 seq_num 1317) 2026-02-21T22:32:17.286 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:3846: finishing copy_from to trial17818531-44 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:3846: got expected ENOENT (src dne) 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:3846: finishing copy_from racing read to trial17818531-44 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:3845: finishing copy_from racing read to trial17818531-9 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:3845: finishing copy_from to trial17818531-9 2026-02-21T22:32:17.359 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 2017 (ObjNum 1326 snap 390 seq_num 1326) dirty exists 2026-02-21T22:32:18.230 INFO:tasks.rados.rados.0.trial178.stdout:3844: done (3 left) 2026-02-21T22:32:18.230 INFO:tasks.rados.rados.0.trial178.stdout:3845: done (2 left) 2026-02-21T22:32:18.230 INFO:tasks.rados.rados.0.trial178.stdout:3846: done (1 left) 2026-02-21T22:32:18.230 INFO:tasks.rados.rados.0.trial178.stdout:3847: done (0 left) 2026-02-21T22:32:18.230 INFO:tasks.rados.rados.0.trial178.stdout:3848: snap_create 2026-02-21T22:32:18.595 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:18.595 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:19.230 INFO:tasks.rados.rados.0.trial178.stdout:3848: done (0 left) 2026-02-21T22:32:19.230 INFO:tasks.rados.rados.0.trial178.stdout:3849: copy_from oid 1 from oid 20 current snap is 398 2026-02-21T22:32:19.230 INFO:tasks.rados.rados.0.trial178.stdout:3850: read oid 3 snap -1 2026-02-21T22:32:19.231 INFO:tasks.rados.rados.0.trial178.stdout:3850: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:32:19.231 INFO:tasks.rados.rados.0.trial178.stdout:3851: snap_remove snap 380 2026-02-21T22:32:20.361 INFO:tasks.rados.rados.0.trial178.stdout:3851: done (2 left) 2026-02-21T22:32:20.361 INFO:tasks.rados.rados.0.trial178.stdout:3852: write oid 6 current snap is 398 2026-02-21T22:32:20.361 INFO:tasks.rados.rados.0.trial178.stdout:3852: seq_num 1349 ranges {672643=505382,1890209=500796} 2026-02-21T22:32:20.368 INFO:tasks.rados.rados.0.trial178.stdout:3852: writing trial17818531-6 from 672643 to 1178025 tid 1 2026-02-21T22:32:20.377 INFO:tasks.rados.rados.0.trial178.stdout:3852: writing trial17818531-6 from 1890209 to 2391005 tid 2 2026-02-21T22:32:20.377 INFO:tasks.rados.rados.0.trial178.stdout:3853: write oid 18 current snap is 398 2026-02-21T22:32:20.377 INFO:tasks.rados.rados.0.trial178.stdout:3853: seq_num 1350 ranges {689450=580631,1948691=740759} 2026-02-21T22:32:20.385 INFO:tasks.rados.rados.0.trial178.stdout:3853: writing trial17818531-18 from 689450 to 1270081 tid 1 2026-02-21T22:32:20.395 INFO:tasks.rados.rados.0.trial178.stdout:3853: writing trial17818531-18 from 1948691 to 2689450 tid 2 2026-02-21T22:32:20.395 INFO:tasks.rados.rados.0.trial178.stdout:3854: write oid 21 current snap is 398 2026-02-21T22:32:20.395 INFO:tasks.rados.rados.0.trial178.stdout:3854: seq_num 1351 ranges {706257=655880,1890820=504465} 2026-02-21T22:32:20.404 INFO:tasks.rados.rados.0.trial178.stdout:3854: writing trial17818531-21 from 706257 to 1362137 tid 1 2026-02-21T22:32:20.409 INFO:tasks.rados.rados.0.trial178.stdout:3854: writing trial17818531-21 from 1890820 to 2395285 tid 2 2026-02-21T22:32:20.410 INFO:tasks.rados.rados.0.trial178.stdout:3855: write (excl) oid 45 current snap is 398 2026-02-21T22:32:20.410 INFO:tasks.rados.rados.0.trial178.stdout:3855: seq_num 1352 ranges {723064=731129,1949302=764476} 2026-02-21T22:32:20.415 INFO:tasks.rados.rados.0.trial178.stdout:3855: writing trial17818531-45 from 723064 to 1454193 tid 1 2026-02-21T22:32:20.420 INFO:tasks.rados.rados.0.trial178.stdout:3855: writing trial17818531-45 from 1949302 to 2713778 tid 2 2026-02-21T22:32:20.420 INFO:tasks.rados.rados.0.trial178.stdout:3856: copy_from oid 15 from oid 26 current snap is 398 2026-02-21T22:32:20.420 INFO:tasks.rados.rados.0.trial178.stdout:3850: expect (ObjNum 1230 snap 350 seq_num 1230) 2026-02-21T22:32:20.465 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 32904 2026-02-21T22:32:20.499 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 950080 2026-02-21T22:32:20.519 INFO:tasks.rados.rados.0.trial178.stdout:3849: finishing copy_from racing read to trial17818531-1 2026-02-21T22:32:20.519 INFO:tasks.rados.rados.0.trial178.stdout:3849: finishing copy_from to trial17818531-1 2026-02-21T22:32:20.519 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1827 (ObjNum 1346 snap 396 seq_num 1346) dirty exists 2026-02-21T22:32:20.519 INFO:tasks.rados.rados.0.trial178.stdout:3852: finishing write tid 1 to trial17818531-6 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: finishing write tid 2 to trial17818531-6 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: finishing write tid 3 to trial17818531-6 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: finishing write tid 4 to trial17818531-6 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: oid 6 updating version 0 to 1669 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: oid 6 version 1669 is already newer than 1668 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: oid 6 updating version 1669 to 1670 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1670 (ObjNum 1349 snap 398 seq_num 1349) dirty exists 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3852: left oid 6 (ObjNum 1349 snap 398 seq_num 1349) 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3853: finishing write tid 1 to trial17818531-18 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3853: finishing write tid 2 to trial17818531-18 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3853: finishing write tid 3 to trial17818531-18 2026-02-21T22:32:20.520 INFO:tasks.rados.rados.0.trial178.stdout:3849: done (6 left) 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3850: done (5 left) 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3852: done (4 left) 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3857: read oid 40 snap -1 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3857: expect deleted 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3858: write oid 48 current snap is 398 2026-02-21T22:32:20.521 INFO:tasks.rados.rados.0.trial178.stdout:3858: seq_num 1353 ranges {739871=406378,1607784=508134} 2026-02-21T22:32:20.523 INFO:tasks.rados.rados.0.trial178.stdout:3858: writing trial17818531-48 from 739871 to 1146249 tid 1 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3858: writing trial17818531-48 from 1607784 to 2115918 tid 2 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3859: copy_from oid 50 from oid 19 current snap is 398 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3853: finishing write tid 4 to trial17818531-18 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3853: oid 18 updating version 0 to 1664 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3853: oid 18 updating version 1664 to 1665 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:3853: oid 18 updating version 1665 to 1666 2026-02-21T22:32:20.528 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 18 v 1666 (ObjNum 1350 snap 398 seq_num 1350) dirty exists 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3853: left oid 18 (ObjNum 1350 snap 398 seq_num 1350) 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: finishing write tid 1 to trial17818531-21 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: finishing write tid 2 to trial17818531-21 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: finishing write tid 3 to trial17818531-21 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: finishing write tid 4 to trial17818531-21 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: oid 21 updating version 0 to 2056 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: oid 21 version 2056 is already newer than 2054 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: oid 21 version 2056 is already newer than 2055 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 2056 (ObjNum 1351 snap 398 seq_num 1351) dirty exists 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: left oid 21 (ObjNum 1351 snap 398 seq_num 1351) 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3855: finishing write tid 1 to trial17818531-45 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3853: done (6 left) 2026-02-21T22:32:20.529 INFO:tasks.rados.rados.0.trial178.stdout:3854: done (5 left) 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3860: copy_from oid 16 from oid 32 current snap is 398 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: finishing write tid 2 to trial17818531-45 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: finishing write tid 3 to trial17818531-45 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: finishing write tid 4 to trial17818531-45 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: oid 45 updating version 0 to 1829 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: oid 45 updating version 1829 to 1830 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: oid 45 version 1830 is already newer than 1828 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1830 (ObjNum 1352 snap 398 seq_num 1352) dirty exists 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: left oid 45 (ObjNum 1352 snap 398 seq_num 1352) 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3856: finishing copy_from racing read to trial17818531-15 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3855: done (5 left) 2026-02-21T22:32:20.530 INFO:tasks.rados.rados.0.trial178.stdout:3861: read oid 33 snap -1 2026-02-21T22:32:20.531 INFO:tasks.rados.rados.0.trial178.stdout:3861: expect (ObjNum 1321 snap 388 seq_num 1321) 2026-02-21T22:32:20.531 INFO:tasks.rados.rados.0.trial178.stdout:3862: write (excl) oid 24 current snap is 398 2026-02-21T22:32:20.531 INFO:tasks.rados.rados.0.trial178.stdout:3862: seq_num 1354 ranges {756678=597980,1782619=768145} 2026-02-21T22:32:20.536 INFO:tasks.rados.rados.0.trial178.stdout:3862: writing trial17818531-24 from 756678 to 1354658 tid 1 2026-02-21T22:32:20.544 INFO:tasks.rados.rados.0.trial178.stdout:3862: writing trial17818531-24 from 1782619 to 2550764 tid 2 2026-02-21T22:32:20.544 INFO:tasks.rados.rados.0.trial178.stdout:3863: read oid 8 snap -1 2026-02-21T22:32:20.544 INFO:tasks.rados.rados.0.trial178.stdout:3863: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:20.545 INFO:tasks.rados.rados.0.trial178.stdout:3864: write (excl) oid 9 current snap is 398 2026-02-21T22:32:20.545 INFO:tasks.rados.rados.0.trial178.stdout:3864: seq_num 1355 ranges {773485=673229,2124748=511803} 2026-02-21T22:32:20.554 INFO:tasks.rados.rados.0.trial178.stdout:3864: writing trial17818531-9 from 773485 to 1446714 tid 1 2026-02-21T22:32:20.560 INFO:tasks.rados.rados.0.trial178.stdout:3864: writing trial17818531-9 from 2124748 to 2636551 tid 2 2026-02-21T22:32:20.560 INFO:tasks.rados.rados.0.trial178.stdout:3865: read oid 38 snap 392 2026-02-21T22:32:20.560 INFO:tasks.rados.rados.0.trial178.stdout:3865: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:20.560 INFO:tasks.rados.rados.0.trial178.stdout:3866: delete oid 17 current snap is 398 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3856: finishing copy_from to trial17818531-15 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 15 v 1430 (ObjNum 1269 snap 362 seq_num 1269) dirty exists 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: finishing write tid 1 to trial17818531-48 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3859: finishing copy_from to trial17818531-50 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3859: got expected ENOENT (src dne) 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3859: finishing copy_from racing read to trial17818531-50 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: finishing write tid 2 to trial17818531-48 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: finishing write tid 3 to trial17818531-48 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: finishing write tid 4 to trial17818531-48 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: oid 48 updating version 0 to 1832 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: oid 48 updating version 1832 to 1833 2026-02-21T22:32:20.561 INFO:tasks.rados.rados.0.trial178.stdout:3858: oid 48 version 1833 is already newer than 1831 2026-02-21T22:32:20.562 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1833 (ObjNum 1353 snap 398 seq_num 1353) dirty exists 2026-02-21T22:32:20.562 INFO:tasks.rados.rados.0.trial178.stdout:3858: left oid 48 (ObjNum 1353 snap 398 seq_num 1353) 2026-02-21T22:32:20.562 INFO:tasks.rados.rados.0.trial178.stdout:3861: expect (ObjNum 1321 snap 388 seq_num 1321) 2026-02-21T22:32:20.562 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.582 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.604 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.627 INFO:tasks.rados.rados.0.trial178.stdout:3860: finishing copy_from racing read to trial17818531-16 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: finishing write tid 1 to trial17818531-24 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3860: finishing copy_from to trial17818531-16 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1827 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: finishing write tid 2 to trial17818531-24 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: finishing write tid 3 to trial17818531-24 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: finishing write tid 4 to trial17818531-24 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: oid 24 updating version 0 to 2058 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: oid 24 updating version 2058 to 2059 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: oid 24 updating version 2059 to 2060 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 2060 (ObjNum 1354 snap 398 seq_num 1354) dirty exists 2026-02-21T22:32:20.628 INFO:tasks.rados.rados.0.trial178.stdout:3862: left oid 24 (ObjNum 1354 snap 398 seq_num 1354) 2026-02-21T22:32:20.629 INFO:tasks.rados.rados.0.trial178.stdout:3863: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:20.692 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2637245 2026-02-21T22:32:20.697 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.738 INFO:tasks.rados.rados.0.trial178.stdout:3864: finishing write tid 1 to trial17818531-9 2026-02-21T22:32:20.738 INFO:tasks.rados.rados.0.trial178.stdout:3864: finishing write tid 2 to trial17818531-9 2026-02-21T22:32:20.738 INFO:tasks.rados.rados.0.trial178.stdout:3864: finishing write tid 3 to trial17818531-9 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3864: finishing write tid 4 to trial17818531-9 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3864: oid 9 updating version 0 to 2020 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3864: oid 9 version 2020 is already newer than 2019 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3864: oid 9 version 2020 is already newer than 2018 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 2020 (ObjNum 1355 snap 398 seq_num 1355) dirty exists 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3864: left oid 9 (ObjNum 1355 snap 398 seq_num 1355) 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:3865: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:20.739 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.840 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3546159 2026-02-21T22:32:20.841 INFO:tasks.rados.rados.0.trial178.stdout:3856: done (10 left) 2026-02-21T22:32:20.841 INFO:tasks.rados.rados.0.trial178.stdout:3857: done (9 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3858: done (8 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3859: done (7 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3860: done (6 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3861: done (5 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3862: done (4 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3863: done (3 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3864: done (2 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3865: done (1 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3866: done (0 left) 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3867: copy_from oid 33 from oid 30 current snap is 398 2026-02-21T22:32:20.842 INFO:tasks.rados.rados.0.trial178.stdout:3868: write oid 40 current snap is 398 2026-02-21T22:32:20.843 INFO:tasks.rados.rados.0.trial178.stdout:3868: seq_num 1357 ranges {407099=423727,1441712=515472,2582060=225039} 2026-02-21T22:32:20.847 INFO:tasks.rados.rados.0.trial178.stdout:3868: writing trial17818531-40 from 407099 to 830826 tid 1 2026-02-21T22:32:20.855 INFO:tasks.rados.rados.0.trial178.stdout:3868: writing trial17818531-40 from 1441712 to 1957184 tid 2 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3868: writing trial17818531-40 from 2582060 to 2807099 tid 3 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3869: delete oid 1 current snap is 398 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3868: finishing write tid 1 to trial17818531-40 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3867: finishing copy_from racing read to trial17818531-33 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3868: finishing write tid 2 to trial17818531-40 2026-02-21T22:32:20.861 INFO:tasks.rados.rados.0.trial178.stdout:3867: finishing copy_from to trial17818531-33 2026-02-21T22:32:20.862 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1631 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3867: done (2 left) 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3869: done (1 left) 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3870: rollback oid 20 current snap is 398 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 387 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3871: read oid 4 snap -1 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3871: expect (ObjNum 1337 snap 392 seq_num 1337) 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3872: rollback oid 37 current snap is 398 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 37 to 396 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3873: write oid 23 current snap is 398 2026-02-21T22:32:20.863 INFO:tasks.rados.rados.0.trial178.stdout:3873: seq_num 1359 ranges {440713=574225,1442323=519141,2636906=203807} 2026-02-21T22:32:20.869 INFO:tasks.rados.rados.0.trial178.stdout:3873: writing trial17818531-23 from 440713 to 1014938 tid 1 2026-02-21T22:32:20.876 INFO:tasks.rados.rados.0.trial178.stdout:3873: writing trial17818531-23 from 1442323 to 1961464 tid 2 2026-02-21T22:32:20.882 INFO:tasks.rados.rados.0.trial178.stdout:3873: writing trial17818531-23 from 2636906 to 2840713 tid 3 2026-02-21T22:32:20.882 INFO:tasks.rados.rados.0.trial178.stdout:3874: read oid 30 snap -1 2026-02-21T22:32:20.882 INFO:tasks.rados.rados.0.trial178.stdout:3874: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:20.882 INFO:tasks.rados.rados.0.trial178.stdout:3875: write (excl) oid 27 current snap is 398 2026-02-21T22:32:20.883 INFO:tasks.rados.rados.0.trial178.stdout:3875: seq_num 1360 ranges {457520=649474,1900805=662799} 2026-02-21T22:32:20.889 INFO:tasks.rados.rados.0.trial178.stdout:3875: writing trial17818531-27 from 457520 to 1106994 tid 1 2026-02-21T22:32:20.896 INFO:tasks.rados.rados.0.trial178.stdout:3875: writing trial17818531-27 from 1900805 to 2563604 tid 2 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3876: read oid 16 snap -1 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3876: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3877: copy_from oid 41 from oid 42 current snap is 398 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: finishing write tid 3 to trial17818531-40 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: finishing write tid 4 to trial17818531-40 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: finishing write tid 5 to trial17818531-40 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: oid 40 updating version 0 to 1666 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: oid 40 version 1666 is already newer than 1664 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: oid 40 version 1666 is already newer than 1665 2026-02-21T22:32:20.897 INFO:tasks.rados.rados.0.trial178.stdout:3868: oid 40 updating version 1666 to 1667 2026-02-21T22:32:20.898 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1667 (ObjNum 1357 snap 398 seq_num 1357) dirty exists 2026-02-21T22:32:20.898 INFO:tasks.rados.rados.0.trial178.stdout:3868: left oid 40 (ObjNum 1357 snap 398 seq_num 1357) 2026-02-21T22:32:20.898 INFO:tasks.rados.rados.0.trial178.stdout:3870: finishing rollback tid 0 to trial17818531-20 2026-02-21T22:32:20.898 INFO:tasks.rados.rados.0.trial178.stdout:3871: expect (ObjNum 1337 snap 392 seq_num 1337) 2026-02-21T22:32:20.916 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.942 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3872: finishing rollback tid 0 to trial17818531-37 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3870: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3872: finishing rollback tid 1 to trial17818531-37 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3868: done (8 left) 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3871: done (7 left) 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3878: read oid 25 snap -1 2026-02-21T22:32:20.969 INFO:tasks.rados.rados.0.trial178.stdout:3878: expect deleted 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3879: delete oid 4 current snap is 398 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3870: finishing rollback tid 2 to trial17818531-20 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1834 (ObjNum 1273 snap 364 seq_num 1273) dirty exists 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3872: finishing rollback tid 2 to trial17818531-37 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1837 (ObjNum 1283 snap 369 seq_num 1283) dirty exists 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: finishing write tid 1 to trial17818531-23 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: finishing write tid 2 to trial17818531-23 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: finishing write tid 3 to trial17818531-23 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: finishing write tid 4 to trial17818531-23 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: finishing write tid 5 to trial17818531-23 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: oid 23 updating version 0 to 1434 2026-02-21T22:32:20.970 INFO:tasks.rados.rados.0.trial178.stdout:3873: oid 23 version 1434 is already newer than 1432 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:3873: oid 23 version 1434 is already newer than 1433 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:3873: oid 23 version 1434 is already newer than 1431 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 23 v 1434 (ObjNum 1359 snap 398 seq_num 1359) dirty exists 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:3873: left oid 23 (ObjNum 1359 snap 398 seq_num 1359) 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:3874: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:20.971 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:21.012 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:21.098 INFO:tasks.rados.rados.0.trial178.stdout:3875: finishing write tid 1 to trial17818531-27 2026-02-21T22:32:21.098 INFO:tasks.rados.rados.0.trial178.stdout:3875: finishing write tid 2 to trial17818531-27 2026-02-21T22:32:21.098 INFO:tasks.rados.rados.0.trial178.stdout:3875: finishing write tid 3 to trial17818531-27 2026-02-21T22:32:21.098 INFO:tasks.rados.rados.0.trial178.stdout:3875: finishing write tid 4 to trial17818531-27 2026-02-21T22:32:21.098 INFO:tasks.rados.rados.0.trial178.stdout:3875: oid 27 updating version 0 to 2022 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:3875: oid 27 version 2022 is already newer than 2021 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:3875: oid 27 updating version 2022 to 2023 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 27 v 2023 (ObjNum 1360 snap 398 seq_num 1360) dirty exists 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:3875: left oid 27 (ObjNum 1360 snap 398 seq_num 1360) 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:3876: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:21.099 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:21.157 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2112521 2026-02-21T22:32:21.171 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3877: finishing copy_from racing read to trial17818531-41 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3877: finishing copy_from to trial17818531-41 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 41 v 2017 (ObjNum 1326 snap 390 seq_num 1326) dirty exists 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3870: done (8 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3872: done (7 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3873: done (6 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3874: done (5 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3875: done (4 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3876: done (3 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3877: done (2 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3878: done (1 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3879: done (0 left) 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3880: read oid 19 snap 391 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3880: expect deleted 2026-02-21T22:32:21.212 INFO:tasks.rados.rados.0.trial178.stdout:3881: snap_remove snap 373 2026-02-21T22:32:21.341 INFO:tasks.rados.rados.0.trial178.stdout:3881: done (1 left) 2026-02-21T22:32:21.342 INFO:tasks.rados.rados.0.trial178.stdout:3882: snap_create 2026-02-21T22:32:22.228 INFO:tasks.rados.rados.0.trial178.stdout:3880: done (1 left) 2026-02-21T22:32:22.228 INFO:tasks.rados.rados.0.trial178.stdout:3882: done (0 left) 2026-02-21T22:32:22.228 INFO:tasks.rados.rados.0.trial178.stdout:3883: copy_from oid 21 from oid 50 current snap is 399 2026-02-21T22:32:22.228 INFO:tasks.rados.rados.0.trial178.stdout:3884: snap_create 2026-02-21T22:32:22.232 INFO:tasks.rados.rados.0.trial178.stdout:3883: finishing copy_from to trial17818531-21 2026-02-21T22:32:22.232 INFO:tasks.rados.rados.0.trial178.stdout:3883: got expected ENOENT (src dne) 2026-02-21T22:32:22.232 INFO:tasks.rados.rados.0.trial178.stdout:3883: finishing copy_from racing read to trial17818531-21 2026-02-21T22:32:23.229 INFO:tasks.rados.rados.0.trial178.stdout:3883: done (1 left) 2026-02-21T22:32:23.229 INFO:tasks.rados.rados.0.trial178.stdout:3884: done (0 left) 2026-02-21T22:32:23.229 INFO:tasks.rados.rados.0.trial178.stdout:3885: read oid 30 snap -1 2026-02-21T22:32:23.229 INFO:tasks.rados.rados.0.trial178.stdout:3885: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:23.229 INFO:tasks.rados.rados.0.trial178.stdout:3886: write (excl) oid 39 current snap is 400 2026-02-21T22:32:23.230 INFO:tasks.rados.rados.0.trial178.stdout:3886: seq_num 1362 ranges {491134=516325,1734122=666468} 2026-02-21T22:32:23.236 INFO:tasks.rados.rados.0.trial178.stdout:3886: writing trial17818531-39 from 491134 to 1007459 tid 1 2026-02-21T22:32:23.245 INFO:tasks.rados.rados.0.trial178.stdout:3886: writing trial17818531-39 from 1734122 to 2400590 tid 2 2026-02-21T22:32:23.246 INFO:tasks.rados.rados.0.trial178.stdout:3887: copy_from oid 26 from oid 33 current snap is 400 2026-02-21T22:32:23.246 INFO:tasks.rados.rados.0.trial178.stdout:3885: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:23.309 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:23.366 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1474935 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: finishing write tid 1 to trial17818531-39 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: finishing write tid 2 to trial17818531-39 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: finishing write tid 3 to trial17818531-39 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: finishing write tid 4 to trial17818531-39 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: oid 39 updating version 0 to 2025 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: oid 39 version 2025 is already newer than 2024 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: oid 39 updating version 2025 to 2026 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 39 v 2026 (ObjNum 1362 snap 400 seq_num 1362) dirty exists 2026-02-21T22:32:23.393 INFO:tasks.rados.rados.0.trial178.stdout:3886: left oid 39 (ObjNum 1362 snap 400 seq_num 1362) 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3887: finishing copy_from racing read to trial17818531-26 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3887: finishing copy_from to trial17818531-26 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1631 (ObjNum 1157 snap 331 seq_num 1157) dirty exists 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3885: done (2 left) 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3886: done (1 left) 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3887: done (0 left) 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3888: write oid 46 current snap is 400 2026-02-21T22:32:23.394 INFO:tasks.rados.rados.0.trial178.stdout:3888: seq_num 1363 ranges {507941=591574,1676251=526479,2746598=161343} 2026-02-21T22:32:23.399 INFO:tasks.rados.rados.0.trial178.stdout:3888: writing trial17818531-46 from 507941 to 1099515 tid 1 2026-02-21T22:32:23.407 INFO:tasks.rados.rados.0.trial178.stdout:3888: writing trial17818531-46 from 1676251 to 2202730 tid 2 2026-02-21T22:32:23.412 INFO:tasks.rados.rados.0.trial178.stdout:3888: writing trial17818531-46 from 2746598 to 2907941 tid 3 2026-02-21T22:32:23.412 INFO:tasks.rados.rados.0.trial178.stdout:3889: read oid 30 snap -1 2026-02-21T22:32:23.412 INFO:tasks.rados.rados.0.trial178.stdout:3889: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:23.413 INFO:tasks.rados.rados.0.trial178.stdout:3890: write (excl) oid 6 current snap is 400 2026-02-21T22:32:23.413 INFO:tasks.rados.rados.0.trial178.stdout:3890: seq_num 1364 ranges {524748=666823,1734733=670137} 2026-02-21T22:32:23.420 INFO:tasks.rados.rados.0.trial178.stdout:3890: writing trial17818531-6 from 524748 to 1191571 tid 1 2026-02-21T22:32:23.428 INFO:tasks.rados.rados.0.trial178.stdout:3890: writing trial17818531-6 from 1734733 to 2404870 tid 2 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3891: copy_from oid 40 from oid 3 current snap is 400 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: finishing write tid 1 to trial17818531-46 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: finishing write tid 2 to trial17818531-46 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: finishing write tid 3 to trial17818531-46 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: finishing write tid 4 to trial17818531-46 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: finishing write tid 5 to trial17818531-46 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: oid 46 updating version 0 to 1438 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: oid 46 version 1438 is already newer than 1437 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: oid 46 version 1438 is already newer than 1435 2026-02-21T22:32:23.429 INFO:tasks.rados.rados.0.trial178.stdout:3888: oid 46 version 1438 is already newer than 1436 2026-02-21T22:32:23.430 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1438 (ObjNum 1363 snap 400 seq_num 1363) dirty exists 2026-02-21T22:32:23.430 INFO:tasks.rados.rados.0.trial178.stdout:3888: left oid 46 (ObjNum 1363 snap 400 seq_num 1363) 2026-02-21T22:32:23.430 INFO:tasks.rados.rados.0.trial178.stdout:3889: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:23.430 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:23.542 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:32:23] 2026-02-21T22:32:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:32:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:32:23, mode upmap, max misplaced 0.050000 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['unique_pool_0', '.mgr', 'rbd'] 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:32:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:32:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:23.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:23.565 INFO:tasks.rados.rados.0.trial178.stdout:3890: finishing write tid 1 to trial17818531-6 2026-02-21T22:32:23.565 INFO:tasks.rados.rados.0.trial178.stdout:3890: finishing write tid 2 to trial17818531-6 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: finishing write tid 3 to trial17818531-6 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: finishing write tid 4 to trial17818531-6 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: oid 6 updating version 0 to 1672 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: oid 6 updating version 1672 to 1673 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: oid 6 version 1673 is already newer than 1671 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1673 (ObjNum 1364 snap 400 seq_num 1364) dirty exists 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3890: left oid 6 (ObjNum 1364 snap 400 seq_num 1364) 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3891: finishing copy_from racing read to trial17818531-40 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3891: finishing copy_from to trial17818531-40 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1793 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:32:23.566 INFO:tasks.rados.rados.0.trial178.stdout:3888: done (3 left) 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3889: done (2 left) 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3890: done (1 left) 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3891: done (0 left) 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3892: read oid 8 snap -1 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3892: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3893: rollback oid 36 current snap is 400 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 396 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3894: write (excl) oid 25 current snap is 400 2026-02-21T22:32:23.567 INFO:tasks.rados.rados.0.trial178.stdout:3894: seq_num 1365 ranges {541555=742072,1793215=530148,2801444=140111} 2026-02-21T22:32:23.573 INFO:tasks.rados.rados.0.trial178.stdout:3894: writing trial17818531-25 from 541555 to 1283627 tid 1 2026-02-21T22:32:23.577 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:32:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:32:23.578 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:32:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:32:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:32:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:32:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:32:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:32:23.580 INFO:tasks.rados.rados.0.trial178.stdout:3894: writing trial17818531-25 from 1793215 to 2323363 tid 2 2026-02-21T22:32:23.582 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:32:23.583 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:32:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:32:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:32:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:32:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:32:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:32:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:32:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:32:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:32:23.593 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:32:23.594 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.595 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:32:23.596 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "unique_pool_0": { 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "rados": {} 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:32:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:16:00.732712+0000", 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:32:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "809", 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:32:23.599 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:32:23.600 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 16, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 16, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 16, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 16, 2026-02-21T22:32:23.601 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 16, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 3, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "unique_pool_0", 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 1.5, 2026-02-21T22:32:23.602 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 1.5, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 1.5, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 1.5, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 809, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 409, 2026-02-21T22:32:23.603 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:32:23.604 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1, -1] pools [1, 2, 3] with 6 osds, pg_target 600 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:32:23.606 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:32:23.607 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'unique_pool_0' root_id -1 using 0.0005514074254918982 of space, bias 1.0, pg target 0.16542222764756945 quantized to 32 (current 16) 2026-02-21T22:32:23.607 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool unique_pool_0 won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3894: writing trial17818531-25 from 2801444 to 2941555 tid 3 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3895: read oid 47 snap -1 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3895: expect (ObjNum 1345 snap 395 seq_num 1345) 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3896: rollback oid 40 current snap is 400 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 40 to 385 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3893: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1626 (ObjNum 129 snap 0 seq_num 99797840) dirty dne 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:3892: expect (ObjNum 1248 snap 358 seq_num 1248) 2026-02-21T22:32:23.608 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:23.609 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:23.609 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:23.623 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:23.694 INFO:tasks.rados.rados.0.trial178.stdout:3894: finishing write tid 1 to trial17818531-25 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: finishing write tid 2 to trial17818531-25 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3896: finishing rollback tid 0 to trial17818531-40 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3896: finishing rollback tid 1 to trial17818531-40 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: finishing write tid 3 to trial17818531-25 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: finishing write tid 4 to trial17818531-25 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3896: finishing rollback tid 2 to trial17818531-40 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1796 (ObjNum 1186 snap 337 seq_num 1186) dirty exists 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: finishing write tid 5 to trial17818531-25 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: oid 25 updating version 0 to 1996 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: oid 25 updating version 1996 to 1997 2026-02-21T22:32:23.695 INFO:tasks.rados.rados.0.trial178.stdout:3894: oid 25 updating version 1997 to 1999 2026-02-21T22:32:23.696 INFO:tasks.rados.rados.0.trial178.stdout:3894: oid 25 version 1999 is already newer than 1998 2026-02-21T22:32:23.696 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1999 (ObjNum 1365 snap 400 seq_num 1365) dirty exists 2026-02-21T22:32:23.696 INFO:tasks.rados.rados.0.trial178.stdout:3894: left oid 25 (ObjNum 1365 snap 400 seq_num 1365) 2026-02-21T22:32:23.696 INFO:tasks.rados.rados.0.trial178.stdout:3895: expect (ObjNum 1345 snap 395 seq_num 1345) 2026-02-21T22:32:23.745 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 773756 2026-02-21T22:32:23.825 INFO:tasks.rados.rados.0.trial178.stdout:3892: done (4 left) 2026-02-21T22:32:23.825 INFO:tasks.rados.rados.0.trial178.stdout:3893: done (3 left) 2026-02-21T22:32:23.826 INFO:tasks.rados.rados.0.trial178.stdout:3894: done (2 left) 2026-02-21T22:32:23.826 INFO:tasks.rados.rados.0.trial178.stdout:3895: done (1 left) 2026-02-21T22:32:23.826 INFO:tasks.rados.rados.0.trial178.stdout:3896: done (0 left) 2026-02-21T22:32:23.826 INFO:tasks.rados.rados.0.trial178.stdout:3897: snap_remove snap 397 2026-02-21T22:32:24.349 INFO:tasks.rados.rados.0.trial178.stdout:3897: done (0 left) 2026-02-21T22:32:24.349 INFO:tasks.rados.rados.0.trial178.stdout:3898: write (excl) oid 33 current snap is 400 2026-02-21T22:32:24.349 INFO:tasks.rados.rados.0.trial178.stdout:3898: seq_num 1366 ranges {558362=417321,1451697=673806,2712514=245848} 2026-02-21T22:32:24.355 INFO:tasks.rados.rados.0.trial178.stdout:3898: writing trial17818531-33 from 558362 to 975683 tid 1 2026-02-21T22:32:24.363 INFO:tasks.rados.rados.0.trial178.stdout:3898: writing trial17818531-33 from 1451697 to 2125503 tid 2 2026-02-21T22:32:24.369 INFO:tasks.rados.rados.0.trial178.stdout:3898: writing trial17818531-33 from 2712514 to 2958362 tid 3 2026-02-21T22:32:24.369 INFO:tasks.rados.rados.0.trial178.stdout:3899: snap_create 2026-02-21T22:32:24.369 INFO:tasks.rados.rados.0.trial178.stdout:3898: finishing write tid 1 to trial17818531-33 2026-02-21T22:32:24.370 INFO:tasks.rados.rados.0.trial178.stdout:3898: finishing write tid 2 to trial17818531-33 2026-02-21T22:32:24.371 INFO:tasks.rados.rados.0.trial178.stdout:3898: finishing write tid 3 to trial17818531-33 2026-02-21T22:32:24.371 INFO:tasks.rados.rados.0.trial178.stdout:3898: finishing write tid 4 to trial17818531-33 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: finishing write tid 5 to trial17818531-33 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: oid 33 updating version 0 to 1824 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: oid 33 updating version 1824 to 1825 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: oid 33 version 1825 is already newer than 1822 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: oid 33 version 1825 is already newer than 1823 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1825 (ObjNum 1366 snap 400 seq_num 1366) dirty exists 2026-02-21T22:32:24.372 INFO:tasks.rados.rados.0.trial178.stdout:3898: left oid 33 (ObjNum 1366 snap 400 seq_num 1366) 2026-02-21T22:32:25.350 INFO:tasks.rados.rados.0.trial178.stdout:3898: done (1 left) 2026-02-21T22:32:25.350 INFO:tasks.rados.rados.0.trial178.stdout:3899: done (0 left) 2026-02-21T22:32:25.350 INFO:tasks.rados.rados.0.trial178.stdout:3900: rollback oid 20 current snap is 401 2026-02-21T22:32:25.350 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 20 to 392 2026-02-21T22:32:25.350 INFO:tasks.rados.rados.0.trial178.stdout:3901: snap_create 2026-02-21T22:32:25.356 INFO:tasks.rados.rados.0.trial178.stdout:3900: finishing rollback tid 0 to trial17818531-20 2026-02-21T22:32:25.356 INFO:tasks.rados.rados.0.trial178.stdout:3900: finishing rollback tid 1 to trial17818531-20 2026-02-21T22:32:25.357 INFO:tasks.rados.rados.0.trial178.stdout:3900: finishing rollback tid 2 to trial17818531-20 2026-02-21T22:32:25.357 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 2020 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3900: done (1 left) 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3901: done (0 left) 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3902: read oid 1 snap -1 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3902: expect deleted 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3903: read oid 14 snap -1 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3903: expect deleted 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3904: read oid 17 snap 395 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3904: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3905: write oid 6 current snap is 402 2026-02-21T22:32:26.355 INFO:tasks.rados.rados.0.trial178.stdout:3905: seq_num 1367 ranges {575169=492570,1793826=417464,2623584=351585} 2026-02-21T22:32:26.360 INFO:tasks.rados.rados.0.trial178.stdout:3905: writing trial17818531-6 from 575169 to 1067739 tid 1 2026-02-21T22:32:26.365 INFO:tasks.rados.rados.0.trial178.stdout:3905: writing trial17818531-6 from 1793826 to 2211290 tid 2 2026-02-21T22:32:26.370 INFO:tasks.rados.rados.0.trial178.stdout:3905: writing trial17818531-6 from 2623584 to 2975169 tid 3 2026-02-21T22:32:26.370 INFO:tasks.rados.rados.0.trial178.stdout:3906: copy_from oid 25 from oid 16 current snap is 402 2026-02-21T22:32:26.370 INFO:tasks.rados.rados.0.trial178.stdout:3905: finishing write tid 1 to trial17818531-6 2026-02-21T22:32:26.370 INFO:tasks.rados.rados.0.trial178.stdout:3905: finishing write tid 2 to trial17818531-6 2026-02-21T22:32:26.370 INFO:tasks.rados.rados.0.trial178.stdout:3904: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:26.427 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:26.469 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1278846 2026-02-21T22:32:26.495 INFO:tasks.rados.rados.0.trial178.stdout:3905: finishing write tid 3 to trial17818531-6 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: finishing write tid 4 to trial17818531-6 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: finishing write tid 5 to trial17818531-6 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: oid 6 updating version 0 to 1675 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: oid 6 version 1675 is already newer than 1674 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: oid 6 updating version 1675 to 1676 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: oid 6 updating version 1676 to 1677 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 6 v 1677 (ObjNum 1367 snap 402 seq_num 1367) dirty exists 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3905: left oid 6 (ObjNum 1367 snap 402 seq_num 1367) 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3906: finishing copy_from racing read to trial17818531-25 2026-02-21T22:32:26.496 INFO:tasks.rados.rados.0.trial178.stdout:3906: finishing copy_from to trial17818531-25 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 25 v 1827 (ObjNum 1248 snap 358 seq_num 1248) dirty exists 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3902: done (4 left) 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3903: done (3 left) 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3904: done (2 left) 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3905: done (1 left) 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3906: done (0 left) 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3907: write (excl) oid 8 current snap is 402 2026-02-21T22:32:26.497 INFO:tasks.rados.rados.0.trial178.stdout:3907: seq_num 1368 ranges {591976=567819,1852308=677475} 2026-02-21T22:32:26.499 INFO:tasks.rados.rados.0.trial178.stdout:3907: writing trial17818531-8 from 591976 to 1159795 tid 1 2026-02-21T22:32:26.504 INFO:tasks.rados.rados.0.trial178.stdout:3907: writing trial17818531-8 from 1852308 to 2529783 tid 2 2026-02-21T22:32:26.505 INFO:tasks.rados.rados.0.trial178.stdout:3908: read oid 34 snap -1 2026-02-21T22:32:26.505 INFO:tasks.rados.rados.0.trial178.stdout:3908: expect (ObjNum 1341 snap 394 seq_num 1341) 2026-02-21T22:32:26.505 INFO:tasks.rados.rados.0.trial178.stdout:3909: delete oid 48 current snap is 402 2026-02-21T22:32:26.507 INFO:tasks.rados.rados.0.trial178.stdout:3909: done (2 left) 2026-02-21T22:32:26.508 INFO:tasks.rados.rados.0.trial178.stdout:3910: read oid 26 snap -1 2026-02-21T22:32:26.508 INFO:tasks.rados.rados.0.trial178.stdout:3910: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:26.508 INFO:tasks.rados.rados.0.trial178.stdout:3911: delete oid 45 current snap is 402 2026-02-21T22:32:26.509 INFO:tasks.rados.rados.0.trial178.stdout:3907: finishing write tid 1 to trial17818531-8 2026-02-21T22:32:26.510 INFO:tasks.rados.rados.0.trial178.stdout:3907: finishing write tid 2 to trial17818531-8 2026-02-21T22:32:26.510 INFO:tasks.rados.rados.0.trial178.stdout:3907: finishing write tid 3 to trial17818531-8 2026-02-21T22:32:26.510 INFO:tasks.rados.rados.0.trial178.stdout:3908: expect (ObjNum 1341 snap 394 seq_num 1341) 2026-02-21T22:32:26.539 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:26.557 INFO:tasks.rados.rados.0.trial178.stdout:3907: finishing write tid 4 to trial17818531-8 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:3907: oid 8 updating version 0 to 2079 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:3907: oid 8 updating version 2079 to 2080 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:3907: oid 8 version 2080 is already newer than 2078 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 2080 (ObjNum 1368 snap 402 seq_num 1368) dirty exists 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:3907: left oid 8 (ObjNum 1368 snap 402 seq_num 1368) 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:3910: expect (ObjNum 1157 snap 331 seq_num 1157) 2026-02-21T22:32:26.558 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:26.668 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3200487 2026-02-21T22:32:26.674 INFO:tasks.rados.rados.0.trial178.stdout:3907: done (3 left) 2026-02-21T22:32:26.674 INFO:tasks.rados.rados.0.trial178.stdout:3908: done (2 left) 2026-02-21T22:32:26.674 INFO:tasks.rados.rados.0.trial178.stdout:3910: done (1 left) 2026-02-21T22:32:26.674 INFO:tasks.rados.rados.0.trial178.stdout:3911: done (0 left) 2026-02-21T22:32:26.675 INFO:tasks.rados.rados.0.trial178.stdout:3912: write (excl) oid 48 current snap is 402 2026-02-21T22:32:26.675 INFO:tasks.rados.rados.0.trial178.stdout:3912: seq_num 1371 ranges {642397=509919,1627754=424802,2733276=309121} 2026-02-21T22:32:26.678 INFO:tasks.rados.rados.0.trial178.stdout:3912: writing trial17818531-48 from 642397 to 1152316 tid 1 2026-02-21T22:32:26.682 INFO:tasks.rados.rados.0.trial178.stdout:3912: writing trial17818531-48 from 1627754 to 2052556 tid 2 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:3912: writing trial17818531-48 from 2733276 to 3042397 tid 3 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:3913: rollback oid 22 current snap is 402 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 22 to 374 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:3912: finishing write tid 1 to trial17818531-48 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:3912: finishing write tid 2 to trial17818531-48 2026-02-21T22:32:26.686 INFO:tasks.rados.rados.0.trial178.stdout:3914: snap_create 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: finishing write tid 3 to trial17818531-48 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: finishing write tid 4 to trial17818531-48 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: finishing write tid 5 to trial17818531-48 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: oid 48 updating version 0 to 1843 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: oid 48 version 1843 is already newer than 1840 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: oid 48 version 1843 is already newer than 1842 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: oid 48 version 1843 is already newer than 1841 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1843 (ObjNum 1371 snap 402 seq_num 1371) dirty exists 2026-02-21T22:32:26.688 INFO:tasks.rados.rados.0.trial178.stdout:3912: left oid 48 (ObjNum 1371 snap 402 seq_num 1371) 2026-02-21T22:32:26.689 INFO:tasks.rados.rados.0.trial178.stdout:3913: finishing rollback tid 0 to trial17818531-22 2026-02-21T22:32:26.689 INFO:tasks.rados.rados.0.trial178.stdout:3913: finishing rollback tid 1 to trial17818531-22 2026-02-21T22:32:26.689 INFO:tasks.rados.rados.0.trial178.stdout:3913: finishing rollback tid 2 to trial17818531-22 2026-02-21T22:32:26.689 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 22 v 1799 (ObjNum 1232 snap 350 seq_num 1232) dirty exists 2026-02-21T22:32:27.230 INFO:tasks.rados.rados.0.trial178.stdout:3912: done (2 left) 2026-02-21T22:32:27.231 INFO:tasks.rados.rados.0.trial178.stdout:3913: done (1 left) 2026-02-21T22:32:27.231 INFO:tasks.rados.rados.0.trial178.stdout:3914: done (0 left) 2026-02-21T22:32:27.231 INFO:tasks.rados.rados.0.trial178.stdout:3915: write oid 9 current snap is 403 2026-02-21T22:32:27.231 INFO:tasks.rados.rados.0.trial178.stdout:3915: seq_num 1372 ranges {659204=585168,1686236=684813} 2026-02-21T22:32:27.236 INFO:tasks.rados.rados.0.trial178.stdout:3915: writing trial17818531-9 from 659204 to 1244372 tid 1 2026-02-21T22:32:27.240 INFO:tasks.rados.rados.0.trial178.stdout:3915: writing trial17818531-9 from 1686236 to 2371049 tid 2 2026-02-21T22:32:27.240 INFO:tasks.rados.rados.0.trial178.stdout:3916: snap_create 2026-02-21T22:32:27.244 INFO:tasks.rados.rados.0.trial178.stdout:3915: finishing write tid 1 to trial17818531-9 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: finishing write tid 2 to trial17818531-9 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: finishing write tid 3 to trial17818531-9 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: finishing write tid 4 to trial17818531-9 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: oid 9 updating version 0 to 2027 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: oid 9 updating version 2027 to 2029 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: oid 9 version 2029 is already newer than 2028 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 2029 (ObjNum 1372 snap 403 seq_num 1372) dirty exists 2026-02-21T22:32:27.245 INFO:tasks.rados.rados.0.trial178.stdout:3915: left oid 9 (ObjNum 1372 snap 403 seq_num 1372) 2026-02-21T22:32:28.234 INFO:tasks.rados.rados.0.trial178.stdout:3915: done (1 left) 2026-02-21T22:32:28.234 INFO:tasks.rados.rados.0.trial178.stdout:3916: done (0 left) 2026-02-21T22:32:28.234 INFO:tasks.rados.rados.0.trial178.stdout:3917: write (excl) oid 26 current snap is 404 2026-02-21T22:32:28.234 INFO:tasks.rados.rados.0.trial178.stdout:3917: seq_num 1373 ranges {676011=660417,1744718=428471,2788122=287889} 2026-02-21T22:32:28.244 INFO:tasks.rados.rados.0.trial178.stdout:3917: writing trial17818531-26 from 676011 to 1336428 tid 1 2026-02-21T22:32:28.249 INFO:tasks.rados.rados.0.trial178.stdout:3917: writing trial17818531-26 from 1744718 to 2173189 tid 2 2026-02-21T22:32:28.254 INFO:tasks.rados.rados.0.trial178.stdout:3917: writing trial17818531-26 from 2788122 to 3076011 tid 3 2026-02-21T22:32:28.254 INFO:tasks.rados.rados.0.trial178.stdout:3918: rollback oid 29 current snap is 404 2026-02-21T22:32:28.255 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 29 to 396 2026-02-21T22:32:28.255 INFO:tasks.rados.rados.0.trial178.stdout:3917: finishing write tid 1 to trial17818531-26 2026-02-21T22:32:28.255 INFO:tasks.rados.rados.0.trial178.stdout:3917: finishing write tid 2 to trial17818531-26 2026-02-21T22:32:28.255 INFO:tasks.rados.rados.0.trial178.stdout:3919: delete oid 18 current snap is 404 2026-02-21T22:32:28.258 INFO:tasks.rados.rados.0.trial178.stdout:3918: finishing rollback tid 0 to trial17818531-29 2026-02-21T22:32:28.258 INFO:tasks.rados.rados.0.trial178.stdout:3918: finishing rollback tid 1 to trial17818531-29 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: finishing write tid 3 to trial17818531-26 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3918: finishing rollback tid 2 to trial17818531-29 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1642 (ObjNum 1261 snap 359 seq_num 1261) dirty exists 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: finishing write tid 4 to trial17818531-26 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: finishing write tid 5 to trial17818531-26 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: oid 26 updating version 0 to 1680 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: oid 26 version 1680 is already newer than 1678 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: oid 26 updating version 1680 to 1681 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: oid 26 version 1681 is already newer than 1679 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 26 v 1681 (ObjNum 1373 snap 404 seq_num 1373) dirty exists 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: left oid 26 (ObjNum 1373 snap 404 seq_num 1373) 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3917: done (2 left) 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3918: done (1 left) 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3919: done (0 left) 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3920: rollback oid 36 current snap is 404 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 36 to 396 2026-02-21T22:32:28.259 INFO:tasks.rados.rados.0.trial178.stdout:3921: read oid 40 snap -1 2026-02-21T22:32:28.260 INFO:tasks.rados.rados.0.trial178.stdout:3921: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:32:28.260 INFO:tasks.rados.rados.0.trial178.stdout:3922: write oid 29 current snap is 404 2026-02-21T22:32:28.260 INFO:tasks.rados.rados.0.trial178.stdout:3922: seq_num 1375 ranges {709625=410915,1745329=432140,2726615=383010} 2026-02-21T22:32:28.266 INFO:tasks.rados.rados.0.trial178.stdout:3922: writing trial17818531-29 from 709625 to 1120540 tid 1 2026-02-21T22:32:28.273 INFO:tasks.rados.rados.0.trial178.stdout:3922: writing trial17818531-29 from 1745329 to 2177469 tid 2 2026-02-21T22:32:28.280 INFO:tasks.rados.rados.0.trial178.stdout:3922: writing trial17818531-29 from 2726615 to 3109625 tid 3 2026-02-21T22:32:28.280 INFO:tasks.rados.rados.0.trial178.stdout:3923: snap_create 2026-02-21T22:32:28.280 INFO:tasks.rados.rados.0.trial178.stdout:3920: finishing rollback tid 1 to trial17818531-36 2026-02-21T22:32:28.280 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1626 (ObjNum 100449120 snap 21866 seq_num 304) dirty dne 2026-02-21T22:32:28.280 INFO:tasks.rados.rados.0.trial178.stdout:3921: expect (ObjNum 1186 snap 337 seq_num 1186) 2026-02-21T22:32:28.388 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 3708733 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: finishing write tid 1 to trial17818531-29 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: finishing write tid 2 to trial17818531-29 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: finishing write tid 3 to trial17818531-29 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: finishing write tid 4 to trial17818531-29 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: finishing write tid 5 to trial17818531-29 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: oid 29 updating version 0 to 1646 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: oid 29 version 1646 is already newer than 1643 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: oid 29 version 1646 is already newer than 1644 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: oid 29 version 1646 is already newer than 1645 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1646 (ObjNum 1375 snap 404 seq_num 1375) dirty exists 2026-02-21T22:32:28.392 INFO:tasks.rados.rados.0.trial178.stdout:3922: left oid 29 (ObjNum 1375 snap 404 seq_num 1375) 2026-02-21T22:32:28.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:28.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:29.235 INFO:tasks.rados.rados.0.trial178.stdout:3920: done (3 left) 2026-02-21T22:32:29.235 INFO:tasks.rados.rados.0.trial178.stdout:3921: done (2 left) 2026-02-21T22:32:29.235 INFO:tasks.rados.rados.0.trial178.stdout:3922: done (1 left) 2026-02-21T22:32:29.235 INFO:tasks.rados.rados.0.trial178.stdout:3923: done (0 left) 2026-02-21T22:32:29.235 INFO:tasks.rados.rados.0.trial178.stdout:3924: snap_create 2026-02-21T22:32:30.239 INFO:tasks.rados.rados.0.trial178.stdout:3924: done (0 left) 2026-02-21T22:32:30.239 INFO:tasks.rados.rados.0.trial178.stdout:3925: snap_create 2026-02-21T22:32:31.241 INFO:tasks.rados.rados.0.trial178.stdout:3925: done (0 left) 2026-02-21T22:32:31.241 INFO:tasks.rados.rados.0.trial178.stdout:3926: rollback oid 2 current snap is 407 2026-02-21T22:32:31.241 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 2 to 394 2026-02-21T22:32:31.241 INFO:tasks.rados.rados.0.trial178.stdout:3927: delete oid 32 current snap is 407 2026-02-21T22:32:31.244 INFO:tasks.rados.rados.0.trial178.stdout:3926: finishing rollback tid 1 to trial17818531-2 2026-02-21T22:32:31.244 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 2 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2026-02-21T22:32:31.246 INFO:tasks.rados.rados.0.trial178.stdout:3926: done (1 left) 2026-02-21T22:32:31.246 INFO:tasks.rados.rados.0.trial178.stdout:3927: done (0 left) 2026-02-21T22:32:31.246 INFO:tasks.rados.rados.0.trial178.stdout:3928: write (excl) oid 33 current snap is 407 2026-02-21T22:32:31.246 INFO:tasks.rados.rados.0.trial178.stdout:3928: seq_num 1377 ranges {743239=677766,1978646=435809,2897814=245425} 2026-02-21T22:32:31.251 INFO:tasks.rados.rados.0.trial178.stdout:3928: writing trial17818531-33 from 743239 to 1421005 tid 1 2026-02-21T22:32:31.256 INFO:tasks.rados.rados.0.trial178.stdout:3928: writing trial17818531-33 from 1978646 to 2414455 tid 2 2026-02-21T22:32:31.262 INFO:tasks.rados.rados.0.trial178.stdout:3928: writing trial17818531-33 from 2897814 to 3143239 tid 3 2026-02-21T22:32:31.262 INFO:tasks.rados.rados.0.trial178.stdout:3929: read oid 9 snap -1 2026-02-21T22:32:31.262 INFO:tasks.rados.rados.0.trial178.stdout:3929: expect (ObjNum 1372 snap 403 seq_num 1372) 2026-02-21T22:32:31.262 INFO:tasks.rados.rados.0.trial178.stdout:3930: snap_remove snap 391 2026-02-21T22:32:32.230 INFO:tasks.rados.rados.0.trial178.stdout:3930: done (2 left) 2026-02-21T22:32:32.231 INFO:tasks.rados.rados.0.trial178.stdout:3931: write oid 36 current snap is 407 2026-02-21T22:32:32.231 INFO:tasks.rados.rados.0.trial178.stdout:3931: seq_num 1378 ranges {760046=753015,2037128=695820} 2026-02-21T22:32:32.240 INFO:tasks.rados.rados.0.trial178.stdout:3931: writing trial17818531-36 from 760046 to 1513061 tid 1 2026-02-21T22:32:32.249 INFO:tasks.rados.rados.0.trial178.stdout:3931: writing trial17818531-36 from 2037128 to 2732948 tid 2 2026-02-21T22:32:32.249 INFO:tasks.rados.rados.0.trial178.stdout:3932: write oid 24 current snap is 407 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: finishing write tid 1 to trial17818531-33 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: finishing write tid 2 to trial17818531-33 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: finishing write tid 3 to trial17818531-33 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: finishing write tid 4 to trial17818531-33 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: finishing write tid 5 to trial17818531-33 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: oid 33 updating version 0 to 1828 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: oid 33 version 1828 is already newer than 1826 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: oid 33 version 1828 is already newer than 1827 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: oid 33 updating version 1828 to 1829 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1829 (ObjNum 1377 snap 407 seq_num 1377) dirty exists 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3928: left oid 33 (ObjNum 1377 snap 407 seq_num 1377) 2026-02-21T22:32:32.250 INFO:tasks.rados.rados.0.trial178.stdout:3929: expect (ObjNum 1372 snap 403 seq_num 1372) 2026-02-21T22:32:32.425 INFO:tasks.rados.rados.0.trial178.stdout:3931: finishing write tid 1 to trial17818531-36 2026-02-21T22:32:32.425 INFO:tasks.rados.rados.0.trial178.stdout:3931: finishing write tid 2 to trial17818531-36 2026-02-21T22:32:32.425 INFO:tasks.rados.rados.0.trial178.stdout:3931: finishing write tid 3 to trial17818531-36 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3931: finishing write tid 4 to trial17818531-36 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3931: oid 36 updating version 0 to 1628 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3931: oid 36 updating version 1628 to 1629 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3931: oid 36 version 1629 is already newer than 1627 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 36 v 1629 (ObjNum 1378 snap 407 seq_num 1378) dirty exists 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3931: left oid 36 (ObjNum 1378 snap 407 seq_num 1378) 2026-02-21T22:32:32.426 INFO:tasks.rados.rados.0.trial178.stdout:3932: seq_num 1379 ranges {776853=428264,1979257=439478,2836307=340546} 2026-02-21T22:32:32.433 INFO:tasks.rados.rados.0.trial178.stdout:3932: writing trial17818531-24 from 776853 to 1205117 tid 1 2026-02-21T22:32:32.442 INFO:tasks.rados.rados.0.trial178.stdout:3932: writing trial17818531-24 from 1979257 to 2418735 tid 2 2026-02-21T22:32:32.447 INFO:tasks.rados.rados.0.trial178.stdout:3932: writing trial17818531-24 from 2836307 to 3176853 tid 3 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3928: done (3 left) 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3929: done (2 left) 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3931: done (1 left) 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3933: delete oid 9 current snap is 407 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3932: finishing write tid 1 to trial17818531-24 2026-02-21T22:32:32.448 INFO:tasks.rados.rados.0.trial178.stdout:3932: finishing write tid 2 to trial17818531-24 2026-02-21T22:32:32.451 INFO:tasks.rados.rados.0.trial178.stdout:3932: finishing write tid 3 to trial17818531-24 2026-02-21T22:32:32.451 INFO:tasks.rados.rados.0.trial178.stdout:3932: finishing write tid 4 to trial17818531-24 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: finishing write tid 5 to trial17818531-24 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: oid 24 updating version 0 to 2091 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: oid 24 version 2091 is already newer than 2089 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: oid 24 version 2091 is already newer than 2090 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: oid 24 version 2091 is already newer than 2088 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 2091 (ObjNum 1379 snap 407 seq_num 1379) dirty exists 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: left oid 24 (ObjNum 1379 snap 407 seq_num 1379) 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3932: done (1 left) 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3933: done (0 left) 2026-02-21T22:32:32.452 INFO:tasks.rados.rados.0.trial178.stdout:3934: rollback oid 8 current snap is 407 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 8 to 370 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:3935: rollback oid 28 current snap is 407 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 385 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:3936: read oid 13 snap -1 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:3936: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:3937: write (excl) oid 33 current snap is 407 2026-02-21T22:32:32.453 INFO:tasks.rados.rados.0.trial178.stdout:3937: seq_num 1381 ranges {410467=578762,1696221=443147,2891153=319314} 2026-02-21T22:32:32.457 INFO:tasks.rados.rados.0.trial178.stdout:3937: writing trial17818531-33 from 410467 to 989229 tid 1 2026-02-21T22:32:32.465 INFO:tasks.rados.rados.0.trial178.stdout:3937: writing trial17818531-33 from 1696221 to 2139368 tid 2 2026-02-21T22:32:32.473 INFO:tasks.rados.rados.0.trial178.stdout:3937: writing trial17818531-33 from 2891153 to 3210467 tid 3 2026-02-21T22:32:32.473 INFO:tasks.rados.rados.0.trial178.stdout:3938: snap_remove snap 401 2026-02-21T22:32:33.248 INFO:tasks.rados.rados.0.trial178.stdout:3938: done (4 left) 2026-02-21T22:32:33.248 INFO:tasks.rados.rados.0.trial178.stdout:3939: snap_create 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:3935: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1816 (ObjNum 129 snap 0 seq_num 100589656) dirty dne 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:3934: finishing rollback tid 0 to trial17818531-8 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:3934: finishing rollback tid 1 to trial17818531-8 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 2094 (ObjNum 305 snap 0 seq_num 1453350893) dirty dne 2026-02-21T22:32:33.249 INFO:tasks.rados.rados.0.trial178.stdout:3936: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:33.316 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2086209 2026-02-21T22:32:33.359 INFO:tasks.rados.rados.0.trial178.stdout:3937: finishing write tid 1 to trial17818531-33 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: finishing write tid 2 to trial17818531-33 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: finishing write tid 3 to trial17818531-33 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: finishing write tid 4 to trial17818531-33 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: finishing write tid 5 to trial17818531-33 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: oid 33 updating version 0 to 1833 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: oid 33 version 1833 is already newer than 1830 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: oid 33 version 1833 is already newer than 1832 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: oid 33 version 1833 is already newer than 1831 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1833 (ObjNum 1381 snap 407 seq_num 1381) dirty exists 2026-02-21T22:32:33.360 INFO:tasks.rados.rados.0.trial178.stdout:3937: left oid 33 (ObjNum 1381 snap 407 seq_num 1381) 2026-02-21T22:32:33.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:33.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:34.251 INFO:tasks.rados.rados.0.trial178.stdout:3934: done (4 left) 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3935: done (3 left) 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3936: done (2 left) 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3937: done (1 left) 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3939: done (0 left) 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3940: read oid 50 snap -1 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3940: expect deleted 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3941: write (excl) oid 33 current snap is 408 2026-02-21T22:32:34.252 INFO:tasks.rados.rados.0.trial178.stdout:3941: seq_num 1382 ranges {427274=654011,1754703=586805,2918576=308698} 2026-02-21T22:32:34.256 INFO:tasks.rados.rados.0.trial178.stdout:3941: writing trial17818531-33 from 427274 to 1081285 tid 1 2026-02-21T22:32:34.263 INFO:tasks.rados.rados.0.trial178.stdout:3941: writing trial17818531-33 from 1754703 to 2341508 tid 2 2026-02-21T22:32:34.269 INFO:tasks.rados.rados.0.trial178.stdout:3941: writing trial17818531-33 from 2918576 to 3227274 tid 3 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3942: copy_from oid 3 from oid 5 current snap is 408 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3941: finishing write tid 1 to trial17818531-33 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3941: finishing write tid 2 to trial17818531-33 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3940: done (2 left) 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3943: copy_from oid 28 from oid 31 current snap is 408 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3944: copy_from oid 20 from oid 50 current snap is 408 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3945: snap_create 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3942: finishing copy_from to trial17818531-3 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3942: got expected ENOENT (src dne) 2026-02-21T22:32:34.270 INFO:tasks.rados.rados.0.trial178.stdout:3944: finishing copy_from to trial17818531-20 2026-02-21T22:32:34.271 INFO:tasks.rados.rados.0.trial178.stdout:3944: got expected ENOENT (src dne) 2026-02-21T22:32:34.271 INFO:tasks.rados.rados.0.trial178.stdout:3942: finishing copy_from racing read to trial17818531-3 2026-02-21T22:32:34.271 INFO:tasks.rados.rados.0.trial178.stdout:3944: finishing copy_from racing read to trial17818531-20 2026-02-21T22:32:34.271 INFO:tasks.rados.rados.0.trial178.stdout:3941: finishing write tid 3 to trial17818531-33 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: finishing write tid 4 to trial17818531-33 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: finishing write tid 5 to trial17818531-33 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: oid 33 updating version 0 to 1836 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: oid 33 updating version 1836 to 1837 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: oid 33 version 1837 is already newer than 1835 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: oid 33 version 1837 is already newer than 1834 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 33 v 1837 (ObjNum 1382 snap 408 seq_num 1382) dirty exists 2026-02-21T22:32:34.272 INFO:tasks.rados.rados.0.trial178.stdout:3941: left oid 33 (ObjNum 1382 snap 408 seq_num 1382) 2026-02-21T22:32:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3943: finishing copy_from racing read to trial17818531-28 2026-02-21T22:32:34.283 INFO:tasks.rados.rados.0.trial178.stdout:3943: finishing copy_from to trial17818531-28 2026-02-21T22:32:34.283 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1995 (ObjNum 1253 snap 358 seq_num 1253) dirty exists 2026-02-21T22:32:35.252 INFO:tasks.rados.rados.0.trial178.stdout:3941: done (4 left) 2026-02-21T22:32:35.252 INFO:tasks.rados.rados.0.trial178.stdout:3942: done (3 left) 2026-02-21T22:32:35.253 INFO:tasks.rados.rados.0.trial178.stdout:3943: done (2 left) 2026-02-21T22:32:35.253 INFO:tasks.rados.rados.0.trial178.stdout:3944: done (1 left) 2026-02-21T22:32:35.253 INFO:tasks.rados.rados.0.trial178.stdout:3945: done (0 left) 2026-02-21T22:32:35.253 INFO:tasks.rados.rados.0.trial178.stdout:3946: write (excl) oid 30 current snap is 409 2026-02-21T22:32:35.253 INFO:tasks.rados.rados.0.trial178.stdout:3946: seq_num 1383 ranges {444081=729260,1813185=446816,2945999=298082} 2026-02-21T22:32:35.260 INFO:tasks.rados.rados.0.trial178.stdout:3946: writing trial17818531-30 from 444081 to 1173341 tid 1 2026-02-21T22:32:35.267 INFO:tasks.rados.rados.0.trial178.stdout:3946: writing trial17818531-30 from 1813185 to 2260001 tid 2 2026-02-21T22:32:35.274 INFO:tasks.rados.rados.0.trial178.stdout:3946: writing trial17818531-30 from 2945999 to 3244081 tid 3 2026-02-21T22:32:35.274 INFO:tasks.rados.rados.0.trial178.stdout:3947: snap_remove snap 371 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:3947: done (1 left) 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:3948: rollback oid 3 current snap is 409 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 3 to 400 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:3946: finishing write tid 1 to trial17818531-30 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:3946: finishing write tid 2 to trial17818531-30 2026-02-21T22:32:36.258 INFO:tasks.rados.rados.0.trial178.stdout:3946: finishing write tid 3 to trial17818531-30 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: finishing write tid 4 to trial17818531-30 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: finishing write tid 5 to trial17818531-30 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: oid 30 updating version 0 to 1671 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: oid 30 version 1671 is already newer than 1669 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: oid 30 version 1671 is already newer than 1670 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: oid 30 version 1671 is already newer than 1668 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 30 v 1671 (ObjNum 1383 snap 409 seq_num 1383) dirty exists 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: left oid 30 (ObjNum 1383 snap 409 seq_num 1383) 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3946: done (1 left) 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3949: write oid 16 current snap is 409 2026-02-21T22:32:36.259 INFO:tasks.rados.rados.0.trial178.stdout:3949: seq_num 1384 ranges {460888=520862,1471667=590474,2573422=455407} 2026-02-21T22:32:36.264 INFO:tasks.rados.rados.0.trial178.stdout:3949: writing trial17818531-16 from 460888 to 981750 tid 1 2026-02-21T22:32:36.271 INFO:tasks.rados.rados.0.trial178.stdout:3949: writing trial17818531-16 from 1471667 to 2062141 tid 2 2026-02-21T22:32:36.278 INFO:tasks.rados.rados.0.trial178.stdout:3949: writing trial17818531-16 from 2573422 to 3028829 tid 3 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3950: rollback oid 10 current snap is 409 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 10 to 370 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3948: finishing rollback tid 0 to trial17818531-3 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3948: finishing rollback tid 1 to trial17818531-3 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3948: finishing rollback tid 2 to trial17818531-3 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 3 v 1855 (ObjNum 1230 snap 350 seq_num 1230) dirty exists 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3949: finishing write tid 1 to trial17818531-16 2026-02-21T22:32:36.279 INFO:tasks.rados.rados.0.trial178.stdout:3949: finishing write tid 2 to trial17818531-16 2026-02-21T22:32:36.280 INFO:tasks.rados.rados.0.trial178.stdout:3948: done (2 left) 2026-02-21T22:32:36.280 INFO:tasks.rados.rados.0.trial178.stdout:3951: write oid 37 current snap is 409 2026-02-21T22:32:36.280 INFO:tasks.rados.rados.0.trial178.stdout:3951: seq_num 1385 ranges {477695=596111,1530149=450485,2600845=666679} 2026-02-21T22:32:36.285 INFO:tasks.rados.rados.0.trial178.stdout:3951: writing trial17818531-37 from 477695 to 1073806 tid 1 2026-02-21T22:32:36.291 INFO:tasks.rados.rados.0.trial178.stdout:3951: writing trial17818531-37 from 1530149 to 1980634 tid 2 2026-02-21T22:32:36.301 INFO:tasks.rados.rados.0.trial178.stdout:3951: writing trial17818531-37 from 2600845 to 3267524 tid 3 2026-02-21T22:32:36.301 INFO:tasks.rados.rados.0.trial178.stdout:3952: read oid 27 snap 385 2026-02-21T22:32:36.301 INFO:tasks.rados.rados.0.trial178.stdout:3952: expect (ObjNum 1290 snap 373 seq_num 1290) 2026-02-21T22:32:36.301 INFO:tasks.rados.rados.0.trial178.stdout:3953: snap_remove snap 407 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:3953: done (4 left) 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:3954: read oid 17 snap -1 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:3954: expect deleted 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:3955: rollback oid 49 current snap is 409 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 49 to 399 2026-02-21T22:32:37.235 INFO:tasks.rados.rados.0.trial178.stdout:3949: finishing write tid 3 to trial17818531-16 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: finishing write tid 4 to trial17818531-16 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3950: finishing rollback tid 0 to trial17818531-10 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3950: finishing rollback tid 1 to trial17818531-10 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 10 v 1683 (ObjNum 33 snap 0 seq_num 2) dirty dne 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: finishing write tid 5 to trial17818531-16 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: oid 16 updating version 0 to 1872 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: oid 16 updating version 1872 to 1873 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: oid 16 version 1873 is already newer than 1871 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: oid 16 updating version 1873 to 1874 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 16 v 1874 (ObjNum 1384 snap 409 seq_num 1384) dirty exists 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3949: left oid 16 (ObjNum 1384 snap 409 seq_num 1384) 2026-02-21T22:32:37.236 INFO:tasks.rados.rados.0.trial178.stdout:3951: finishing write tid 1 to trial17818531-37 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: finishing write tid 2 to trial17818531-37 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: finishing write tid 3 to trial17818531-37 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: finishing write tid 4 to trial17818531-37 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: finishing write tid 5 to trial17818531-37 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: oid 37 updating version 0 to 1846 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: oid 37 version 1846 is already newer than 1844 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: oid 37 version 1846 is already newer than 1845 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: oid 37 updating version 1846 to 1847 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 37 v 1847 (ObjNum 1385 snap 409 seq_num 1385) dirty exists 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3951: left oid 37 (ObjNum 1385 snap 409 seq_num 1385) 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:3952: expect (ObjNum 1290 snap 373 seq_num 1290) 2026-02-21T22:32:37.237 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:37.327 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:37.366 INFO:tasks.rados.rados.0.trial178.stdout:3955: finishing rollback tid 0 to trial17818531-49 2026-02-21T22:32:37.366 INFO:tasks.rados.rados.0.trial178.stdout:3955: finishing rollback tid 1 to trial17818531-49 2026-02-21T22:32:37.366 INFO:tasks.rados.rados.0.trial178.stdout:3955: finishing rollback tid 2 to trial17818531-49 2026-02-21T22:32:37.366 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 49 v 2002 (ObjNum 1311 snap 383 seq_num 1311) dirty exists 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3949: done (5 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3950: done (4 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3951: done (3 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3952: done (2 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3954: done (1 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3955: done (0 left) 2026-02-21T22:32:37.367 INFO:tasks.rados.rados.0.trial178.stdout:3956: snap_remove snap 374 2026-02-21T22:32:38.261 INFO:tasks.rados.rados.0.trial178.stdout:3956: done (0 left) 2026-02-21T22:32:38.261 INFO:tasks.rados.rados.0.trial178.stdout:3957: copy_from oid 24 from oid 6 current snap is 409 2026-02-21T22:32:38.261 INFO:tasks.rados.rados.0.trial178.stdout:3958: write oid 43 current snap is 409 2026-02-21T22:32:38.261 INFO:tasks.rados.rados.0.trial178.stdout:3958: seq_num 1386 ranges {494502=671360,1588631=594143,2628268=477951} 2026-02-21T22:32:38.269 INFO:tasks.rados.rados.0.trial178.stdout:3958: writing trial17818531-43 from 494502 to 1165862 tid 1 2026-02-21T22:32:38.276 INFO:tasks.rados.rados.0.trial178.stdout:3958: writing trial17818531-43 from 1588631 to 2182774 tid 2 2026-02-21T22:32:38.282 INFO:tasks.rados.rados.0.trial178.stdout:3958: writing trial17818531-43 from 2628268 to 3106219 tid 3 2026-02-21T22:32:38.282 INFO:tasks.rados.rados.0.trial178.stdout:3959: write (excl) oid 8 current snap is 409 2026-02-21T22:32:38.282 INFO:tasks.rados.rados.0.trial178.stdout:3958: finishing write tid 1 to trial17818531-43 2026-02-21T22:32:38.282 INFO:tasks.rados.rados.0.trial178.stdout:3957: finishing copy_from racing read to trial17818531-24 2026-02-21T22:32:38.282 INFO:tasks.rados.rados.0.trial178.stdout:3959: seq_num 1387 ranges {511309=746609,2047113=454154,3055691=255618} 2026-02-21T22:32:38.291 INFO:tasks.rados.rados.0.trial178.stdout:3959: writing trial17818531-8 from 511309 to 1257918 tid 1 2026-02-21T22:32:38.300 INFO:tasks.rados.rados.0.trial178.stdout:3959: writing trial17818531-8 from 2047113 to 2501267 tid 2 2026-02-21T22:32:38.305 INFO:tasks.rados.rados.0.trial178.stdout:3959: writing trial17818531-8 from 3055691 to 3311309 tid 3 2026-02-21T22:32:38.306 INFO:tasks.rados.rados.0.trial178.stdout:3960: write (excl) oid 45 current snap is 409 2026-02-21T22:32:38.306 INFO:tasks.rados.rados.0.trial178.stdout:3960: seq_num 1388 ranges {528116=421858,1589242=597812,2966761=361355} 2026-02-21T22:32:38.312 INFO:tasks.rados.rados.0.trial178.stdout:3960: writing trial17818531-45 from 528116 to 949974 tid 1 2026-02-21T22:32:38.317 INFO:tasks.rados.rados.0.trial178.stdout:3960: writing trial17818531-45 from 1589242 to 2187054 tid 2 2026-02-21T22:32:38.321 INFO:tasks.rados.rados.0.trial178.stdout:3960: writing trial17818531-45 from 2966761 to 3328116 tid 3 2026-02-21T22:32:38.321 INFO:tasks.rados.rados.0.trial178.stdout:3961: write (excl) oid 12 current snap is 409 2026-02-21T22:32:38.321 INFO:tasks.rados.rados.0.trial178.stdout:3961: seq_num 1389 ranges {544923=497107,1647724=457823,2710537=428120} 2026-02-21T22:32:38.325 INFO:tasks.rados.rados.0.trial178.stdout:3961: writing trial17818531-12 from 544923 to 1042030 tid 1 2026-02-21T22:32:38.329 INFO:tasks.rados.rados.0.trial178.stdout:3961: writing trial17818531-12 from 1647724 to 2105547 tid 2 2026-02-21T22:32:38.334 INFO:tasks.rados.rados.0.trial178.stdout:3961: writing trial17818531-12 from 2710537 to 3138657 tid 3 2026-02-21T22:32:38.334 INFO:tasks.rados.rados.0.trial178.stdout:3962: delete oid 38 current snap is 409 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3957: finishing copy_from to trial17818531-24 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 1677 (ObjNum 1367 snap 402 seq_num 1367) dirty exists 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: finishing write tid 2 to trial17818531-43 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: finishing write tid 3 to trial17818531-43 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: finishing write tid 4 to trial17818531-43 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: finishing write tid 5 to trial17818531-43 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: oid 43 updating version 0 to 1622 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: oid 43 updating version 1622 to 1624 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: oid 43 updating version 1624 to 1625 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:3958: oid 43 version 1625 is already newer than 1623 2026-02-21T22:32:38.335 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 43 v 1625 (ObjNum 1386 snap 409 seq_num 1386) dirty exists 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3958: left oid 43 (ObjNum 1386 snap 409 seq_num 1386) 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: finishing write tid 1 to trial17818531-8 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: finishing write tid 2 to trial17818531-8 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: finishing write tid 3 to trial17818531-8 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: finishing write tid 4 to trial17818531-8 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: finishing write tid 5 to trial17818531-8 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: oid 8 updating version 0 to 2128 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: oid 8 version 2128 is already newer than 2127 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: oid 8 version 2128 is already newer than 2126 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: oid 8 version 2128 is already newer than 2125 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 8 v 2128 (ObjNum 1387 snap 409 seq_num 1387) dirty exists 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3959: left oid 8 (ObjNum 1387 snap 409 seq_num 1387) 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: finishing write tid 1 to trial17818531-45 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: finishing write tid 2 to trial17818531-45 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: finishing write tid 3 to trial17818531-45 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: finishing write tid 4 to trial17818531-45 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: finishing write tid 5 to trial17818531-45 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: oid 45 updating version 0 to 1848 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: oid 45 updating version 1848 to 1851 2026-02-21T22:32:38.336 INFO:tasks.rados.rados.0.trial178.stdout:3960: oid 45 version 1851 is already newer than 1849 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3960: oid 45 version 1851 is already newer than 1850 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 45 v 1851 (ObjNum 1388 snap 409 seq_num 1388) dirty exists 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3960: left oid 45 (ObjNum 1388 snap 409 seq_num 1388) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3961: finishing write tid 1 to trial17818531-12 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3961: finishing write tid 2 to trial17818531-12 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3961: finishing write tid 3 to trial17818531-12 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3957: done (5 left) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3958: done (4 left) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3959: done (3 left) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3960: done (2 left) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3962: done (1 left) 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3963: write (excl) oid 21 current snap is 409 2026-02-21T22:32:38.337 INFO:tasks.rados.rados.0.trial178.stdout:3963: seq_num 1391 ranges {578537=647605,1764688=745139,3049030=329507} 2026-02-21T22:32:38.340 INFO:tasks.rados.rados.0.trial178.stdout:3963: writing trial17818531-21 from 578537 to 1226142 tid 1 2026-02-21T22:32:38.345 INFO:tasks.rados.rados.0.trial178.stdout:3963: writing trial17818531-21 from 1764688 to 2509827 tid 2 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3963: writing trial17818531-21 from 3049030 to 3378537 tid 3 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3964: delete oid 18 current snap is 409 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: finishing write tid 4 to trial17818531-12 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: finishing write tid 5 to trial17818531-12 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: oid 12 updating version 0 to 1877 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: oid 12 updating version 1877 to 1878 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: oid 12 version 1878 is already newer than 1875 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: oid 12 version 1878 is already newer than 1876 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 12 v 1878 (ObjNum 1389 snap 409 seq_num 1389) dirty exists 2026-02-21T22:32:38.348 INFO:tasks.rados.rados.0.trial178.stdout:3961: left oid 12 (ObjNum 1389 snap 409 seq_num 1389) 2026-02-21T22:32:38.349 INFO:tasks.rados.rados.0.trial178.stdout:3963: finishing write tid 1 to trial17818531-21 2026-02-21T22:32:38.349 INFO:tasks.rados.rados.0.trial178.stdout:3963: finishing write tid 2 to trial17818531-21 2026-02-21T22:32:38.349 INFO:tasks.rados.rados.0.trial178.stdout:3961: done (2 left) 2026-02-21T22:32:38.349 INFO:tasks.rados.rados.0.trial178.stdout:3964: done (1 left) 2026-02-21T22:32:38.349 INFO:tasks.rados.rados.0.trial178.stdout:3965: snap_remove snap 408 2026-02-21T22:32:38.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:38.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3965: done (1 left) 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3966: delete oid 41 current snap is 409 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3963: finishing write tid 3 to trial17818531-21 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3963: finishing write tid 4 to trial17818531-21 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3963: finishing write tid 5 to trial17818531-21 2026-02-21T22:32:39.265 INFO:tasks.rados.rados.0.trial178.stdout:3963: oid 21 updating version 0 to 2131 2026-02-21T22:32:39.266 INFO:tasks.rados.rados.0.trial178.stdout:3963: oid 21 updating version 2131 to 2132 2026-02-21T22:32:39.266 INFO:tasks.rados.rados.0.trial178.stdout:3963: oid 21 updating version 2132 to 2133 2026-02-21T22:32:39.266 INFO:tasks.rados.rados.0.trial178.stdout:3963: oid 21 version 2133 is already newer than 2130 2026-02-21T22:32:39.266 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 21 v 2133 (ObjNum 1391 snap 409 seq_num 1391) dirty exists 2026-02-21T22:32:39.266 INFO:tasks.rados.rados.0.trial178.stdout:3963: left oid 21 (ObjNum 1391 snap 409 seq_num 1391) 2026-02-21T22:32:39.269 INFO:tasks.rados.rados.0.trial178.stdout:3963: done (1 left) 2026-02-21T22:32:39.269 INFO:tasks.rados.rados.0.trial178.stdout:3966: done (0 left) 2026-02-21T22:32:39.269 INFO:tasks.rados.rados.0.trial178.stdout:3967: snap_create 2026-02-21T22:32:41.059 INFO:tasks.rados.rados.0.trial178.stdout:3967: done (0 left) 2026-02-21T22:32:41.059 INFO:tasks.rados.rados.0.trial178.stdout:3968: copy_from oid 18 from oid 41 current snap is 410 2026-02-21T22:32:41.059 INFO:tasks.rados.rados.0.trial178.stdout:3969: snap_remove snap 409 2026-02-21T22:32:41.274 INFO:tasks.rados.rados.0.trial178.stdout:3969: done (1 left) 2026-02-21T22:32:41.274 INFO:tasks.rados.rados.0.trial178.stdout:3970: write (excl) oid 1 current snap is 410 2026-02-21T22:32:41.274 INFO:tasks.rados.rados.0.trial178.stdout:3970: seq_num 1394 ranges {628958=589705,1940134=608819,3131299=297659} 2026-02-21T22:32:41.283 INFO:tasks.rados.rados.0.trial178.stdout:3970: writing trial17818531-1 from 628958 to 1218663 tid 1 2026-02-21T22:32:41.292 INFO:tasks.rados.rados.0.trial178.stdout:3970: writing trial17818531-1 from 1940134 to 2548953 tid 2 2026-02-21T22:32:41.298 INFO:tasks.rados.rados.0.trial178.stdout:3970: writing trial17818531-1 from 3131299 to 3428958 tid 3 2026-02-21T22:32:41.298 INFO:tasks.rados.rados.0.trial178.stdout:3971: snap_remove snap 398 2026-02-21T22:32:42.233 INFO:tasks.rados.rados.0.trial178.stdout:3971: done (2 left) 2026-02-21T22:32:42.233 INFO:tasks.rados.rados.0.trial178.stdout:3972: write (excl) oid 32 current snap is 410 2026-02-21T22:32:42.233 INFO:tasks.rados.rados.0.trial178.stdout:3972: seq_num 1395 ranges {645765=664954,1998616=752477,3158722=287043} 2026-02-21T22:32:42.242 INFO:tasks.rados.rados.0.trial178.stdout:3972: writing trial17818531-32 from 645765 to 1310719 tid 1 2026-02-21T22:32:42.251 INFO:tasks.rados.rados.0.trial178.stdout:3972: writing trial17818531-32 from 1998616 to 2751093 tid 2 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3972: writing trial17818531-32 from 3158722 to 3445765 tid 3 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3973: read oid 26 snap -1 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3973: expect (ObjNum 1373 snap 404 seq_num 1373) 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3974: delete oid 25 current snap is 410 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3968: finishing copy_from to trial17818531-18 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3968: got expected ENOENT (src dne) 2026-02-21T22:32:42.257 INFO:tasks.rados.rados.0.trial178.stdout:3968: finishing copy_from racing read to trial17818531-18 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: finishing write tid 1 to trial17818531-1 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: finishing write tid 2 to trial17818531-1 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: finishing write tid 3 to trial17818531-1 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: finishing write tid 4 to trial17818531-1 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: finishing write tid 5 to trial17818531-1 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: oid 1 updating version 0 to 1852 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: oid 1 updating version 1852 to 1853 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: oid 1 updating version 1853 to 1854 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: oid 1 updating version 1854 to 1855 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 1 v 1855 (ObjNum 1394 snap 410 seq_num 1394) dirty exists 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3970: left oid 1 (ObjNum 1394 snap 410 seq_num 1394) 2026-02-21T22:32:42.258 INFO:tasks.rados.rados.0.trial178.stdout:3972: finishing write tid 1 to trial17818531-32 2026-02-21T22:32:42.259 INFO:tasks.rados.rados.0.trial178.stdout:3972: finishing write tid 2 to trial17818531-32 2026-02-21T22:32:42.261 INFO:tasks.rados.rados.0.trial178.stdout:3972: finishing write tid 3 to trial17818531-32 2026-02-21T22:32:42.261 INFO:tasks.rados.rados.0.trial178.stdout:3972: finishing write tid 4 to trial17818531-32 2026-02-21T22:32:42.261 INFO:tasks.rados.rados.0.trial178.stdout:3972: finishing write tid 5 to trial17818531-32 2026-02-21T22:32:42.261 INFO:tasks.rados.rados.0.trial178.stdout:3972: oid 32 updating version 0 to 1880 2026-02-21T22:32:42.261 INFO:tasks.rados.rados.0.trial178.stdout:3972: oid 32 version 1880 is already newer than 1878 2026-02-21T22:32:42.262 INFO:tasks.rados.rados.0.trial178.stdout:3972: oid 32 updating version 1880 to 1881 2026-02-21T22:32:42.262 INFO:tasks.rados.rados.0.trial178.stdout:3972: oid 32 version 1881 is already newer than 1879 2026-02-21T22:32:42.262 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 32 v 1881 (ObjNum 1395 snap 410 seq_num 1395) dirty exists 2026-02-21T22:32:42.262 INFO:tasks.rados.rados.0.trial178.stdout:3972: left oid 32 (ObjNum 1395 snap 410 seq_num 1395) 2026-02-21T22:32:42.262 INFO:tasks.rados.rados.0.trial178.stdout:3973: expect (ObjNum 1373 snap 404 seq_num 1373) 2026-02-21T22:32:42.334 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1330363 2026-02-21T22:32:42.409 INFO:tasks.rados.rados.0.trial178.stdout:3968: done (4 left) 2026-02-21T22:32:42.409 INFO:tasks.rados.rados.0.trial178.stdout:3970: done (3 left) 2026-02-21T22:32:42.409 INFO:tasks.rados.rados.0.trial178.stdout:3972: done (2 left) 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3973: done (1 left) 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3974: done (0 left) 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3975: read oid 7 snap -1 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3975: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3976: write (excl) oid 29 current snap is 410 2026-02-21T22:32:42.410 INFO:tasks.rados.rados.0.trial178.stdout:3976: seq_num 1397 ranges {679379=415452,1599227=756146,3097215=382164} 2026-02-21T22:32:42.416 INFO:tasks.rados.rados.0.trial178.stdout:3976: writing trial17818531-29 from 679379 to 1094831 tid 1 2026-02-21T22:32:42.425 INFO:tasks.rados.rados.0.trial178.stdout:3976: writing trial17818531-29 from 1599227 to 2355373 tid 2 2026-02-21T22:32:42.433 INFO:tasks.rados.rados.0.trial178.stdout:3976: writing trial17818531-29 from 3097215 to 3479379 tid 3 2026-02-21T22:32:42.433 INFO:tasks.rados.rados.0.trial178.stdout:3977: copy_from oid 27 from oid 41 current snap is 410 2026-02-21T22:32:42.433 INFO:tasks.rados.rados.0.trial178.stdout:3976: finishing write tid 1 to trial17818531-29 2026-02-21T22:32:42.433 INFO:tasks.rados.rados.0.trial178.stdout:3975: expect (ObjNum 1179 snap 336 seq_num 1179) 2026-02-21T22:32:42.433 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:42.490 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:42.563 INFO:tasks.rados.rados.0.trial178.stdout:3976: finishing write tid 2 to trial17818531-29 2026-02-21T22:32:42.563 INFO:tasks.rados.rados.0.trial178.stdout:3977: finishing copy_from to trial17818531-27 2026-02-21T22:32:42.563 INFO:tasks.rados.rados.0.trial178.stdout:3977: got expected ENOENT (src dne) 2026-02-21T22:32:42.563 INFO:tasks.rados.rados.0.trial178.stdout:3976: finishing write tid 3 to trial17818531-29 2026-02-21T22:32:42.563 INFO:tasks.rados.rados.0.trial178.stdout:3977: finishing copy_from racing read to trial17818531-27 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: finishing write tid 4 to trial17818531-29 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3975: done (2 left) 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3977: done (1 left) 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3978: delete oid 36 current snap is 410 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: finishing write tid 5 to trial17818531-29 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: oid 29 updating version 0 to 1647 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: oid 29 updating version 1647 to 1650 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: oid 29 version 1650 is already newer than 1649 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: oid 29 version 1650 is already newer than 1648 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1650 (ObjNum 1397 snap 410 seq_num 1397) dirty exists 2026-02-21T22:32:42.564 INFO:tasks.rados.rados.0.trial178.stdout:3976: left oid 29 (ObjNum 1397 snap 410 seq_num 1397) 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:3976: done (1 left) 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:3978: done (0 left) 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:3979: rollback oid 28 current snap is 410 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 28 to 387 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:3980: write oid 29 current snap is 410 2026-02-21T22:32:42.565 INFO:tasks.rados.rados.0.trial178.stdout:3980: seq_num 1399 ranges {712993=682303,1832544=759815,3268414=244579} 2026-02-21T22:32:42.569 INFO:tasks.rados.rados.0.trial178.stdout:3980: writing trial17818531-29 from 712993 to 1395296 tid 1 2026-02-21T22:32:42.574 INFO:tasks.rados.rados.0.trial178.stdout:3980: writing trial17818531-29 from 1832544 to 2592359 tid 2 2026-02-21T22:32:42.578 INFO:tasks.rados.rados.0.trial178.stdout:3980: writing trial17818531-29 from 3268414 to 3512993 tid 3 2026-02-21T22:32:42.578 INFO:tasks.rados.rados.0.trial178.stdout:3981: write (excl) oid 50 current snap is 410 2026-02-21T22:32:42.578 INFO:tasks.rados.rados.0.trial178.stdout:3981: seq_num 1400 ranges {729800=757552,2174673=619826} 2026-02-21T22:32:42.583 INFO:tasks.rados.rados.0.trial178.stdout:3981: writing trial17818531-50 from 729800 to 1487352 tid 1 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3981: writing trial17818531-50 from 2174673 to 2794499 tid 2 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3982: read oid 32 snap -1 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3982: expect (ObjNum 1395 snap 410 seq_num 1395) 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3983: delete oid 5 current snap is 410 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3979: finishing rollback tid 0 to trial17818531-28 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3979: finishing rollback tid 1 to trial17818531-28 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 28 v 1998 (ObjNum 129 snap 0 seq_num 100749984) dirty dne 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3980: finishing write tid 1 to trial17818531-29 2026-02-21T22:32:42.589 INFO:tasks.rados.rados.0.trial178.stdout:3980: finishing write tid 2 to trial17818531-29 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: finishing write tid 3 to trial17818531-29 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: finishing write tid 4 to trial17818531-29 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: finishing write tid 5 to trial17818531-29 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: oid 29 updating version 0 to 1652 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: oid 29 updating version 1652 to 1653 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: oid 29 version 1653 is already newer than 1651 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: oid 29 updating version 1653 to 1654 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 29 v 1654 (ObjNum 1399 snap 410 seq_num 1399) dirty exists 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3980: left oid 29 (ObjNum 1399 snap 410 seq_num 1399) 2026-02-21T22:32:42.590 INFO:tasks.rados.rados.0.trial178.stdout:3981: finishing write tid 1 to trial17818531-50 2026-02-21T22:32:42.591 INFO:tasks.rados.rados.0.trial178.stdout:3979: done (4 left) 2026-02-21T22:32:42.591 INFO:tasks.rados.rados.0.trial178.stdout:3980: done (3 left) 2026-02-21T22:32:42.591 INFO:tasks.rados.rados.0.trial178.stdout:3983: done (2 left) 2026-02-21T22:32:42.591 INFO:tasks.rados.rados.0.trial178.stdout:3984: read oid 17 snap 382 2026-02-21T22:32:42.592 INFO:tasks.rados.rados.0.trial178.stdout:3984: expect (ObjNum 1307 snap 381 seq_num 1307) 2026-02-21T22:32:42.592 INFO:tasks.rados.rados.0.trial178.stdout:3985: copy_from oid 48 from oid 35 current snap is 410 2026-02-21T22:32:42.592 INFO:tasks.rados.rados.0.trial178.stdout:3986: rollback oid 11 current snap is 410 2026-02-21T22:32:42.592 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 11 to 379 2026-02-21T22:32:42.592 INFO:tasks.rados.rados.0.trial178.stdout:3987: snap_create 2026-02-21T22:32:42.595 INFO:tasks.rados.rados.0.trial178.stdout:3981: finishing write tid 2 to trial17818531-50 2026-02-21T22:32:42.595 INFO:tasks.rados.rados.0.trial178.stdout:3982: expect (ObjNum 1395 snap 410 seq_num 1395) 2026-02-21T22:32:42.669 INFO:tasks.rados.rados.0.trial178.stdout:3981: finishing write tid 3 to trial17818531-50 2026-02-21T22:32:42.669 INFO:tasks.rados.rados.0.trial178.stdout:3981: finishing write tid 4 to trial17818531-50 2026-02-21T22:32:42.669 INFO:tasks.rados.rados.0.trial178.stdout:3981: oid 50 updating version 0 to 1802 2026-02-21T22:32:42.670 INFO:tasks.rados.rados.0.trial178.stdout:3981: oid 50 version 1802 is already newer than 1801 2026-02-21T22:32:42.670 INFO:tasks.rados.rados.0.trial178.stdout:3981: oid 50 version 1802 is already newer than 1800 2026-02-21T22:32:42.670 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 1802 (ObjNum 1400 snap 410 seq_num 1400) dirty exists 2026-02-21T22:32:42.670 INFO:tasks.rados.rados.0.trial178.stdout:3981: left oid 50 (ObjNum 1400 snap 410 seq_num 1400) 2026-02-21T22:32:42.670 INFO:tasks.rados.rados.0.trial178.stdout:3984: expect (ObjNum 1307 snap 381 seq_num 1307) 2026-02-21T22:32:42.722 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:3986: finishing rollback tid 0 to trial17818531-11 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:3986: finishing rollback tid 1 to trial17818531-11 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:3986: finishing rollback tid 2 to trial17818531-11 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 11 v 1885 (ObjNum 1265 snap 362 seq_num 1265) dirty exists 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:3985: finishing copy_from racing read to trial17818531-48 2026-02-21T22:32:42.822 INFO:tasks.rados.rados.0.trial178.stdout:3985: finishing copy_from to trial17818531-48 2026-02-21T22:32:42.823 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1817 (ObjNum 1319 snap 387 seq_num 1319) dirty exists 2026-02-21T22:32:43.280 INFO:tasks.rados.rados.0.trial178.stdout:3981: done (5 left) 2026-02-21T22:32:43.280 INFO:tasks.rados.rados.0.trial178.stdout:3982: done (4 left) 2026-02-21T22:32:43.280 INFO:tasks.rados.rados.0.trial178.stdout:3984: done (3 left) 2026-02-21T22:32:43.280 INFO:tasks.rados.rados.0.trial178.stdout:3985: done (2 left) 2026-02-21T22:32:43.280 INFO:tasks.rados.rados.0.trial178.stdout:3986: done (1 left) 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3987: done (0 left) 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3988: copy_from oid 40 from oid 31 current snap is 411 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3989: read oid 15 snap -1 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3989: expect (ObjNum 1269 snap 362 seq_num 1269) 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3990: write oid 48 current snap is 411 2026-02-21T22:32:43.281 INFO:tasks.rados.rados.0.trial178.stdout:3990: seq_num 1402 ranges {763414=508050,1891637=623495,3234330=329084} 2026-02-21T22:32:43.287 INFO:tasks.rados.rados.0.trial178.stdout:3990: writing trial17818531-48 from 763414 to 1271464 tid 1 2026-02-21T22:32:43.292 INFO:tasks.rados.rados.0.trial178.stdout:3990: writing trial17818531-48 from 1891637 to 2515132 tid 2 2026-02-21T22:32:43.296 INFO:tasks.rados.rados.0.trial178.stdout:3990: writing trial17818531-48 from 3234330 to 3563414 tid 3 2026-02-21T22:32:43.296 INFO:tasks.rados.rados.0.trial178.stdout:3991: write oid 9 current snap is 411 2026-02-21T22:32:43.296 INFO:tasks.rados.rados.0.trial178.stdout:3991: seq_num 1403 ranges {780221=583299,1950119=767153,3261753=318468} 2026-02-21T22:32:43.301 INFO:tasks.rados.rados.0.trial178.stdout:3991: writing trial17818531-9 from 780221 to 1363520 tid 1 2026-02-21T22:32:43.306 INFO:tasks.rados.rados.0.trial178.stdout:3991: writing trial17818531-9 from 1950119 to 2717272 tid 2 2026-02-21T22:32:43.310 INFO:tasks.rados.rados.0.trial178.stdout:3991: writing trial17818531-9 from 3261753 to 3580221 tid 3 2026-02-21T22:32:43.310 INFO:tasks.rados.rados.0.trial178.stdout:3992: rollback oid 46 current snap is 411 2026-02-21T22:32:43.310 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 46 to 403 2026-02-21T22:32:43.310 INFO:tasks.rados.rados.0.trial178.stdout:3989: expect (ObjNum 1269 snap 362 seq_num 1269) 2026-02-21T22:32:43.426 INFO:tasks.rados.rados.0.trial178.stdout:3988: finishing copy_from racing read to trial17818531-40 2026-02-21T22:32:43.426 INFO:tasks.rados.rados.0.trial178.stdout:3989: done (4 left) 2026-02-21T22:32:43.427 INFO:tasks.rados.rados.0.trial178.stdout:3993: write (excl) oid 24 current snap is 411 2026-02-21T22:32:43.427 INFO:tasks.rados.rados.0.trial178.stdout:3993: seq_num 1404 ranges {797028=658548,1892248=510811,3172823=424205} 2026-02-21T22:32:43.431 INFO:tasks.rados.rados.0.trial178.stdout:3993: writing trial17818531-24 from 797028 to 1455576 tid 1 2026-02-21T22:32:43.435 INFO:tasks.rados.rados.0.trial178.stdout:3993: writing trial17818531-24 from 1892248 to 2403059 tid 2 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3993: writing trial17818531-24 from 3172823 to 3597028 tid 3 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3994: copy_from oid 50 from oid 27 current snap is 411 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3988: finishing copy_from to trial17818531-40 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 40 v 1995 (ObjNum 1253 snap 358 seq_num 1253) dirty exists 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3990: finishing write tid 1 to trial17818531-48 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3990: finishing write tid 2 to trial17818531-48 2026-02-21T22:32:43.441 INFO:tasks.rados.rados.0.trial178.stdout:3990: finishing write tid 3 to trial17818531-48 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: finishing write tid 4 to trial17818531-48 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: finishing write tid 5 to trial17818531-48 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: oid 48 updating version 0 to 1859 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: oid 48 version 1859 is already newer than 1857 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: oid 48 version 1859 is already newer than 1858 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: oid 48 version 1859 is already newer than 1856 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 48 v 1859 (ObjNum 1402 snap 411 seq_num 1402) dirty exists 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: left oid 48 (ObjNum 1402 snap 411 seq_num 1402) 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3991: finishing write tid 1 to trial17818531-9 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3988: done (5 left) 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3990: done (4 left) 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3995: read oid 1 snap -1 2026-02-21T22:32:43.442 INFO:tasks.rados.rados.0.trial178.stdout:3995: expect (ObjNum 1394 snap 410 seq_num 1394) 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3996: rollback oid 19 current snap is 411 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: finishing write tid 2 to trial17818531-9 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3992: finishing rollback tid 0 to trial17818531-46 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:rollback oid 19 to 399 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: finishing write tid 3 to trial17818531-9 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: finishing write tid 4 to trial17818531-9 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3992: finishing rollback tid 1 to trial17818531-46 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: finishing write tid 5 to trial17818531-9 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: oid 9 updating version 0 to 2031 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: oid 9 updating version 2031 to 2033 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: oid 9 updating version 2033 to 2034 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:3991: oid 9 version 2034 is already newer than 2032 2026-02-21T22:32:43.443 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 9 v 2034 (ObjNum 1403 snap 411 seq_num 1403) dirty exists 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3991: left oid 9 (ObjNum 1403 snap 411 seq_num 1403) 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3992: finishing rollback tid 2 to trial17818531-46 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 46 v 1441 (ObjNum 1363 snap 400 seq_num 1363) dirty exists 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3993: finishing write tid 1 to trial17818531-24 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3993: finishing write tid 2 to trial17818531-24 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3991: done (5 left) 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3992: done (4 left) 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3997: read oid 13 snap -1 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3997: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3998: copy_from oid 20 from oid 46 current snap is 411 2026-02-21T22:32:43.444 INFO:tasks.rados.rados.0.trial178.stdout:3999: snap_create 2026-02-21T22:32:43.445 INFO:tasks.rados.rados.0.trial178.stdout:3995: expect (ObjNum 1394 snap 410 seq_num 1394) 2026-02-21T22:32:43.508 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 1790258 2026-02-21T22:32:43.524 INFO:tasks.rados.rados.0.trial178.stdout:3997: expect (ObjNum 1244 snap 356 seq_num 1244) 2026-02-21T22:32:43.524 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:43.580 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 2303821 2026-02-21T22:32:43.587 INFO:tasks.rados.rados.0.trial178.stdout:seeking to 0 2026-02-21T22:32:43.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:43.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:43.623 INFO:tasks.rados.rados.0.trial178.stdout:3993: finishing write tid 3 to trial17818531-24 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: finishing write tid 4 to trial17818531-24 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3996: finishing rollback tid 1 to trial17818531-19 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 19 v 2021 (ObjNum 161 snap 0 seq_num 1398773517) dirty dne 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: finishing write tid 5 to trial17818531-24 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: oid 24 updating version 0 to 2148 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: oid 24 version 2148 is already newer than 2145 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: oid 24 version 2148 is already newer than 2146 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: oid 24 version 2148 is already newer than 2147 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 24 v 2148 (ObjNum 1404 snap 411 seq_num 1404) dirty exists 2026-02-21T22:32:43.624 INFO:tasks.rados.rados.0.trial178.stdout:3993: left oid 24 (ObjNum 1404 snap 411 seq_num 1404) 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:3994: finishing copy_from racing read to trial17818531-50 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:3994: finishing copy_from to trial17818531-50 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 50 v 2023 (ObjNum 1360 snap 398 seq_num 1360) dirty exists 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:3998: finishing copy_from racing read to trial17818531-20 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:3998: finishing copy_from to trial17818531-20 2026-02-21T22:32:43.625 INFO:tasks.rados.rados.0.trial178.stdout:update_object_version oid 20 v 1441 (ObjNum 1363 snap 400 seq_num 1363) dirty exists 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3993: done (6 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3994: done (5 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3995: done (4 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3996: done (3 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3997: done (2 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3998: done (1 left) 2026-02-21T22:32:44.288 INFO:tasks.rados.rados.0.trial178.stdout:3999: done (0 left) 2026-02-21T22:32:44.292 INFO:tasks.rados.rados.0.trial178.stderr:0 errors. 2026-02-21T22:32:44.293 INFO:tasks.rados.rados.0.trial178.stderr: 2026-02-21T22:32:44.444 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:32:44.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.508+0000 7fd940215640 10 monclient: get_monmap_and_config 2026-02-21T22:32:44.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.508+0000 7fd940215640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.512 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.508+0000 7fd940215640 10 monclient: monmap: 2026-02-21T22:32:44.512 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.512 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:32:44.512398+0000 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:32:44.512398+0000 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 Processor -- start 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- start start 2026-02-21T22:32:44.513 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient: init 2026-02-21T22:32:44.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:32:44.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient(hunting): picked mon.noname-a con 0x7fd938142b80 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.518 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient(hunting): picked mon.noname-b con 0x7fd93808c560 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient(hunting): picked mon.noname-c con 0x7fd938143660 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93808abc0 con 0x7fd938142b80 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93808aea0 con 0x7fd938143660 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd938088ea0 con 0x7fd93808c560 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient(hunting): authenticate will time out at 2173.748562s 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93df8a640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd938142b80 0x7fd938142f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56342/0 (socket says 10.20.193.102:56342) 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93e78b640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fd938143660 legacy=0x7fd9381a57e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93df8a640 1 -- 10.20.193.102:0/2768105024 learned_addr learned my addr 10.20.193.102:0/2768105024 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93df8a640 1 -- 10.20.193.102:0/2768105024 >> v1:10.20.193.102:6789/0 conn(0x7fd938142b80 legacy=0x7fd938142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93d789640 1 -- 10.20.193.102:0/2768105024 >> v1:10.20.193.102:6790/0 conn(0x7fd93808c560 legacy=0x7fd9381840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.519 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691757290 0 0) 0x7fd93808abc0 con 0x7fd938142b80 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): my global_id is 34851 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd920003620 con 0x7fd938142b80 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876201967 0 0) 0x7fd938088ea0 con 0x7fd93808c560 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient(hunting): my global_id is 94343 2026-02-21T22:32:44.520 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd93808abc0 con 0x7fd93808c560 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 844360978 0 0) 0x7fd920003620 con 0x7fd938142b80 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 >> v1:10.20.193.102:6790/0 conn(0x7fd93808c560 legacy=0x7fd9381840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 >> v1:10.20.193.178:6789/0 conn(0x7fd938143660 legacy=0x7fd9381a57e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 monclient: found mon.noname-a 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9380890e0 con 0x7fd938142b80 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 5 monclient: authenticate success, global_id 34851 2026-02-21T22:32:44.521 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd928003110 con 0x7fd938142b80 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: dump: 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.522 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd9280033e0 con 0x7fd938142b80 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 1 -- 10.20.193.102:0/2768105024 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd928004b50 con 0x7fd938142b80 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd93cf88640 10 monclient: dump: 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.523 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient: get_monmap_and_config success 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 4 set_mon_vals no callback set 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 10 monclient: shutdown 2026-02-21T22:32:44.524 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- 10.20.193.102:0/2768105024 >> v1:10.20.193.102:6789/0 conn(0x7fd938142b80 legacy=0x7fd938142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- 10.20.193.102:0/2768105024 shutdown_connections 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- 10.20.193.102:0/2768105024 >> 10.20.193.102:0/2768105024 conn(0x7fd938091a50 msgr2=0x7fd938091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- 10.20.193.102:0/2768105024 shutdown_connections 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.512+0000 7fd940215640 1 -- 10.20.193.102:0/2768105024 wait complete. 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: monmap: 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:32:44.525 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 Processor -- start 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- start start 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: init 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient(hunting): picked mon.noname-c con 0x7fd93808c560 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.526 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient(hunting): picked mon.noname-b con 0x7fd938142b80 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient(hunting): picked mon.noname-a con 0x7fd938143660 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93805d2c0 con 0x7fd938143660 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd938088a70 con 0x7fd938142b80 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93807fc00 con 0x7fd93808c560 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient(hunting): authenticate will time out at 2173.752657s 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93e78b640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fd938143660 0x7fd938195da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56352/0 (socket says 10.20.193.102:56352) 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93e78b640 1 -- 10.20.193.102:0/658467785 learned_addr learned my addr 10.20.193.102:0/658467785 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93df8a640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fd93808c560 0x7fd9381533e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:53980/0 (socket says 10.20.193.102:53980) 2026-02-21T22:32:44.527 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93d789640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.178:6789/0 conn(0x7fd938142b80 legacy=0x7fd938194680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93e78b640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.102:6789/0 conn(0x7fd938143660 legacy=0x7fd938195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93df8a640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.102:6790/0 conn(0x7fd93808c560 legacy=0x7fd9381533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516009876 0 0) 0x7fd93805d2c0 con 0x7fd938143660 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): my global_id is 34854 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd908003620 con 0x7fd938143660 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956485551 0 0) 0x7fd93807fc00 con 0x7fd93808c560 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): my global_id is 94346 2026-02-21T22:32:44.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd93805d2c0 con 0x7fd93808c560 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd93e78b640 1 --1- 10.20.193.102:0/658467785 >> v1:10.20.193.102:6789/0 conn(0x7fd938143660 0x7fd938195da0 :-1 s=OPENED pgs=650 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7fd9340059c0 mon_map magic: 0 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3066160153 0 0) 0x7fd908003620 con 0x7fd938143660 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd93807fc00 con 0x7fd938143660 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4020987349 0 0) 0x7fd93805d2c0 con 0x7fd93808c560 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd908003620 con 0x7fd93808c560 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9280048e0 con 0x7fd93808c560 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): dump: 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.533 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd9340059c0 con 0x7fd938143660 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient(hunting): dump: 2026-02-21T22:32:44.534 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 480371030 0 0) 0x7fd908003620 con 0x7fd93808c560 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.178:6789/0 conn(0x7fd938142b80 legacy=0x7fd938194680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:32:44.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.102:6789/0 conn(0x7fd938143660 legacy=0x7fd938195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 monclient: found mon.c 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd938081f60 con 0x7fd93808c560 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1761540361 0 0) 0x7fd93807fc00 con 0x7fd938143660 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 5 monclient: authenticate success, global_id 94346 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _renew_subs 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd938087bb0 con 0x7fd93808c560 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _renew_subs 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd93807f150 con 0x7fd93808c560 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd928003390 con 0x7fd93808c560 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fd928005130 con 0x7fd93808c560 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd9267fc640 10 monclient: dump: 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.537 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.516+0000 7fd927fff640 4 set_mon_vals no callback set 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd940215640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd940215640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd940215640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd938081720 con 0x7fd93808c560 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fd9280117c0 con 0x7fd93808c560 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(830..830 src has 527..830) ==== 3947+0+0 (unknown 2193512138 0 0) 0x7fd92804c5e0 con 0x7fd93808c560 2026-02-21T22:32:44.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd93d789640 10 monclient: get_auth_request con 0x7fd90803d510 auth_method 0 2026-02-21T22:32:44.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fd9280175d0 con 0x7fd93808c560 2026-02-21T22:32:44.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd9267fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.520+0000 7fd9267fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd938062ce0 con 0x7fd93808c560 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd9267fc640 1 -- 10.20.193.102:0/658467785 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v830) ==== 74+0+12903 (unknown 2963225162 0 1818123376) 0x7fd928016eb0 con 0x7fd93808c560 2026-02-21T22:32:44.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd9267fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:32:44.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd9267fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:32:44.607 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:32:44.607 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":830,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:32:44.281015+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T22:15:33.157956+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2026-02-21T22:16:00.732712+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"830","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":421,"snap_epoch":830,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":381,"length":1},{"begin":384,"length":1},{"begin":408,"length":1},{"begin":417,"length":3}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:32:44.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.178:6800/2549949439 conn(0x7fd90803d510 legacy=0x7fd90805d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 10 monclient: shutdown 2026-02-21T22:32:44.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:44.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 >> v1:10.20.193.102:6790/0 conn(0x7fd93808c560 legacy=0x7fd9381533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 shutdown_connections 2026-02-21T22:32:44.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 >> 10.20.193.102:0/658467785 conn(0x7fd938091a50 msgr2=0x7fd93805c650 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:44.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 shutdown_connections 2026-02-21T22:32:44.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.604+0000 7fd940215640 1 -- 10.20.193.102:0/658467785 wait complete. 2026-02-21T22:32:44.617 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:32:44.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: get_monmap_and_config 2026-02-21T22:32:44.687 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: monmap: 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:32:44.687033+0000 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:32:44.687033+0000 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:32:44.688 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 1 Processor -- start 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 1 -- start start 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: init 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-a con 0x7ff7cc08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-c con 0x7ff7cc08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-b con 0x7ff7cc08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc086a10 con 0x7ff7cc08ce00 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc08b130 con 0x7ff7cc08b7f0 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc08bd60 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.689 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7d1ad9640 10 monclient(hunting): authenticate will time out at 2173.922935s 2026-02-21T22:32:44.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7caffd640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7ff7cc08b7f0 legacy=0x7ff7cc181e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7cb7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7ff7cc08ce00 0x7ff7cc08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56368/0 (socket says 10.20.193.102:56368) 2026-02-21T22:32:44.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7cbfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 0x7ff7cc1a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:53988/0 (socket says 10.20.193.102:53988) 2026-02-21T22:32:44.690 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.684+0000 7ff7cb7fe640 1 -- 10.20.193.102:0/761615742 learned_addr learned my addr 10.20.193.102:0/761615742 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7cbfff640 1 -- 10.20.193.102:0/761615742 >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 legacy=0x7ff7cc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7cb7fe640 1 -- 10.20.193.102:0/761615742 >> v1:10.20.193.102:6789/0 conn(0x7ff7cc08ce00 legacy=0x7ff7cc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437596513 0 0) 0x7ff7cc08bd60 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.693 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): my global_id is 94349 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7b0003620 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717538954 0 0) 0x7ff7cc086a10 con 0x7ff7cc08ce00 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient(hunting): my global_id is 34857 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7cc08bd60 con 0x7ff7cc08ce00 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1901956866 0 0) 0x7ff7b0003620 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.694 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 >> v1:10.20.193.178:6789/0 conn(0x7ff7cc08b7f0 legacy=0x7ff7cc181e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 >> v1:10.20.193.102:6789/0 conn(0x7ff7cc08ce00 legacy=0x7ff7cc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 monclient: found mon.noname-b 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7cc08b370 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c0004620 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.695 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: dump: 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.696 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 5 monclient: authenticate success, global_id 94349 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff7c00048f0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 1 -- 10.20.193.102:0/761615742 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c0005fa0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7ca7fc640 10 monclient: dump: 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.697 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: get_monmap_and_config success 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 4 set_mon_vals no callback set 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: shutdown 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/761615742 >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 legacy=0x7ff7cc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/761615742 shutdown_connections 2026-02-21T22:32:44.698 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/761615742 >> 10.20.193.102:0/761615742 conn(0x7ff7cc091a50 msgr2=0x7ff7cc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/761615742 shutdown_connections 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/761615742 wait complete. 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: monmap: 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.699 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 Processor -- start 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- start start 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: init 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-b con 0x7ff7cc08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-a con 0x7ff7cc08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient(hunting): picked mon.noname-c con 0x7ff7cc08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc0827e0 con 0x7ff7cc08ce00 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc088a70 con 0x7ff7cc08b7f0 2026-02-21T22:32:44.700 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7cc080d20 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7d1ad9640 10 monclient(hunting): authenticate will time out at 2173.927765s 2026-02-21T22:32:44.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7cb7fe640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7ff7cc08b7f0 legacy=0x7ff7cc153190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7cbfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 0x7ff7cc195b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:54004/0 (socket says 10.20.193.102:54004) 2026-02-21T22:32:44.701 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.688+0000 7ff7cbfff640 1 -- 10.20.193.102:0/1216906150 learned_addr learned my addr 10.20.193.102:0/1216906150 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7cbfff640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 legacy=0x7ff7cc195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7caffd640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.102:6789/0 conn(0x7ff7cc08ce00 legacy=0x7ff7cc194430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074658014 0 0) 0x7ff7cc080d20 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.704 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): my global_id is 94352 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff79c003620 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240009104 0 0) 0x7ff7bc003e80 con 0x7ff7cc08ce00 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): my global_id is 34860 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7cc080d20 con 0x7ff7cc08ce00 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 152122 0 0) 0x7ff79c003620 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff7bc003e80 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c0005df0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): dump: 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.706 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2766546980 0 0) 0x7ff7cc080d20 con 0x7ff7cc08ce00 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff79c003620 con 0x7ff7cc08ce00 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7bc0046a0 con 0x7ff7cc08ce00 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient(hunting): dump: 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.707 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3365545048 0 0) 0x7ff7bc003e80 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.178:6789/0 conn(0x7ff7cc08b7f0 legacy=0x7ff7cc153190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.102:6789/0 conn(0x7ff7cc08ce00 legacy=0x7ff7cc194430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 monclient: found mon.c 2026-02-21T22:32:44.708 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7cc083080 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 5 monclient: authenticate success, global_id 94352 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _renew_subs 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff7cc083a40 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _renew_subs 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff7cc080270 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.709 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff7c0004890 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7ff7c00066f0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 10 monclient: dump: 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.710 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7c9ffb640 4 set_mon_vals no callback set 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7cc0008e0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7ff7c0012d80 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(830..830 src has 527..830) ==== 3947+0+0 (unknown 2193512138 0 0) 0x7ff7c004dac0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.692+0000 7ff7cb7fe640 10 monclient: get_auth_request con 0x7ff79c03d600 auth_method 0 2026-02-21T22:32:44.711 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.696+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7ff7c0018ab0 con 0x7ff7cc08d8e0 2026-02-21T22:32:44.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.696+0000 7ff7a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.712 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.696+0000 7ff7a7fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:32:44.782 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.780+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7cc082c50 con 0x7ff79c03d600 2026-02-21T22:32:44.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.780+0000 7ff7a7fff640 1 -- 10.20.193.102:0/1216906150 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+78398 (unknown 2965378022 0 871162538) 0x7ff7b8002fc0 con 0x7ff79c03d600 2026-02-21T22:32:44.784 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:32:44.785 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.178:6800/2549949439 conn(0x7ff79c03d600 legacy=0x7ff79c05dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 10 monclient: shutdown 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 >> v1:10.20.193.102:6790/0 conn(0x7ff7cc08d8e0 legacy=0x7ff7cc195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 shutdown_connections 2026-02-21T22:32:44.786 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 >> 10.20.193.102:0/1216906150 conn(0x7ff7cc091a50 msgr2=0x7ff7cc05c180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:44.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 shutdown_connections 2026-02-21T22:32:44.787 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.784+0000 7ff7d1ad9640 1 -- 10.20.193.102:0/1216906150 wait complete. 2026-02-21T22:32:44.794 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1350,"stamp":"2026-02-21T22:32:44.289878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":361999442,"num_objects":175,"num_object_clones":122,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":13,"num_read":7992,"num_read_kb":6279569,"num_write":5734,"num_write_kb":2199934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16382,"ondisk_log_size":16382,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":646920,"kb_used_data":449064,"kb_used_omap":8080,"kb_used_meta":189679,"kb_avail":565584120,"statfs":{"total":579820584960,"available":579158138880,"internally_reserved":0,"allocated":459841536,"data_stored":752522178,"data_compressed":200515010,"data_compressed_allocated":144084992,"data_compressed_original":358533265,"omap_allocated":8274009,"internal_metadata":194232231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8801569,"num_objects":4,"num_object_clones":4,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":31454,"num_write":26,"num_write_kb":8198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019165"},"pg_stats":[{"pgid":"3.f","version":"828'1098","reported_seq":5194,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.294132+0000","last_change":"2026-02-21T22:32:42.246179+0000","last_active":"2026-02-21T22:32:43.294132+0000","last_peered":"2026-02-21T22:32:43.294132+0000","last_clean":"2026-02-21T22:32:43.294132+0000","last_became_active":"2026-02-21T22:16:01.792185+0000","last_became_peered":"2026-02-21T22:16:01.792185+0000","last_unstale":"2026-02-21T22:32:43.294132+0000","last_undegraded":"2026-02-21T22:32:43.294132+0000","last_fullsized":"2026-02-21T22:32:43.294132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:57:25.496880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00172934,"stat_sum":{"num_bytes":30613846,"num_objects":14,"num_object_clones":10,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":672,"num_read_kb":587770,"num_write":385,"num_write_kb":137707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"3.e","version":"829'639","reported_seq":4652,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.316783+0000","last_change":"2026-02-21T22:32:42.246442+0000","last_active":"2026-02-21T22:32:43.316783+0000","last_peered":"2026-02-21T22:32:43.316783+0000","last_clean":"2026-02-21T22:32:43.316783+0000","last_became_active":"2026-02-21T22:16:01.792915+0000","last_became_peered":"2026-02-21T22:16:01.792915+0000","last_unstale":"2026-02-21T22:32:43.316783+0000","last_undegraded":"2026-02-21T22:32:43.316783+0000","last_fullsized":"2026-02-21T22:32:43.316783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:13:14.514770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001575927,"stat_sum":{"num_bytes":7533197,"num_objects":6,"num_object_clones":4,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":274,"num_read_kb":204854,"num_write":220,"num_write_kb":87469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"3.d","version":"828'902","reported_seq":4870,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336794+0000","last_change":"2026-02-21T22:32:42.239428+0000","last_active":"2026-02-21T22:32:43.336794+0000","last_peered":"2026-02-21T22:32:43.336794+0000","last_clean":"2026-02-21T22:32:43.336794+0000","last_became_active":"2026-02-21T22:16:01.792460+0000","last_became_peered":"2026-02-21T22:16:01.792460+0000","last_unstale":"2026-02-21T22:32:43.336794+0000","last_undegraded":"2026-02-21T22:32:43.336794+0000","last_fullsized":"2026-02-21T22:32:43.336794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":902,"log_dups_size":0,"ondisk_log_size":902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:21:36.566068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002623486,"stat_sum":{"num_bytes":21847737,"num_objects":10,"num_object_clones":7,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":340,"num_read_kb":236221,"num_write":311,"num_write_kb":91562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"3.c","version":"828'327","reported_seq":4338,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.294152+0000","last_change":"2026-02-21T22:32:42.246171+0000","last_active":"2026-02-21T22:32:43.294152+0000","last_peered":"2026-02-21T22:32:43.294152+0000","last_clean":"2026-02-21T22:32:43.294152+0000","last_became_active":"2026-02-21T22:16:01.792638+0000","last_became_peered":"2026-02-21T22:16:01.792638+0000","last_unstale":"2026-02-21T22:32:43.294152+0000","last_undegraded":"2026-02-21T22:32:43.294152+0000","last_fullsized":"2026-02-21T22:32:43.294152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:56:13.155998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001616492,"stat_sum":{"num_bytes":15351668,"num_objects":6,"num_object_clones":5,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":90925,"num_write":121,"num_write_kb":53408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"3.b","version":"824'677","reported_seq":4674,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.265287+0000","last_change":"2026-02-21T22:32:39.266114+0000","last_active":"2026-02-21T22:32:39.265287+0000","last_peered":"2026-02-21T22:32:39.265287+0000","last_clean":"2026-02-21T22:32:39.265287+0000","last_became_active":"2026-02-21T22:16:01.790207+0000","last_became_peered":"2026-02-21T22:16:01.790207+0000","last_unstale":"2026-02-21T22:32:39.265287+0000","last_undegraded":"2026-02-21T22:32:39.265287+0000","last_fullsized":"2026-02-21T22:32:39.265287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:18:43.504504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010846639999999999,"stat_sum":{"num_bytes":10151506,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":284,"num_read_kb":220770,"num_write":227,"num_write_kb":84967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"3.a","version":"0'0","reported_seq":3938,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336598+0000","last_change":"2026-02-21T22:32:42.237067+0000","last_active":"2026-02-21T22:32:43.336598+0000","last_peered":"2026-02-21T22:32:43.336598+0000","last_clean":"2026-02-21T22:32:43.336598+0000","last_became_active":"2026-02-21T22:16:01.792855+0000","last_became_peered":"2026-02-21T22:16:01.792855+0000","last_unstale":"2026-02-21T22:32:43.336598+0000","last_undegraded":"2026-02-21T22:32:43.336598+0000","last_fullsized":"2026-02-21T22:32:43.336598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:04:07.663209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000699966,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"3.9","version":"825'942","reported_seq":4942,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.276783+0000","last_change":"2026-02-21T22:32:39.276899+0000","last_active":"2026-02-21T22:32:39.276783+0000","last_peered":"2026-02-21T22:32:39.276783+0000","last_clean":"2026-02-21T22:32:39.276783+0000","last_became_active":"2026-02-21T22:16:01.793376+0000","last_became_peered":"2026-02-21T22:16:01.793376+0000","last_unstale":"2026-02-21T22:32:39.276783+0000","last_undegraded":"2026-02-21T22:32:39.276783+0000","last_fullsized":"2026-02-21T22:32:39.276783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":942,"log_dups_size":0,"ondisk_log_size":942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:23:47.870628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011780810000000001,"stat_sum":{"num_bytes":18326503,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":425,"num_read_kb":312310,"num_write":317,"num_write_kb":132819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"3.8","version":"828'1086","reported_seq":5129,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.278894+0000","last_change":"2026-02-21T22:32:42.239749+0000","last_active":"2026-02-21T22:32:43.278894+0000","last_peered":"2026-02-21T22:32:43.278894+0000","last_clean":"2026-02-21T22:32:43.278894+0000","last_became_active":"2026-02-21T22:16:01.790974+0000","last_became_peered":"2026-02-21T22:16:01.790974+0000","last_unstale":"2026-02-21T22:32:43.278894+0000","last_undegraded":"2026-02-21T22:32:43.278894+0000","last_fullsized":"2026-02-21T22:32:43.278894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1086,"log_dups_size":0,"ondisk_log_size":1086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:22:35.638530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004026646,"stat_sum":{"num_bytes":26206806,"num_objects":12,"num_object_clones":9,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":613,"num_read_kb":506190,"num_write":401,"num_write_kb":155122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"1.0","version":"14'112","reported_seq":1755,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336759+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:32:43.336759+0000","last_peered":"2026-02-21T22:32:43.336759+0000","last_clean":"2026-02-21T22:32:43.336759+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:32:43.336759+0000","last_undegraded":"2026-02-21T22:32:43.336759+0000","last_fullsized":"2026-02-21T22:32:43.336759+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1633,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.294316+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:32:43.294316+0000","last_peered":"2026-02-21T22:32:43.294316+0000","last_clean":"2026-02-21T22:32:43.294316+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:32:43.294316+0000","last_undegraded":"2026-02-21T22:32:43.294316+0000","last_fullsized":"2026-02-21T22:32:43.294316+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"825'1512","reported_seq":5508,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.268234+0000","last_change":"2026-02-21T22:32:39.268234+0000","last_active":"2026-02-21T22:32:39.268234+0000","last_peered":"2026-02-21T22:32:39.268234+0000","last_clean":"2026-02-21T22:32:39.268234+0000","last_became_active":"2026-02-21T22:16:01.790242+0000","last_became_peered":"2026-02-21T22:16:01.790242+0000","last_unstale":"2026-02-21T22:32:39.268234+0000","last_undegraded":"2026-02-21T22:32:39.268234+0000","last_fullsized":"2026-02-21T22:32:39.268234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1512,"log_dups_size":0,"ondisk_log_size":1512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:43:06.379542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002891298,"stat_sum":{"num_bytes":40392197,"num_objects":18,"num_object_clones":14,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":694,"num_read_kb":516293,"num_write":505,"num_write_kb":193753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":1618,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336702+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:32:43.336702+0000","last_peered":"2026-02-21T22:32:43.336702+0000","last_clean":"2026-02-21T22:32:43.336702+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:32:43.336702+0000","last_undegraded":"2026-02-21T22:32:43.336702+0000","last_fullsized":"2026-02-21T22:32:43.336702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"829'2149","reported_seq":6198,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.446169+0000","last_change":"2026-02-21T22:32:42.243105+0000","last_active":"2026-02-21T22:32:43.446169+0000","last_peered":"2026-02-21T22:32:43.446169+0000","last_clean":"2026-02-21T22:32:43.446169+0000","last_became_active":"2026-02-21T22:16:01.790287+0000","last_became_peered":"2026-02-21T22:16:01.790287+0000","last_unstale":"2026-02-21T22:32:43.446169+0000","last_undegraded":"2026-02-21T22:32:43.446169+0000","last_fullsized":"2026-02-21T22:32:43.446169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":2149,"log_dups_size":0,"ondisk_log_size":2149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:53:33.827252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003688668,"stat_sum":{"num_bytes":49648982,"num_objects":24,"num_object_clones":17,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":1,"num_read":1098,"num_read_kb":927973,"num_write":734,"num_write_kb":299193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":1620,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.279173+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:32:43.279173+0000","last_peered":"2026-02-21T22:32:43.279173+0000","last_clean":"2026-02-21T22:32:43.279173+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:32:43.279173+0000","last_undegraded":"2026-02-21T22:32:43.279173+0000","last_fullsized":"2026-02-21T22:32:43.279173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"825'1873","reported_seq":5872,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.269329+0000","last_change":"2026-02-21T22:32:39.269435+0000","last_active":"2026-02-21T22:32:39.269329+0000","last_peered":"2026-02-21T22:32:39.269329+0000","last_clean":"2026-02-21T22:32:39.269329+0000","last_became_active":"2026-02-21T22:16:01.791082+0000","last_became_peered":"2026-02-21T22:16:01.791082+0000","last_unstale":"2026-02-21T22:32:39.269329+0000","last_undegraded":"2026-02-21T22:32:39.269329+0000","last_fullsized":"2026-02-21T22:32:39.269329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1873,"log_dups_size":0,"ondisk_log_size":1873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:17:51.948493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039804710000000002,"stat_sum":{"num_bytes":16093068,"num_objects":12,"num_object_clones":7,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":870,"num_read_kb":674443,"num_write":604,"num_write_kb":227567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"2.2","version":"19'2","reported_seq":1634,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.294382+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:32:43.294382+0000","last_peered":"2026-02-21T22:32:43.294382+0000","last_clean":"2026-02-21T22:32:43.294382+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:32:43.294382+0000","last_undegraded":"2026-02-21T22:32:43.294382+0000","last_fullsized":"2026-02-21T22:32:43.294382+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"824'346","reported_seq":4352,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.275942+0000","last_change":"2026-02-21T22:32:39.276071+0000","last_active":"2026-02-21T22:32:39.275942+0000","last_peered":"2026-02-21T22:32:39.275942+0000","last_clean":"2026-02-21T22:32:39.275942+0000","last_became_active":"2026-02-21T22:16:01.792669+0000","last_became_peered":"2026-02-21T22:16:01.792669+0000","last_unstale":"2026-02-21T22:32:39.275942+0000","last_undegraded":"2026-02-21T22:32:39.275942+0000","last_fullsized":"2026-02-21T22:32:39.275942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:07:30.858386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031297900000000001,"stat_sum":{"num_bytes":6236024,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":246,"num_read_kb":223968,"num_write":131,"num_write_kb":56158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"3.5","version":"829'1380","reported_seq":5377,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.446660+0000","last_change":"2026-02-21T22:32:42.248471+0000","last_active":"2026-02-21T22:32:43.446660+0000","last_peered":"2026-02-21T22:32:43.446660+0000","last_clean":"2026-02-21T22:32:43.446660+0000","last_became_active":"2026-02-21T22:16:01.793498+0000","last_became_peered":"2026-02-21T22:16:01.793498+0000","last_unstale":"2026-02-21T22:32:43.446660+0000","last_undegraded":"2026-02-21T22:32:43.446660+0000","last_fullsized":"2026-02-21T22:32:43.446660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:11:42.863052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003770388,"stat_sum":{"num_bytes":38457834,"num_objects":19,"num_object_clones":15,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":574,"num_read_kb":470698,"num_write":484,"num_write_kb":212930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":1619,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.265690+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:39.265690+0000","last_peered":"2026-02-21T22:32:39.265690+0000","last_clean":"2026-02-21T22:32:39.265690+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:32:39.265690+0000","last_undegraded":"2026-02-21T22:32:39.265690+0000","last_fullsized":"2026-02-21T22:32:39.265690+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"824'944","reported_seq":4977,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.265354+0000","last_change":"2026-02-21T22:32:39.266344+0000","last_active":"2026-02-21T22:32:39.265354+0000","last_peered":"2026-02-21T22:32:39.265354+0000","last_clean":"2026-02-21T22:32:39.265354+0000","last_became_active":"2026-02-21T22:16:01.791089+0000","last_became_peered":"2026-02-21T22:16:01.791089+0000","last_unstale":"2026-02-21T22:32:39.265354+0000","last_undegraded":"2026-02-21T22:32:39.265354+0000","last_fullsized":"2026-02-21T22:32:39.265354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:58:06.509517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001245683,"stat_sum":{"num_bytes":23158002,"num_objects":9,"num_object_clones":6,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":549,"num_read_kb":477189,"num_write":301,"num_write_kb":124811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"19b","length":"1"},{"start":"1a1","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":1618,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336698+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:32:43.336698+0000","last_peered":"2026-02-21T22:32:43.336698+0000","last_clean":"2026-02-21T22:32:43.336698+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:32:43.336698+0000","last_undegraded":"2026-02-21T22:32:43.336698+0000","last_fullsized":"2026-02-21T22:32:43.336698+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"828'1089","reported_seq":5074,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.279003+0000","last_change":"2026-02-21T22:32:42.237742+0000","last_active":"2026-02-21T22:32:43.279003+0000","last_peered":"2026-02-21T22:32:43.279003+0000","last_clean":"2026-02-21T22:32:43.279003+0000","last_became_active":"2026-02-21T22:16:01.790495+0000","last_became_peered":"2026-02-21T22:16:01.790495+0000","last_unstale":"2026-02-21T22:32:43.279003+0000","last_undegraded":"2026-02-21T22:32:43.279003+0000","last_fullsized":"2026-02-21T22:32:43.279003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1089,"log_dups_size":0,"ondisk_log_size":1089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:16:09.778171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020066350000000001,"stat_sum":{"num_bytes":27135702,"num_objects":11,"num_object_clones":8,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":461,"num_read_kb":286748,"num_write":378,"num_write_kb":156875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":1618,"reported_epoch":825,"state":"active+clean","last_fresh":"2026-02-21T22:32:39.265649+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:39.265649+0000","last_peered":"2026-02-21T22:32:39.265649+0000","last_clean":"2026-02-21T22:32:39.265649+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:32:39.265649+0000","last_undegraded":"2026-02-21T22:32:39.265649+0000","last_fullsized":"2026-02-21T22:32:39.265649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"829'1303","reported_seq":5350,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.313442+0000","last_change":"2026-02-21T22:32:42.242194+0000","last_active":"2026-02-21T22:32:43.313442+0000","last_peered":"2026-02-21T22:32:43.313442+0000","last_clean":"2026-02-21T22:32:43.313442+0000","last_became_active":"2026-02-21T22:16:01.790704+0000","last_became_peered":"2026-02-21T22:16:01.790704+0000","last_unstale":"2026-02-21T22:32:43.313442+0000","last_undegraded":"2026-02-21T22:32:43.313442+0000","last_fullsized":"2026-02-21T22:32:43.313442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:16:00.782794+0000","last_clean_scrub_stamp":"2026-02-21T22:16:00.782794+0000","objects_scrubbed":0,"log_size":1303,"log_dups_size":0,"ondisk_log_size":1303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:42:46.323102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028451629999999999,"stat_sum":{"num_bytes":30255983,"num_objects":15,"num_object_clones":11,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":658,"num_read_kb":543144,"num_write":468,"num_write_kb":182919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17d","length":"1"},{"start":"180","length":"1"},{"start":"198","length":"1"},{"start":"1a1","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":1617,"reported_epoch":829,"state":"active+clean","last_fresh":"2026-02-21T22:32:43.336640+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:32:43.336640+0000","last_peered":"2026-02-21T22:32:43.336640+0000","last_clean":"2026-02-21T22:32:43.336640+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:32:43.336640+0000","last_undegraded":"2026-02-21T22:32:43.336640+0000","last_fullsized":"2026-02-21T22:32:43.336640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":361409055,"num_objects":171,"num_object_clones":122,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":13,"num_read":7906,"num_read_kb":6279496,"num_write":5587,"num_write_kb":2197260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":453029888,"data_stored":758165663,"data_compressed":205170319,"data_compressed_allocated":147386368,"data_compressed_original":366307239,"omap_allocated":0,"internal_metadata":0},"log_size":16267,"ondisk_log_size":16267,"up":32,"acting":32,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607762,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82772,"kb_used_data":55812,"kb_used_omap":1147,"kb_used_meta":25796,"kb_avail":94289068,"statfs":{"total":96636764160,"available":96552005632,"internally_reserved":0,"allocated":57151488,"data_stored":94718396,"data_compressed":22286861,"data_compressed_allocated":16936960,"data_compressed_original":42496785,"omap_allocated":1175422,"internal_metadata":26415234},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107728,"kb_used_data":80768,"kb_used_omap":1283,"kb_used_meta":25660,"kb_avail":94264112,"statfs":{"total":96636764160,"available":96526450688,"internally_reserved":0,"allocated":82706432,"data_stored":131187247,"data_compressed":32927150,"data_compressed_allocated":25284608,"data_compressed_original":56116987,"omap_allocated":1314438,"internal_metadata":26276218},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58280,"kb_used_data":31320,"kb_used_omap":842,"kb_used_meta":26101,"kb_avail":94313560,"statfs":{"total":96636764160,"available":96577085440,"internally_reserved":0,"allocated":32071680,"data_stored":49803332,"data_compressed":13618743,"data_compressed_allocated":9789440,"data_compressed_original":23132068,"omap_allocated":862852,"internal_metadata":26727804},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152664,"kb_used_data":107656,"kb_used_omap":1762,"kb_used_meta":43229,"kb_avail":94219176,"statfs":{"total":96636764160,"available":96480436224,"internally_reserved":0,"allocated":110239744,"data_stored":182066630,"data_compressed":47044140,"data_compressed_allocated":32120832,"data_compressed_original":82128002,"omap_allocated":1805278,"internal_metadata":44266530},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640465,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145472,"kb_used_data":100464,"kb_used_omap":1783,"kb_used_meta":43208,"kb_avail":94226368,"statfs":{"total":96636764160,"available":96487800832,"internally_reserved":0,"allocated":102875136,"data_stored":174176433,"data_compressed":53607010,"data_compressed_allocated":38092800,"data_compressed_original":95346652,"omap_allocated":1826512,"internal_metadata":44245296},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640467,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100004,"kb_used_data":73044,"kb_used_omap":1259,"kb_used_meta":25684,"kb_avail":94271836,"statfs":{"total":96636764160,"available":96534360064,"internally_reserved":0,"allocated":74797056,"data_stored":120570140,"data_compressed":31031106,"data_compressed_allocated":21860352,"data_compressed_original":59312771,"omap_allocated":1289507,"internal_metadata":26301149},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75538432,"data_stored":123692228,"data_compressed":31914759,"data_compressed_allocated":21807104,"data_compressed_original":60526660,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100421632,"data_stored":172832500,"data_compressed":53607010,"data_compressed_allocated":38092800,"data_compressed_original":95346652,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109707264,"data_stored":183557562,"data_compressed":48831254,"data_compressed_allocated":34078720,"data_compressed_original":84985522,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29523968,"data_stored":47850866,"data_compressed":13610576,"data_compressed_allocated":9736192,"data_compressed_original":22550436,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":54693888,"data_stored":93369936,"data_compressed":22286861,"data_compressed_allocated":16936960,"data_compressed_original":42496785,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":83144704,"data_stored":136862571,"data_compressed":34919859,"data_compressed_allocated":26734592,"data_compressed_original":60401184,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:32:44.796 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-21T22:32:44.797 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-21T22:32:44.867 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: get_monmap_and_config 2026-02-21T22:32:44.867 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: monmap: 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:32:44.867940+0000 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:32:44.867940+0000 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:32:44.868 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 1 Processor -- start 2026-02-21T22:32:44.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 1 -- start start 2026-02-21T22:32:44.869 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: init 2026-02-21T22:32:44.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.873 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-c con 0x7fdf34088ca0 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.864+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-a con 0x7fdf34089780 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-b con 0x7fdf340860d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf34089fa0 con 0x7fdf34089780 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3408a420 con 0x7fdf34088ca0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3405edc0 con 0x7fdf340860d0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient(hunting): authenticate will time out at 2174.104064s 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3a165640 1 -- >> v1:10.20.193.178:6789/0 conn(0x7fdf34088ca0 legacy=0x7fdf34089070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39964640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdf34089780 0x7fdf34180d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56392/0 (socket says 10.20.193.102:56392) 2026-02-21T22:32:44.874 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39964640 1 -- 10.20.193.102:0/451556387 learned_addr learned my addr 10.20.193.102:0/451556387 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39964640 1 -- 10.20.193.102:0/451556387 >> v1:10.20.193.102:6789/0 conn(0x7fdf34089780 legacy=0x7fdf34180d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3a966640 1 -- 10.20.193.102:0/451556387 >> v1:10.20.193.102:6790/0 conn(0x7fdf340860d0 legacy=0x7fdf341a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156846046 0 0) 0x7fdf34089fa0 con 0x7fdf34089780 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): my global_id is 34863 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf1c003620 con 0x7fdf34089780 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611038397 0 0) 0x7fdf3405edc0 con 0x7fdf340860d0 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.875 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient(hunting): my global_id is 94355 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf34089fa0 con 0x7fdf340860d0 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2826767599 0 0) 0x7fdf1c003620 con 0x7fdf34089780 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 >> v1:10.20.193.102:6790/0 conn(0x7fdf340860d0 legacy=0x7fdf341a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 >> v1:10.20.193.178:6789/0 conn(0x7fdf34088ca0 legacy=0x7fdf34089070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 monclient: found mon.noname-a 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf34089d60 con 0x7fdf34089780 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.876 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 5 monclient: authenticate success, global_id 34863 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf28003110 con 0x7fdf34089780 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: dump: 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.877 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf28003450 con 0x7fdf34089780 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 1 -- 10.20.193.102:0/451556387 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf28004b50 con 0x7fdf34089780 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf39163640 10 monclient: dump: 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.878 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient: get_monmap_and_config success 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 4 set_mon_vals no callback set 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient: shutdown 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:44.879 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/451556387 >> v1:10.20.193.102:6789/0 conn(0x7fdf34089780 legacy=0x7fdf34180d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/451556387 shutdown_connections 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/451556387 >> 10.20.193.102:0/451556387 conn(0x7fdf340917a0 msgr2=0x7fdf34091ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/451556387 shutdown_connections 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/451556387 wait complete. 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient: build_initial_monmap 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.868+0000 7fdf3c3f0640 10 monclient: monmap: 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:32:44.880 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 Processor -- start 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- start start 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: init 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _reopen_session rank -1 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-a con 0x7fdf340860d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:32:44.881 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-b con 0x7fdf34088ca0 addr v1:10.20.193.178:6789/0 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient(hunting): picked mon.noname-c con 0x7fdf34089780 addr v1:10.20.193.102:6790/0 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3413b350 con 0x7fdf340860d0 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3413d6b0 con 0x7fdf34088ca0 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3413ce10 con 0x7fdf34089780 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient(hunting): _renew_subs 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient(hunting): authenticate will time out at 2174.108654s 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3a165640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fdf340860d0 0x7fdf34150f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:56394/0 (socket says 10.20.193.102:56394) 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3a165640 1 -- 10.20.193.102:0/1524467730 learned_addr learned my addr 10.20.193.102:0/1524467730 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:32:44.882 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf39964640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.178:6789/0 conn(0x7fdf34088ca0 legacy=0x7fdf3413fbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:10.20.193.178:6789/0 2026-02-21T22:32:44.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3a165640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.102:6789/0 conn(0x7fdf340860d0 legacy=0x7fdf34150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.885 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3a966640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.102:6790/0 conn(0x7fdf34089780 legacy=0x7fdf341932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455534909 0 0) 0x7fdf3413b350 con 0x7fdf340860d0 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): my global_id is 34866 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf04003620 con 0x7fdf340860d0 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197592470 0 0) 0x7fdf3413ce10 con 0x7fdf34089780 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _negotiate 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): my global_id is 94358 2026-02-21T22:32:44.886 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf3413b350 con 0x7fdf34089780 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4228708933 0 0) 0x7fdf04003620 con 0x7fdf340860d0 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf3413ce10 con 0x7fdf340860d0 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf24004590 con 0x7fdf340860d0 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): dump: 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.887 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3381283980 0 0) 0x7fdf3413b350 con 0x7fdf34089780 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf04003620 con 0x7fdf34089780 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf30004900 con 0x7fdf34089780 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient(hunting): dump: 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.888 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2097657317 0 0) 0x7fdf3413ce10 con 0x7fdf340860d0 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.102:6790/0 conn(0x7fdf34089780 legacy=0x7fdf341932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.178:6789/0 conn(0x7fdf34088ca0 legacy=0x7fdf3413fbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: _finish_hunting 0 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 monclient: found mon.a 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.889 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3413beb0 con 0x7fdf340860d0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: _finish_auth 0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: _check_auth_tickets 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 5 monclient: authenticate success, global_id 34866 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _renew_subs 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdf34140300 con 0x7fdf340860d0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _renew_subs 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdf34194b90 con 0x7fdf340860d0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf24004860 con 0x7fdf340860d0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf24004bc0 con 0x7fdf340860d0 2026-02-21T22:32:44.890 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf1affd640 10 monclient: dump: 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf38962640 4 set_mon_vals no callback set 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.872+0000 7fdf3c3f0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.891 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf3c3f0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf3c3f0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf00005130 con 0x7fdf340860d0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdf24004e90 con 0x7fdf340860d0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(830..830 src has 527..830) ==== 3947+0+0 (unknown 2193512138 0 0) 0x7fdf2404c770 con 0x7fdf340860d0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf39964640 10 monclient: get_auth_request con 0x7fdf0403d610 auth_method 0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fdf24017700 con 0x7fdf340860d0 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:32:44.892 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.876+0000 7fdf1affd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:32:44.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.960+0000 7fdf3c3f0640 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-21T22:32:44.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.960+0000 7fdf3c3f0640 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-21T22:32:44.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.960+0000 7fdf3c3f0640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:32:44.963 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:44.960+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf00005420 con 0x7fdf340860d0 2026-02-21T22:32:45.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf1affd640 1 -- 10.20.193.102:0/1524467730 <== mon.0 v1:10.20.193.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]=0 pool 'unique_pool_0' removed v831) ==== 175+0+0 (unknown 2237243167 0 0) 0x7fdf24016fe0 con 0x7fdf340860d0 2026-02-21T22:32:45.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2026-02-21T22:32:45.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf1affd640 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' removed 2026-02-21T22:32:45.290 INFO:teuthology.orchestra.run.trial102.stderr:pool 'unique_pool_0' removed 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.178:6800/2549949439 conn(0x7fdf0403d610 legacy=0x7fdf0405dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 10 monclient: shutdown 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 >> v1:10.20.193.102:6789/0 conn(0x7fdf340860d0 legacy=0x7fdf34150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 shutdown_connections 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 >> 10.20.193.102:0/1524467730 conn(0x7fdf340917a0 msgr2=0x7fdf3408d240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 shutdown_connections 2026-02-21T22:32:45.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:32:45.288+0000 7fdf3c3f0640 1 -- 10.20.193.102:0/1524467730 wait complete. 2026-02-21T22:32:45.301 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2026-02-21T22:32:45.312 INFO:tasks.mon_thrash:joining mon_thrasher 2026-02-21T22:32:48.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:48.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:53.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:53.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:53.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:53.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:32:53.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:53.556 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:32:53.596 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:53.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:32:58.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:32:58.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:03.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:03.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:08.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:08.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:13.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:13.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:15.830 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:33:15.830 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2026-02-21T22:33:15.830 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-21T22:33:15.830 DEBUG:teuthology.orchestra.run.trial178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2026-02-21T22:33:15.831 INFO:tasks.ceph.mon.b:Started 2026-02-21T22:33:15.831 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:33:15.832 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 10 monclient: get_monmap_and_config 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 10 monclient: build_initial_monmap 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 10 monclient: monmap: 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:15.899972+0000 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:15.899972+0000 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:15.899 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 1 Processor -- start 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 1 -- start start 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.895+0000 7fb0a1bba640 10 monclient: init 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-c con 0x7fb09c088d80 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-a con 0x7fb09c08a1d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-b con 0x7fb09c089860 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c085920 con 0x7fb09c08a1d0 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c05e780 con 0x7fb09c088d80 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c086330 con 0x7fb09c089860 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient(hunting): authenticate will time out at 2205.136153s 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09affd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fb09c08a1d0 0x7fb09c180c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37712/0 (socket says 10.20.193.102:37712) 2026-02-21T22:33:15.900 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09affd640 1 -- 10.20.193.102:0/2739074198 learned_addr learned my addr 10.20.193.102:0/2739074198 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:15.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09bfff640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.102:6790/0 conn(0x7fb09c089860 legacy=0x7fb09c1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09affd640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.102:6789/0 conn(0x7fb09c08a1d0 legacy=0x7fb09c180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.902 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09b7fe640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.178:6789/0 conn(0x7fb09c088d80 legacy=0x7fb09c089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295291775 0 0) 0x7fb09c086330 con 0x7fb09c089860 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient(hunting): my global_id is 94361 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb080003620 con 0x7fb09c089860 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260865718 0 0) 0x7fb080003620 con 0x7fb09c089860 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.178:6789/0 conn(0x7fb09c088d80 legacy=0x7fb09c089150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.102:6789/0 conn(0x7fb09c08a1d0 legacy=0x7fb09c180c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 monclient: found mon.noname-b 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c08c6b0 con 0x7fb09c089860 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: _finish_auth 0 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:15.903 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb090004670 con 0x7fb09c089860 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: dump: 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 5 monclient: authenticate success, global_id 94361 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb090004940 con 0x7fb09c089860 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 1 -- 10.20.193.102:0/2739074198 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb090004dd0 con 0x7fb09c089860 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb09a7fc640 10 monclient: dump: 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:15.904 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 4 set_mon_vals no callback set 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: shutdown 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- 10.20.193.102:0/2739074198 >> v1:10.20.193.102:6790/0 conn(0x7fb09c089860 legacy=0x7fb09c1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- 10.20.193.102:0/2739074198 shutdown_connections 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- 10.20.193.102:0/2739074198 >> 10.20.193.102:0/2739074198 conn(0x7fb09c091a50 msgr2=0x7fb09c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- 10.20.193.102:0/2739074198 shutdown_connections 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 1 -- 10.20.193.102:0/2739074198 wait complete. 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: build_initial_monmap 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.899+0000 7fb0a1bba640 10 monclient: monmap: 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:15.905 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 Processor -- start 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- start start 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: init 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-b con 0x7fb09c088d80 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-a con 0x7fb09c089860 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient(hunting): picked mon.noname-c con 0x7fb09c08a1d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c0818f0 con 0x7fb09c089860 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c082880 con 0x7fb09c088d80 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09c087e10 con 0x7fb09c08a1d0 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient(hunting): authenticate will time out at 2205.140337s 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb09bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fb09c08a1d0 0x7fb09c07f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:58976/0 (socket says 10.20.193.102:58976) 2026-02-21T22:33:15.906 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb09bfff640 1 -- 10.20.193.102:0/826097442 learned_addr learned my addr 10.20.193.102:0/826097442 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb09bfff640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.102:6790/0 conn(0x7fb09c08a1d0 legacy=0x7fb09c07f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb09affd640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.102:6789/0 conn(0x7fb09c089860 legacy=0x7fb09c07f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2690700769 0 0) 0x7fb09c087e10 con 0x7fb09c08a1d0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): my global_id is 94364 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb06c003620 con 0x7fb09c08a1d0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283318354 0 0) 0x7fb09c0818f0 con 0x7fb09c089860 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): my global_id is 34869 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb09c087e10 con 0x7fb09c089860 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb09b7fe640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.178:6789/0 conn(0x7fb09c088d80 legacy=0x7fb09c07eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1932144210 0 0) 0x7fb06c003620 con 0x7fb09c08a1d0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb09c0818f0 con 0x7fb09c08a1d0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb090004c20 con 0x7fb09c08a1d0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): dump: 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.908 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 740206775 0 0) 0x7fb09c0818f0 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.178:6789/0 conn(0x7fb09c088d80 legacy=0x7fb09c07eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.102:6789/0 conn(0x7fb09c089860 legacy=0x7fb09c07f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 monclient: found mon.c 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c088a70 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: _finish_auth 0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 5 monclient: authenticate success, global_id 94364 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _renew_subs 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c0825a0 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _renew_subs 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fb09c0851c0 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb090004940 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fb0900055a0 con 0x7fb09c08a1d0 2026-02-21T22:33:15.909 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb077fff640 10 monclient: dump: 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb099ffb640 4 set_mon_vals no callback set 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.903+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09c082220 con 0x7fb09c08a1d0 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fb090005870 con 0x7fb09c08a1d0 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7fb09004d980 con 0x7fb09c08a1d0 2026-02-21T22:33:15.910 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb09b7fe640 10 monclient: get_auth_request con 0x7fb06c03d540 auth_method 0 2026-02-21T22:33:15.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fb090018b40 con 0x7fb09c08a1d0 2026-02-21T22:33:15.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb077fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:15.911 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:15.907+0000 7fb077fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:16.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:33:16.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:33:16.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb09c062ce0 con 0x7fb09c08a1d0 2026-02-21T22:33:16.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb077fff640 1 -- 10.20.193.102:0/826097442 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3285036197) 0x7fb090018420 con 0x7fb09c08a1d0 2026-02-21T22:33:16.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb077fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:33:16.027 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb077fff640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:16.027 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:33:16.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.178:6800/2549949439 conn(0x7fb06c03d540 legacy=0x7fb06c05d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 10 monclient: shutdown 2026-02-21T22:33:16.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 >> v1:10.20.193.102:6790/0 conn(0x7fb09c08a1d0 legacy=0x7fb09c07f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 shutdown_connections 2026-02-21T22:33:16.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 >> 10.20.193.102:0/826097442 conn(0x7fb09c091a50 msgr2=0x7fb09c05cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 shutdown_connections 2026-02-21T22:33:16.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.023+0000 7fb0a1bba640 1 -- 10.20.193.102:0/826097442 wait complete. 2026-02-21T22:33:16.037 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:33:16.038 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2026-02-21T22:33:16.038 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2026-02-21T22:33:16.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: get_monmap_and_config 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: monmap: 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:16.103013+0000 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:16.103013+0000 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:16.104 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 1 Processor -- start 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 1 -- start start 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: init 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-c con 0x7f64fc085970 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-a con 0x7f64fc085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-b con 0x7f64fc1854c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc08df30 con 0x7f64fc085d40 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc08d900 con 0x7f64fc085970 2026-02-21T22:33:16.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc056dd0 con 0x7f64fc1854c0 2026-02-21T22:33:16.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f6502e69640 10 monclient(hunting): authenticate will time out at 2205.338962s 2026-02-21T22:33:16.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f64f3fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f64fc085d40 0x7f64fc183da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37738/0 (socket says 10.20.193.102:37738) 2026-02-21T22:33:16.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.099+0000 7f64f3fff640 1 -- 10.20.193.102:0/3424720833 learned_addr learned my addr 10.20.193.102:0/3424720833 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f3fff640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.102:6789/0 conn(0x7f64fc085d40 legacy=0x7f64fc183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f65013df640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.102:6790/0 conn(0x7f64fc1854c0 legacy=0x7f64fc1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.111 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468603719 0 0) 0x7f64fc08df30 con 0x7f64fc085d40 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient(hunting): my global_id is 34872 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e8003620 con 0x7f64fc085d40 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6500bde640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.178:6789/0 conn(0x7f64fc085970 legacy=0x7f64fc08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3237402225 0 0) 0x7f64e8003620 con 0x7f64fc085d40 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.102:6790/0 conn(0x7f64fc1854c0 legacy=0x7f64fc1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.178:6789/0 conn(0x7f64fc085970 legacy=0x7f64fc08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.112 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 monclient: found mon.noname-a 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc08d620 con 0x7f64fc085d40 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64e4003110 con 0x7f64fc085d40 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: dump: 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.113 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 5 monclient: authenticate success, global_id 34872 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f64e40033e0 con 0x7f64fc085d40 2026-02-21T22:33:16.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 1 -- 10.20.193.102:0/3424720833 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64e4004b50 con 0x7f64fc085d40 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f64f37fe640 10 monclient: dump: 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.115 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 4 set_mon_vals no callback set 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: shutdown 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- 10.20.193.102:0/3424720833 >> v1:10.20.193.102:6789/0 conn(0x7f64fc085d40 legacy=0x7f64fc183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- 10.20.193.102:0/3424720833 shutdown_connections 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- 10.20.193.102:0/3424720833 >> 10.20.193.102:0/3424720833 conn(0x7f64fc054e40 msgr2=0x7f64fc092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- 10.20.193.102:0/3424720833 shutdown_connections 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- 10.20.193.102:0/3424720833 wait complete. 2026-02-21T22:33:16.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: monmap: 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 Processor -- start 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- start start 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: init 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-c con 0x7f64fc085970 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-a con 0x7f64fc085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient(hunting): picked mon.noname-b con 0x7f64fc1854c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc07fdc0 con 0x7f64fc085d40 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc07e5c0 con 0x7f64fc1854c0 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc1852b0 con 0x7f64fc085970 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6502e69640 10 monclient(hunting): authenticate will time out at 2205.342903s 2026-02-21T22:33:16.118 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6500bde640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f64fc085970 0x7f64fc1432a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59002/0 (socket says 10.20.193.102:59002) 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.103+0000 7f6500bde640 1 -- 10.20.193.102:0/386958562 learned_addr learned my addr 10.20.193.102:0/386958562 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6500bde640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6790/0 conn(0x7f64fc085970 legacy=0x7f64fc1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f3fff640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc085d40 legacy=0x7f64fc183b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397225778 0 0) 0x7f64fc07fdc0 con 0x7f64fc085d40 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): my global_id is 34875 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64cc003620 con 0x7f64fc085d40 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f65013df640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.178:6789/0 conn(0x7f64fc1854c0 legacy=0x7f64fc1219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3409113725 0 0) 0x7f64cc003620 con 0x7f64fc085d40 2026-02-21T22:33:16.119 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64fc07fdc0 con 0x7f64fc085d40 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64e4002b70 con 0x7f64fc085d40 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): dump: 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.120 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2580234833 0 0) 0x7f64fc07fdc0 con 0x7f64fc085d40 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6790/0 conn(0x7f64fc085970 legacy=0x7f64fc1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.178:6789/0 conn(0x7f64fc1854c0 legacy=0x7f64fc1219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 monclient: found mon.a 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0842d0 con 0x7f64fc085d40 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.121 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 5 monclient: authenticate success, global_id 34875 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 10 monclient: _renew_subs 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc07da60 con 0x7f64fc085d40 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 10 monclient: _renew_subs 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f64fc07f800 con 0x7f64fc085d40 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f64e4004b90 con 0x7f64fc085d40 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64e4005130 con 0x7f64fc085d40 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.122 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f2ffd640 4 set_mon_vals no callback set 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 10 monclient: dump: 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.123 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f64fc08f100 con 0x7f64fc07d040 2026-02-21T22:33:16.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f64e40117c0 con 0x7f64fc085d40 2026-02-21T22:33:16.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f64e404c4f0 con 0x7f64fc085d40 2026-02-21T22:33:16.124 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.107+0000 7f65013df640 10 monclient: get_auth_request con 0x7f64cc03d450 auth_method 0 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f6500bde640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc07d040 legacy=0x7f64fc089d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547556254 0 0) 0x7f64fc08f100 con 0x7f64fc07d040 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient(hunting): my global_id is 34878 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc07fdc0 con 0x7f64fc07d040 2026-02-21T22:33:16.127 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1012075179 0 0) 0x7f64fc07fdc0 con 0x7f64fc07d040 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64fc08f100 con 0x7f64fc07d040 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64ec003270 con 0x7f64fc07d040 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient: dump: 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.128 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3951794221 0 0) 0x7f64fc08f100 con 0x7f64fc07d040 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64fc089a80 con 0x7f64fc07d040 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f64ec0058c0 con 0x7f64fc07d040 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.129 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc07d040 legacy=0x7f64fc089d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.111+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc07d040 legacy=0x7f64fc089d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f64fc062ce0 con 0x7f64fc08a480 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6500bde640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc08a480 legacy=0x7f64fc124120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004063840 0 0) 0x7f64fc062ce0 con 0x7f64fc08a480 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient(hunting): my global_id is 34881 2026-02-21T22:33:16.130 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc08f100 con 0x7f64fc08a480 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 312067286 0 0) 0x7f64fc08f100 con 0x7f64fc08a480 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64fc062ce0 con 0x7f64fc08a480 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f64ec003b70 con 0x7f64fc08a480 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient: dump: 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.131 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1835046220 0 0) 0x7f64fc062ce0 con 0x7f64fc08a480 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 --> v1:10.20.193.102:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64fc0564d0 con 0x7f64fc08a480 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 <== mon.0 v1:10.20.193.102:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4516 (unknown 0 0 367009024) 0x7f64ec005b40 con 0x7f64fc08a480 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc08a480 legacy=0x7f64fc124120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.132 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f64f17fa640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc08a480 legacy=0x7f64fc124120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "state": "leader", 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 64, 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 944984, 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:33:16.133 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.134 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.135 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:33:16.136 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.137 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.138 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.139 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.140 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:33:16.141 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.142 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "osd": [ 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: "num": 6 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.143 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: "num": 5 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:33:16.144 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:16.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.178:6800/2549949439 conn(0x7f64cc03d450 legacy=0x7f64cc05d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 10 monclient: shutdown 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 >> v1:10.20.193.102:6789/0 conn(0x7f64fc085d40 legacy=0x7f64fc183b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f65013df640 1 -- 10.20.193.102:0/386958562 reap_dead start 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f65013df640 1 -- 10.20.193.102:0/386958562 reap_dead start 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 shutdown_connections 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 >> 10.20.193.102:0/386958562 conn(0x7f64fc054e40 msgr2=0x7f64fc087760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 shutdown_connections 2026-02-21T22:33:16.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.115+0000 7f6502e69640 1 -- 10.20.193.102:0/386958562 wait complete. 2026-02-21T22:33:16.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:33:16.147 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2026-02-21T22:33:16.147 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: get_monmap_and_config 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: monmap: 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:16.208091+0000 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:16.208091+0000 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:16.207 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 1 Processor -- start 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 1 -- start start 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: init 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-c con 0x7f80ec085970 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-b con 0x7f80ec085d40 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-a con 0x7f80ec1854c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec08df30 con 0x7f80ec1854c0 2026-02-21T22:33:16.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.203+0000 7f80f2eaf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec08d900 con 0x7f80ec085970 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec056dd0 con 0x7f80ec085d40 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient(hunting): authenticate will time out at 2205.444020s 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f1425640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f80ec1854c0 0x7f80ec1a58a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37778/0 (socket says 10.20.193.102:37778) 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e3fff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f80ec085d40 0x7f80ec183da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59012/0 (socket says 10.20.193.102:59012) 2026-02-21T22:33:16.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f1425640 1 -- 10.20.193.102:0/2338928841 learned_addr learned my addr 10.20.193.102:0/2338928841 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f1425640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.102:6789/0 conn(0x7f80ec1854c0 legacy=0x7f80ec1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e3fff640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.102:6790/0 conn(0x7f80ec085d40 legacy=0x7f80ec183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f0c24640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.178:6789/0 conn(0x7f80ec085970 legacy=0x7f80ec08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896007530 0 0) 0x7f80ec08df30 con 0x7f80ec1854c0 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): my global_id is 34884 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d8003620 con 0x7f80ec1854c0 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124114293 0 0) 0x7f80ec08d900 con 0x7f80ec085970 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient(hunting): my global_id is 134098 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80ec08df30 con 0x7f80ec085970 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 938801276 0 0) 0x7f80d8003620 con 0x7f80ec1854c0 2026-02-21T22:33:16.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.102:6790/0 conn(0x7f80ec085d40 legacy=0x7f80ec183da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.178:6789/0 conn(0x7f80ec085970 legacy=0x7f80ec08e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 monclient: found mon.noname-a 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec08d620 con 0x7f80ec1854c0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 5 monclient: authenticate success, global_id 34884 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80e8004620 con 0x7f80ec1854c0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: dump: 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f80e80048f0 con 0x7f80ec1854c0 2026-02-21T22:33:16.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 1 -- 10.20.193.102:0/2338928841 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80e8006060 con 0x7f80ec1854c0 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80e37fe640 10 monclient: dump: 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 4 set_mon_vals no callback set 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient: shutdown 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2338928841 >> v1:10.20.193.102:6789/0 conn(0x7f80ec1854c0 legacy=0x7f80ec1a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2338928841 shutdown_connections 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2338928841 >> 10.20.193.102:0/2338928841 conn(0x7f80ec054e40 msgr2=0x7f80ec092890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2338928841 shutdown_connections 2026-02-21T22:33:16.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2338928841 wait complete. 2026-02-21T22:33:16.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.207+0000 7f80f2eaf640 10 monclient: monmap: 2026-02-21T22:33:16.218 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.218 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:16.218 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 Processor -- start 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- start start 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: init 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-b con 0x7f80ec085970 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-a con 0x7f80ec085d40 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient(hunting): picked mon.noname-c con 0x7f80ec1854c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec089a80 con 0x7f80ec085d40 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec07f760 con 0x7f80ec085970 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80ec07cad0 con 0x7f80ec1854c0 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient(hunting): authenticate will time out at 2205.448371s 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e3fff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f80ec085d40 0x7f80ec07c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37782/0 (socket says 10.20.193.102:37782) 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e3fff640 1 -- 10.20.193.102:0/2637635357 learned_addr learned my addr 10.20.193.102:0/2637635357 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e3fff640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6789/0 conn(0x7f80ec085d40 legacy=0x7f80ec07c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f1425640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec1854c0 legacy=0x7f80ec184ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458342147 0 0) 0x7f80ec056dd0 con 0x7f80ec085d40 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): my global_id is 34887 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80bc003620 con 0x7f80ec085d40 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f0c24640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.178:6789/0 conn(0x7f80ec085970 legacy=0x7f80ec07bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3395416598 0 0) 0x7f80bc003620 con 0x7f80ec085d40 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80ec056dd0 con 0x7f80ec085d40 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80d40033d0 con 0x7f80ec085d40 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): dump: 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1426742191 0 0) 0x7f80ec07f760 con 0x7f80ec085970 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient(hunting): my global_id is 134101 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80bc003620 con 0x7f80ec085970 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2940046419 0 0) 0x7f80ec056dd0 con 0x7f80ec085d40 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec1854c0 legacy=0x7f80ec184ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.178:6789/0 conn(0x7f80ec085970 legacy=0x7f80ec07bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 monclient: found mon.a 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80ec07fef0 con 0x7f80ec085d40 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 5 monclient: authenticate success, global_id 34887 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _renew_subs 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f80ec07fa40 con 0x7f80ec085d40 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _renew_subs 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f80ec07ebd0 con 0x7f80ec085d40 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f80d4004b90 con 0x7f80ec085d40 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80d4004ef0 con 0x7f80ec085d40 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e2ffd640 4 set_mon_vals no callback set 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 10 monclient: dump: 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f80ec08f100 con 0x7f80ec089c90 2026-02-21T22:33:16.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.211+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f80d4005180 con 0x7f80ec085d40 2026-02-21T22:33:16.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.215+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f80d404c670 con 0x7f80ec085d40 2026-02-21T22:33:16.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.215+0000 7f80f1425640 10 monclient: get_auth_request con 0x7f80bc03d5b0 auth_method 0 2026-02-21T22:33:16.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.215+0000 7f80f0c24640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec089c90 legacy=0x7f80ec08a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 939639416 0 0) 0x7f80ec08f100 con 0x7f80ec089c90 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient(hunting): my global_id is 94367 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80ec056dd0 con 0x7f80ec089c90 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 195559546 0 0) 0x7f80ec056dd0 con 0x7f80ec089c90 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80ec08f100 con 0x7f80ec089c90 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80dc003320 con 0x7f80ec089c90 2026-02-21T22:33:16.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient: dump: 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.329 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4128848195 0 0) 0x7f80ec08f100 con 0x7f80ec089c90 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80ec07f4e0 con 0x7f80ec089c90 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f80dc006910 con 0x7f80ec089c90 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec089c90 legacy=0x7f80ec08a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.323+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec089c90 legacy=0x7f80ec08a060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.327+0000 7f80f2eaf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:33:16.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.327+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f80ec062ce0 con 0x7f80ec08a770 2026-02-21T22:33:16.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f0c24640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec08a770 legacy=0x7f80ec134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850176852 0 0) 0x7f80ec062ce0 con 0x7f80ec08a770 2026-02-21T22:33:16.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient(hunting): my global_id is 94370 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80ec08f100 con 0x7f80ec08a770 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2612374508 0 0) 0x7f80ec08f100 con 0x7f80ec08a770 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80ec062ce0 con 0x7f80ec08a770 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f80dc003b80 con 0x7f80ec08a770 2026-02-21T22:33:16.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient: dump: 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.337 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 374395633 0 0) 0x7f80ec062ce0 con 0x7f80ec08a770 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 --> v1:10.20.193.102:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f80ec08ab70 con 0x7f80ec08a770 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 <== mon.2 v1:10.20.193.102:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4514 (unknown 0 0 1924115486) 0x7f80dc006b70 con 0x7f80ec08a770 2026-02-21T22:33:16.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:33:16.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:16.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec08a770 legacy=0x7f80ec134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80e17fa640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6790/0 conn(0x7f80ec08a770 legacy=0x7f80ec134d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.339 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 64, 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 342606, 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:33:16.340 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:33:16.341 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.342 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:33:16.343 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.344 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:33:16.345 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:33:16.346 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:33:16.347 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.348 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:33:16.349 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:33:16.350 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.351 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.352 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.353 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:33:16.354 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.355 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.356 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.357 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: "num": 2 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.358 INFO:teuthology.orchestra.run.trial102.stdout: "mgr": [ 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:33:16.359 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:16.361 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.178:6800/2549949439 conn(0x7f80bc03d5b0 legacy=0x7f80bc05da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.361 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 10 monclient: shutdown 2026-02-21T22:33:16.361 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.361 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 >> v1:10.20.193.102:6789/0 conn(0x7f80ec085d40 legacy=0x7f80ec07c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.361 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f1425640 1 -- 10.20.193.102:0/2637635357 reap_dead start 2026-02-21T22:33:16.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f1425640 1 -- 10.20.193.102:0/2637635357 reap_dead start 2026-02-21T22:33:16.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 shutdown_connections 2026-02-21T22:33:16.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 >> 10.20.193.102:0/2637635357 conn(0x7f80ec054e40 msgr2=0x7f80ec087740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 shutdown_connections 2026-02-21T22:33:16.362 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.331+0000 7f80f2eaf640 1 -- 10.20.193.102:0/2637635357 wait complete. 2026-02-21T22:33:16.362 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:33:16.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2026-02-21T22:33:16.363 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 10 monclient: get_monmap_and_config 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 10 monclient: monmap: 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.423 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:16.424098+0000 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:16.424098+0000 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:16.424 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 1 Processor -- start 2026-02-21T22:33:16.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 1 -- start start 2026-02-21T22:33:16.425 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.419+0000 7f9d2ad0d640 10 monclient: init 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-a con 0x7f9d24091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d240909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d241854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d24057a20 con 0x7f9d24091240 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d24091d20 con 0x7f9d241854b0 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d24057740 con 0x7f9d240909a0 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.426 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient(hunting): authenticate will time out at 2205.660420s 2026-02-21T22:33:16.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9d240909a0 0x7f9d24183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59064/0 (socket says 10.20.193.102:59064) 2026-02-21T22:33:16.427 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1bfff640 1 -- 10.20.193.102:0/3114448734 learned_addr learned my addr 10.20.193.102:0/3114448734 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1bfff640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.102:6790/0 conn(0x7f9d240909a0 legacy=0x7f9d24183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d28a82640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.102:6789/0 conn(0x7f9d24091240 legacy=0x7f9d24091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725947038 0 0) 0x7f9d24057740 con 0x7f9d240909a0 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.433 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): my global_id is 94373 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d10003620 con 0x7f9d240909a0 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039041245 0 0) 0x7f9d24057a20 con 0x7f9d24091240 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient(hunting): my global_id is 34890 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d24057740 con 0x7f9d24091240 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d29283640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.178:6789/0 conn(0x7f9d241854b0 legacy=0x7f9d241a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2326214739 0 0) 0x7f9d10003620 con 0x7f9d240909a0 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.178:6789/0 conn(0x7f9d241854b0 legacy=0x7f9d241a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.434 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.102:6789/0 conn(0x7f9d24091240 legacy=0x7f9d24091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 monclient: found mon.noname-b 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d24090dc0 con 0x7f9d240909a0 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d0c003110 con 0x7f9d240909a0 2026-02-21T22:33:16.435 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: dump: 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.436 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 5 monclient: authenticate success, global_id 94373 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847782293 0 0) 0x7f9d24091d20 con 0x7f9d241854b0 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9d0c0033e0 con 0x7f9d240909a0 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 1 -- 10.20.193.102:0/3114448734 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d0c004b50 con 0x7f9d240909a0 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d1b7fe640 10 monclient: dump: 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.437 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 4 set_mon_vals no callback set 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: shutdown 2026-02-21T22:33:16.438 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/3114448734 >> v1:10.20.193.102:6790/0 conn(0x7f9d240909a0 legacy=0x7f9d24183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/3114448734 shutdown_connections 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/3114448734 >> 10.20.193.102:0/3114448734 conn(0x7f9d2405e660 msgr2=0x7f9d24095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/3114448734 shutdown_connections 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/3114448734 wait complete. 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.423+0000 7f9d2ad0d640 10 monclient: monmap: 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:16.439 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 Processor -- start 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- start start 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: init 2026-02-21T22:33:16.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-a con 0x7f9d240909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-c con 0x7f9d24091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient(hunting): picked mon.noname-b con 0x7f9d241854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2412bd10 con 0x7f9d240909a0 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d24112f60 con 0x7f9d241854b0 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d241120f0 con 0x7f9d24091240 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient(hunting): authenticate will time out at 2205.664403s 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d1bfff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9d24091240 0x7f9d241729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59080/0 (socket says 10.20.193.102:59080) 2026-02-21T22:33:16.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d1bfff640 1 -- 10.20.193.102:0/52179955 learned_addr learned my addr 10.20.193.102:0/52179955 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d1bfff640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.102:6790/0 conn(0x7f9d24091240 legacy=0x7f9d241729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d28a82640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.102:6789/0 conn(0x7f9d240909a0 legacy=0x7f9d2412b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049253262 0 0) 0x7f9d241120f0 con 0x7f9d24091240 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): my global_id is 94376 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cf4003620 con 0x7f9d24091240 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899804913 0 0) 0x7f9d2412bd10 con 0x7f9d240909a0 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): my global_id is 34893 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d241120f0 con 0x7f9d240909a0 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d29283640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d241854b0 legacy=0x7f9d241740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3850052368 0 0) 0x7f9cf4003620 con 0x7f9d24091240 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d2412bd10 con 0x7f9d24091240 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d0c0048e0 con 0x7f9d24091240 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): dump: 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.443 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1697784572 0 0) 0x7f9d241120f0 con 0x7f9d240909a0 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cf4003620 con 0x7f9d240909a0 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d14003710 con 0x7f9d240909a0 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.444 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): dump: 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252005674 0 0) 0x7f9d24112f60 con 0x7f9d241854b0 2026-02-21T22:33:16.445 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient(hunting): my global_id is 134107 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d241120f0 con 0x7f9d241854b0 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1794951957 0 0) 0x7f9d2412bd10 con 0x7f9d24091240 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d241854b0 legacy=0x7f9d241740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.102:6789/0 conn(0x7f9d240909a0 legacy=0x7f9d2412b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 monclient: found mon.c 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d24110c00 con 0x7f9d24091240 2026-02-21T22:33:16.446 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 5 monclient: authenticate success, global_id 94376 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _renew_subs 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9d241126c0 con 0x7f9d24091240 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _renew_subs 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9d24111930 con 0x7f9d24091240 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9d240008e0 con 0x7f9d2412e580 2026-02-21T22:33:16.447 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9d0c0033c0 con 0x7f9d24091240 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d0c0052e0 con 0x7f9d24091240 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d1affd640 4 set_mon_vals no callback set 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 10 monclient: dump: 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.448 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.427+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9d0c011970 con 0x7f9d24091240 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f9d0c04c600 con 0x7f9d24091240 2026-02-21T22:33:16.449 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d29283640 10 monclient: get_auth_request con 0x7f9cf403d910 auth_method 0 2026-02-21T22:33:16.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d28a82640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2412e580 legacy=0x7f9d2408d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909290974 0 0) 0x7f9d240008e0 con 0x7f9d2412e580 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient(hunting): my global_id is 134110 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2412bd10 con 0x7f9d2412e580 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2394251562 0 0) 0x7f9d2412bd10 con 0x7f9d2412e580 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d240008e0 con 0x7f9d2412e580 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d14003c90 con 0x7f9d2412e580 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient: dump: 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.453 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1254234330 0 0) 0x7f9d240008e0 con 0x7f9d2412e580 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d24110680 con 0x7f9d2412e580 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+10764 (unknown 0 0 870318384) 0x7f9d140058c0 con 0x7f9d2412e580 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:16.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2412e580 legacy=0x7f9d2408d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.431+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2412e580 legacy=0x7f9d2408d9f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.435+0000 7f9d2ad0d640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.435+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9d2412e950 con 0x7f9d2404d280 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d28a82640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2404d280 legacy=0x7f9d24176810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 81239390 0 0) 0x7f9d2412e950 con 0x7f9d2404d280 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient(hunting): my global_id is 134113 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d240008e0 con 0x7f9d2404d280 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4209719369 0 0) 0x7f9d240008e0 con 0x7f9d2404d280 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9d2412e950 con 0x7f9d2404d280 2026-02-21T22:33:16.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9d140041d0 con 0x7f9d2404d280 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient: dump: 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.456 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2067537109 0 0) 0x7f9d2412e950 con 0x7f9d2404d280 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 --> v1:10.20.193.178:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d24095010 con 0x7f9d2404d280 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 <== mon.1 v1:10.20.193.178:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+4344 (unknown 0 0 1446996242) 0x7f9d14004590 con 0x7f9d2404d280 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2404d280 legacy=0x7f9d24176810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d197fa640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6789/0 conn(0x7f9d2404d280 legacy=0x7f9d24176810 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout: "state": "peon", 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout: "election_epoch": 64, 2026-02-21T22:33:16.458 INFO:teuthology.orchestra.run.trial102.stdout: "uptime": 567, 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: "quorum": [ 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: 0, 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: 1, 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: 2 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_age": 0, 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.459 INFO:teuthology.orchestra.run.trial102.stdout: "required_con": "2453899407728394244", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "required_mon": [ 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.460 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.461 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_con": "4544132024016699391", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "quorum_mon": [ 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.462 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "outside_quorum": [], 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "extra_probe_peers": [], 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "sync_provider": [], 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "monmap": { 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "epoch": 1, 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "fsid": "740d24fb-83fc-48ed-963d-a0ec1185b484", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "modified": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "created": "2026-02-21T22:15:04.891122Z", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release": 20, 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "min_mon_release_name": "tentacle", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "election_strategy": 3, 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "disallowed_leaders": "", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false, 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "tiebreaker_mon": "", 2026-02-21T22:33:16.463 INFO:teuthology.orchestra.run.trial102.stdout: "removed_ranks": "", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "features": { 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "persistent": [ 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "kraken", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "luminous", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "mimic", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "osdmap-prune", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "nautilus", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "octopus", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "pacific", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "elector-pinging", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "quincy", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "reef", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "squid", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "tentacle", 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "nvmeof_beacon_diff" 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "optional": [] 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: "mons": [ 2026-02-21T22:33:16.464 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 0, 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "name": "a", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6789/0", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891168+0000", 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.465 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 1, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "name": "b", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.178:6789/0", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891172+0000", 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "rank": 2, 2026-02-21T22:33:16.466 INFO:teuthology.orchestra.run.trial102.stdout: "name": "c", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "public_addrs": { 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "addrvec": [ 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "type": "v1", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "nonce": 0 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "public_addr": "10.20.193.102:6790/0", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "priority": 0, 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "weight": 0, 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "time_added": "2026-02-21T22:15:04:891173+0000", 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "crush_location": [] 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "feature_map": { 2026-02-21T22:33:16.467 INFO:teuthology.orchestra.run.trial102.stdout: "mon": [ 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: ], 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "client": [ 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: { 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "features": "0x3f0ffffffffdffff", 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "release": "squid", 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "num": 1 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: } 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: ] 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: }, 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout: "stretch_mode": false 2026-02-21T22:33:16.468 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:16.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9cf403d910 legacy=0x7f9cf405ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 10 monclient: shutdown 2026-02-21T22:33:16.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.469 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 >> v1:10.20.193.102:6790/0 conn(0x7f9d24091240 legacy=0x7f9d241729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d29283640 1 -- 10.20.193.102:0/52179955 reap_dead start 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d29283640 1 -- 10.20.193.102:0/52179955 reap_dead start 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 shutdown_connections 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 >> 10.20.193.102:0/52179955 conn(0x7f9d2405e660 msgr2=0x7f9d24089c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 shutdown_connections 2026-02-21T22:33:16.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.439+0000 7f9d2ad0d640 1 -- 10.20.193.102:0/52179955 wait complete. 2026-02-21T22:33:16.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2026-02-21T22:33:16.470 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2026-02-21T22:33:16.470 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2026-02-21T22:33:16.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 10 monclient: get_monmap_and_config 2026-02-21T22:33:16.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 10 monclient: monmap: 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:16.532124+0000 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:16.532124+0000 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:16.532 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 1 Processor -- start 2026-02-21T22:33:16.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 1 -- start start 2026-02-21T22:33:16.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.527+0000 7f5f08b22640 10 monclient: init 2026-02-21T22:33:16.533 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-a con 0x7f5f04091240 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-b con 0x7f5f040909a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-c con 0x7f5f041854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f04057a20 con 0x7f5f04091240 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f04091d20 con 0x7f5f041854b0 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f04057740 con 0x7f5f040909a0 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient(hunting): authenticate will time out at 2205.768529s 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5f040909a0 0x7f5f04183d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59082/0 (socket says 10.20.193.102:59082) 2026-02-21T22:33:16.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01d74640 1 -- 10.20.193.102:0/1961342319 learned_addr learned my addr 10.20.193.102:0/1961342319 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01d74640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.102:6790/0 conn(0x7f5f040909a0 legacy=0x7f5f04183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f02575640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.102:6789/0 conn(0x7f5f04091240 legacy=0x7f5f04091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953949161 0 0) 0x7f5f04057740 con 0x7f5f040909a0 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): my global_id is 94379 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef0003620 con 0x7f5f040909a0 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2731751806 0 0) 0x7f5f04057a20 con 0x7f5f04091240 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f02d76640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.178:6789/0 conn(0x7f5f041854b0 legacy=0x7f5f041a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.536 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient(hunting): my global_id is 34896 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f04057740 con 0x7f5f04091240 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 289392603 0 0) 0x7f5ef0003620 con 0x7f5f040909a0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.178:6789/0 conn(0x7f5f041854b0 legacy=0x7f5f041a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.102:6789/0 conn(0x7f5f04091240 legacy=0x7f5f04091610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 monclient: found mon.noname-b 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04090dc0 con 0x7f5f040909a0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 5 monclient: authenticate success, global_id 94379 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5eec003110 con 0x7f5f040909a0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: dump: 2026-02-21T22:33:16.537 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5eec0033e0 con 0x7f5f040909a0 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 1 -- 10.20.193.102:0/1961342319 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5eec004b50 con 0x7f5f040909a0 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f01573640 10 monclient: dump: 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.538 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 4 set_mon_vals no callback set 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 10 monclient: shutdown 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- 10.20.193.102:0/1961342319 >> v1:10.20.193.102:6790/0 conn(0x7f5f040909a0 legacy=0x7f5f04183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- 10.20.193.102:0/1961342319 shutdown_connections 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- 10.20.193.102:0/1961342319 >> 10.20.193.102:0/1961342319 conn(0x7f5f0405e660 msgr2=0x7f5f04095fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- 10.20.193.102:0/1961342319 shutdown_connections 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.531+0000 7f5f08b22640 1 -- 10.20.193.102:0/1961342319 wait complete. 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: build_initial_monmap 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: monmap: 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:16.539 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 Processor -- start 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- start start 2026-02-21T22:33:16.540 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: init 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-a con 0x7f5f040909a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-c con 0x7f5f04091240 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient(hunting): picked mon.noname-b con 0x7f5f041854b0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f0412bd80 con 0x7f5f040909a0 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f04112fd0 con 0x7f5f041854b0 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f04112160 con 0x7f5f04091240 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient(hunting): authenticate will time out at 2205.772594s 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f02575640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5f040909a0 0x7f5f0412b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37820/0 (socket says 10.20.193.102:37820) 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f01d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f5f04091240 0x7f5f04172a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59094/0 (socket says 10.20.193.102:59094) 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f02575640 1 -- 10.20.193.102:0/128590145 learned_addr learned my addr 10.20.193.102:0/128590145 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f01d74640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.102:6790/0 conn(0x7f5f04091240 legacy=0x7f5f04172a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f02575640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.102:6789/0 conn(0x7f5f040909a0 legacy=0x7f5f0412b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701424573 0 0) 0x7f5f04112160 con 0x7f5f04091240 2026-02-21T22:33:16.543 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): my global_id is 94382 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ed0003620 con 0x7f5f04091240 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1231831176 0 0) 0x7f5f0412bd80 con 0x7f5f040909a0 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): my global_id is 34899 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f04112160 con 0x7f5f040909a0 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f02d76640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.178:6789/0 conn(0x7f5f041854b0 legacy=0x7f5f04174150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1735260714 0 0) 0x7f5ed0003620 con 0x7f5f04091240 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5f0412bd80 con 0x7f5f04091240 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5eec0048e0 con 0x7f5f04091240 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): dump: 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.544 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3375322321 0 0) 0x7f5f04112160 con 0x7f5f040909a0 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5ed0003620 con 0x7f5f040909a0 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5ef8003710 con 0x7f5f040909a0 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): dump: 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.545 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082444442 0 0) 0x7f5f04112fd0 con 0x7f5f041854b0 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient(hunting): my global_id is 134119 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f04112160 con 0x7f5f041854b0 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1077321762 0 0) 0x7f5f0412bd80 con 0x7f5f04091240 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.178:6789/0 conn(0x7f5f041854b0 legacy=0x7f5f04174150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.102:6789/0 conn(0x7f5f040909a0 legacy=0x7f5f0412b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 monclient: found mon.c 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04110c70 con 0x7f5f04091240 2026-02-21T22:33:16.546 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 5 monclient: authenticate success, global_id 94382 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _renew_subs 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5f04112730 con 0x7f5f04091240 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _renew_subs 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5f041119a0 con 0x7f5f04091240 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5eec0033e0 con 0x7f5f04091240 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5eec005130 con 0x7f5f04091240 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f00d72640 4 set_mon_vals no callback set 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5eeaffd640 10 monclient: dump: 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:16.547 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.535+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f04110670 con 0x7f5f04091240 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5eec0117c0 con 0x7f5f04091240 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f5eec04c520 con 0x7f5f04091240 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5f02575640 10 monclient: get_auth_request con 0x7f5ed003d830 auth_method 0 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5eec017680 con 0x7f5f04091240 2026-02-21T22:33:16.548 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5eeaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:16.549 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.539+0000 7f5eeaffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:16.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.655+0000 7f5f08b22640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2026-02-21T22:33:16.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.655+0000 7f5f08b22640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2026-02-21T22:33:16.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.655+0000 7f5f08b22640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:16.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.655+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f5f0405eb70 con 0x7f5f04091240 2026-02-21T22:33:16.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5eeaffd640 1 -- 10.20.193.102:0/128590145 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f5eec016f60 con 0x7f5f04091240 2026-02-21T22:33:16.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5eeaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2026-02-21T22:33:16.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5eeaffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:16.662 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.662 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=13,kv=12,logm=51} crc {auth=1373248753,config=1618278320,health=2861225789,kv=514132712,logm=1072047962}) 2026-02-21T22:33:16.662 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=23,config=2,health=13,kv=12,logm=50} crc {auth=510256251,config=1618278320,health=536807881,kv=514132712,logm=4210462539}) 2026-02-21T22:33:16.662 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.662 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=13,kv=12,logm=51} crc {auth=1373248753,config=1618278320,health=2861225789,kv=514132712,logm=1072047962}) 2026-02-21T22:33:16.663 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=22,config=2,health=13,kv=11,logm=52} crc {auth=2054186944,config=1618278320,health=2861225789,kv=3483615117,logm=2579271704}) 2026-02-21T22:33:16.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5ed003d830 legacy=0x7f5ed005dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 10 monclient: shutdown 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 >> v1:10.20.193.102:6790/0 conn(0x7f5f04091240 legacy=0x7f5f04172a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 shutdown_connections 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 >> 10.20.193.102:0/128590145 conn(0x7f5f0405e660 msgr2=0x7f5f0408b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 shutdown_connections 2026-02-21T22:33:16.664 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:16.659+0000 7f5f08b22640 1 -- 10.20.193.102:0/128590145 wait complete. 2026-02-21T22:33:16.667 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.667 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245354800}) 2026-02-21T22:33:16.667 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1322294958}) 2026-02-21T22:33:16.667 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1245354800}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=821554414}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=804445662}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=917473445}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=804445662}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.659+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1709872229}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=869137528}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=798172379}) 2026-02-21T22:33:16.668 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=869137528}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4138313929}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3868514326}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2711801764}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3868514326}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3694799539}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356843503}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1974799839}) 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.669 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3356843503}) 2026-02-21T22:33:16.670 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4206910990}) 2026-02-21T22:33:16.672 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.672 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1953149623}) 2026-02-21T22:33:16.672 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2739164418}) 2026-02-21T22:33:16.672 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1953149623}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.663+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1574948341}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110721704}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=784577417}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=110721704}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2311070275}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=117229453}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1122526445}) 2026-02-21T22:33:16.673 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=117229453}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2177095189}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=795874092}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=283506578}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=795874092}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.667+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2412221534}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3636258140}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3497535774}) 2026-02-21T22:33:16.674 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.675 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3636258140}) 2026-02-21T22:33:16.675 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2064428447}) 2026-02-21T22:33:16.676 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912348819}) 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3477246253}) 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2912348819}) 2026-02-21T22:33:16.679 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=782814837}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360754056}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2590801594}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1360754056}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1341150829}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1656026258}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3585333205}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1656026258}) 2026-02-21T22:33:16.680 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.671+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=749992684}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=411601400}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3137104469}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=411601400}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1464578308}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163154240}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=557587480}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4163154240}) 2026-02-21T22:33:16.681 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3025322094}) 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2984994067}) 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=139030716}) 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2984994067}) 2026-02-21T22:33:16.682 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3380349311}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.675+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326663001}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2224864398}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2326663001}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3945356575}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=303946556}) 2026-02-21T22:33:16.686 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1627331693}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=303946556}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3440267675}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594004881}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=647165938}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2594004881}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3394671037}) 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.687 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1089889015}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2715731126}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1089889015}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.679+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=111591071}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1480083034}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=134044830}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1480083034}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1950535293}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006915365}) 2026-02-21T22:33:16.688 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3972118859}) 2026-02-21T22:33:16.689 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.689 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4006915365}) 2026-02-21T22:33:16.689 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2590631188}) 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531600615}) 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=830833310}) 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1531600615}) 2026-02-21T22:33:16.690 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3825236475}) 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3507727656}) 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3475968428}) 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3507727656}) 2026-02-21T22:33:16.691 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.683+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1831516680}) 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1650062832}) 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1006415884}) 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1650062832}) 2026-02-21T22:33:16.888 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.883+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1543747787}) 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2345208423}) 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4023362947}) 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2345208423}) 2026-02-21T22:33:16.899 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=183537941}) 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1336499001}) 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3257882364}) 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1336499001}) 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.895+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=88565619}) 2026-02-21T22:33:16.901 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.902 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=265761941}) 2026-02-21T22:33:16.902 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1513097582}) 2026-02-21T22:33:16.902 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.902 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=265761941}) 2026-02-21T22:33:16.902 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2948890377}) 2026-02-21T22:33:16.904 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.904 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3524584492}) 2026-02-21T22:33:16.904 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1269560240}) 2026-02-21T22:33:16.904 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3524584492}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3330969960}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2699000540}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4143231430}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2699000540}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1404757346}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1086438503}) 2026-02-21T22:33:16.905 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1968103392}) 2026-02-21T22:33:16.906 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.899+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.906 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.903+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1086438503}) 2026-02-21T22:33:16.906 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.903+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2695049446}) 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743690970}) 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2668718272}) 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3743690970}) 2026-02-21T22:33:16.913 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3639004547}) 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=401246183}) 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2644545759}) 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=401246183}) 2026-02-21T22:33:16.914 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2688859754}) 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=191077057}) 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4161275864}) 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=191077057}) 2026-02-21T22:33:16.915 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.911+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1898220748}) 2026-02-21T22:33:16.916 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.916 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3550767988}) 2026-02-21T22:33:16.916 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1853679448}) 2026-02-21T22:33:16.916 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.916 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3550767988}) 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=118743414}) 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057307556}) 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=792118340}) 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.917 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1057307556}) 2026-02-21T22:33:16.918 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=834165641}) 2026-02-21T22:33:16.918 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.918 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=325241468}) 2026-02-21T22:33:16.918 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1760715333}) 2026-02-21T22:33:16.919 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.919 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=325241468}) 2026-02-21T22:33:16.919 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.915+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3689039536}) 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=2} crc {logm=3598783286,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=7383416,mon_config_key=4017729600}) 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=2} crc {logm=2308004964,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=290791067,mon_config_key=4017729600}) 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=2} crc {logm=3598783286,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgr_metadata=2589708306,mgrstat=7383416,mon_config_key=4017729600}) 2026-02-21T22:33:16.920 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=2} crc {logm=3030673372,mdsmap=3074481156,mgr=369547122,mgr_command_descs=2141687499,mgrstat=1359359360,mon_config_key=4017729600}) 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=80} crc {mon_config_key=2696305677,monmap=918322475,nvmeofgw=935146538,osd_metadata=3232105792,osd_pg_creating=4013621700,osd_snap=691256185}) 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=6,monmap=3,nvmeofgw=3,osd_metadata=6,osd_snap=82} crc {mon_config_key=1746546803,monmap=918322475,nvmeofgw=3360707892,osd_metadata=3232105792,osd_snap=3502324345}) 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=8,monmap=3,nvmeofgw=2,osd_metadata=6,osd_pg_creating=1,osd_snap=80} crc {mon_config_key=2696305677,monmap=918322475,nvmeofgw=935146538,osd_metadata=3232105792,osd_pg_creating=4013621700,osd_snap=691256185}) 2026-02-21T22:33:16.921 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=6,monmap=2,nvmeofgw=3,osd_metadata=5,osd_pg_creating=1,osd_snap=83} crc {mon_config_key=1814466103,monmap=3140514009,nvmeofgw=3360707892,osd_metadata=295559448,osd_pg_creating=4013621700,osd_snap=3643117531}) 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=81,osdmap=19} crc {osd_snap=725418929,osdmap=1496506779}) 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=82,osdmap=18} crc {osd_snap=406180812,osdmap=1602543870}) 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=81,osdmap=19} crc {osd_snap=725418929,osdmap=1496506779}) 2026-02-21T22:33:16.922 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=82,osdmap=18} crc {osd_snap=2873999078,osdmap=13278626}) 2026-02-21T22:33:16.923 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.923 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3741397070}) 2026-02-21T22:33:16.923 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1685909824}) 2026-02-21T22:33:16.923 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.924 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3741397070}) 2026-02-21T22:33:16.924 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.919+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3883893612}) 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2009512165}) 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2709970340}) 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2009512165}) 2026-02-21T22:33:16.925 INFO:tasks.ceph.mon.a.trial102.stderr:2026-02-21T22:33:16.923+0000 7fc9c381e640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2540225596}) 2026-02-21T22:33:17.677 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2026-02-21T22:33:17.677 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2026-02-21T22:33:17.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: get_monmap_and_config 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: build_initial_monmap 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: monmap: 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:17.747008+0000 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:17.747008+0000 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.748 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 1 Processor -- start 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 1 -- start start 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: init 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-a con 0x7f67e808e6c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:17.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-b con 0x7f67e8059100 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-c con 0x7f67e808f1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e80904a0 con 0x7f67e808e6c0 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e8090260 con 0x7f67e808f1a0 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e8058820 con 0x7f67e8059100 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ef2d0640 10 monclient(hunting): authenticate will time out at 2206.982973s 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ec844640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 0x7f67e8184140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59102/0 (socket says 10.20.193.102:59102) 2026-02-21T22:33:17.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.743+0000 7f67ec844640 1 -- 10.20.193.102:0/890780184 learned_addr learned my addr 10.20.193.102:0/890780184 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:17.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ec844640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 legacy=0x7f67e8184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ed045640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.102:6789/0 conn(0x7f67e808e6c0 legacy=0x7f67e808ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891738336 0 0) 0x7f67e8058820 con 0x7f67e8059100 2026-02-21T22:33:17.758 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient(hunting): my global_id is 94385 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67d0003620 con 0x7f67e8059100 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ed846640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.178:6789/0 conn(0x7f67e808f1a0 legacy=0x7f67e81a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 903899412 0 0) 0x7f67d0003620 con 0x7f67e8059100 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.178:6789/0 conn(0x7f67e808f1a0 legacy=0x7f67e81a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.102:6789/0 conn(0x7f67e808e6c0 legacy=0x7f67e808ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 monclient: found mon.noname-b 2026-02-21T22:33:17.759 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e8057f80 con 0x7f67e8059100 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: _finish_auth 0 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67dc003190 con 0x7f67e8059100 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 5 monclient: authenticate success, global_id 94385 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:17.760 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: dump: 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:17.761 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67dc0033a0 con 0x7f67e8059100 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 1 -- 10.20.193.102:0/890780184 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67dc004b50 con 0x7f67e8059100 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67d7fff640 10 monclient: dump: 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.762 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 4 set_mon_vals no callback set 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: shutdown 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- 10.20.193.102:0/890780184 >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 legacy=0x7f67e8184140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- 10.20.193.102:0/890780184 shutdown_connections 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- 10.20.193.102:0/890780184 >> 10.20.193.102:0/890780184 conn(0x7f67e8056b30 msgr2=0x7f67e8093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- 10.20.193.102:0/890780184 shutdown_connections 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- 10.20.193.102:0/890780184 wait complete. 2026-02-21T22:33:17.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: build_initial_monmap 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: monmap: 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 Processor -- start 2026-02-21T22:33:17.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- start start 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: init 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-c con 0x7f67e8059100 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-a con 0x7f67e808e6c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient(hunting): picked mon.noname-b con 0x7f67e808f1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e808b1f0 con 0x7f67e808e6c0 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e810a560 con 0x7f67e808f1a0 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67e808c090 con 0x7f67e8059100 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ef2d0640 10 monclient(hunting): authenticate will time out at 2206.986674s 2026-02-21T22:33:17.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ed045640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 0x7f67e808a3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59114/0 (socket says 10.20.193.102:59114) 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.747+0000 7f67ed045640 1 -- 10.20.193.102:0/816511428 learned_addr learned my addr 10.20.193.102:0/816511428 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ed045640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 legacy=0x7f67e808a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ec844640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.102:6789/0 conn(0x7f67e808e6c0 legacy=0x7f67e808aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798103957 0 0) 0x7f67e808c090 con 0x7f67e8059100 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): my global_id is 94388 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67b8003620 con 0x7f67e8059100 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3488048829 0 0) 0x7f67e808b1f0 con 0x7f67e808e6c0 2026-02-21T22:33:17.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): my global_id is 34905 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67e808c090 con 0x7f67e808e6c0 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ed846640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.178:6789/0 conn(0x7f67e808f1a0 legacy=0x7f67e81a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2395164241 0 0) 0x7f67b8003620 con 0x7f67e8059100 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f67e808b1f0 con 0x7f67e8059100 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67d8003400 con 0x7f67e8059100 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:17.767 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2742355380 0 0) 0x7f67e808c090 con 0x7f67e808e6c0 2026-02-21T22:33:17.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f67b8003620 con 0x7f67e808e6c0 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67dc0048e0 con 0x7f67e808e6c0 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.769 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4190366990 0 0) 0x7f67e808b1f0 con 0x7f67e8059100 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.178:6789/0 conn(0x7f67e808f1a0 legacy=0x7f67e81a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.102:6789/0 conn(0x7f67e808e6c0 legacy=0x7f67e808aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 monclient: found mon.c 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e808be30 con 0x7f67e8059100 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:17.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 5 monclient: authenticate success, global_id 94388 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _renew_subs 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f67e811ee40 con 0x7f67e8059100 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _renew_subs 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f67e81a6880 con 0x7f67e8059100 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67d80039c0 con 0x7f67e8059100 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67d8003d20 con 0x7f67e8059100 2026-02-21T22:33:17.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: dump: 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.772 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d77fe640 4 set_mon_vals no callback set 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f67d8003ff0 con 0x7f67e8059100 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e80008e0 con 0x7f67e8059100 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f67d804c610 con 0x7f67e8059100 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67ec844640 10 monclient: get_auth_request con 0x7f67b803d6c0 auth_method 0 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f67d80176b0 con 0x7f67e8059100 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.751+0000 7f67d5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f67e80949d0 con 0x7f67e8059100 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67d5ffb640 1 -- 10.20.193.102:0/816511428 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1610 (unknown 254746657 0 3564098757) 0x7f67d801c9d0 con 0x7f67e8059100 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67d5ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2026-02-21T22:33:17.870 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67d5ffb640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:17.871 INFO:teuthology.orchestra.run.trial102.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","modified":"2026-02-21T22:15:04.891122Z","created":"2026-02-21T22:15:04.891122Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":3,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6789","nonce":0}]},"addr":"10.20.193.102:6789/0","public_addr":"10.20.193.102:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891168+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6789","nonce":0}]},"addr":"10.20.193.178:6789/0","public_addr":"10.20.193.178:6789/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891172+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6790","nonce":0}]},"addr":"10.20.193.102:6790/0","public_addr":"10.20.193.102:6790/0","priority":0,"weight":0,"time_added":"2026-02-21T22:15:04:891173+0000","crush_location":[]}]}} 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.178:6800/2549949439 conn(0x7f67b803d6c0 legacy=0x7f67b805db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 10 monclient: shutdown 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 >> v1:10.20.193.102:6790/0 conn(0x7f67e8059100 legacy=0x7f67e808a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 shutdown_connections 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 >> 10.20.193.102:0/816511428 conn(0x7f67e8056b30 msgr2=0x7f67e813e640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 shutdown_connections 2026-02-21T22:33:17.872 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.867+0000 7f67ef2d0640 1 -- 10.20.193.102:0/816511428 wait complete. 2026-02-21T22:33:17.880 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2026-02-21T22:33:17.880 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T22:33:17.898 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T22:33:17.899 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:33:17.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: get_monmap_and_config 2026-02-21T22:33:17.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: build_initial_monmap 2026-02-21T22:33:17.962 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: monmap: 2026-02-21T22:33:17.962 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:17.963015+0000 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:17.963015+0000 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:17.963 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 1 Processor -- start 2026-02-21T22:33:17.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 1 -- start start 2026-02-21T22:33:17.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: init 2026-02-21T22:33:17.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:17.968 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-b con 0x7f7490142a40 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-a con 0x7f74900900c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-c con 0x7f7490143520 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490057bf0 con 0x7f74900900c0 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74900563d0 con 0x7f7490143520 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490057350 con 0x7f7490142a40 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f7497c2f640 10 monclient(hunting): authenticate will time out at 2207.199182s 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f74951a3640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f74900900c0 0x7f7490184010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37842/0 (socket says 10.20.193.102:37842) 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f74951a3640 1 -- 10.20.193.102:0/2205630448 learned_addr learned my addr 10.20.193.102:0/2205630448 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:17.969 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.959+0000 7f74959a4640 1 --1- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6790/0 conn(0x7f7490142a40 0x7f7490142e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59116/0 (socket says 10.20.193.102:59116) 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74959a4640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6790/0 conn(0x7f7490142a40 legacy=0x7f7490142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74951a3640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6789/0 conn(0x7f74900900c0 legacy=0x7f7490184010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74961a5640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.178:6789/0 conn(0x7f7490143520 legacy=0x7f74901a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110915071 0 0) 0x7f7490057350 con 0x7f7490142a40 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): my global_id is 94391 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7478003620 con 0x7f7490142a40 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619149738 0 0) 0x7f7490057bf0 con 0x7f74900900c0 2026-02-21T22:33:17.970 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): my global_id is 34908 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7490057350 con 0x7f74900900c0 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925086967 0 0) 0x7f74900563d0 con 0x7f7490143520 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient(hunting): my global_id is 134125 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7490057bf0 con 0x7f7490143520 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2947849448 0 0) 0x7f7478003620 con 0x7f7490142a40 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.178:6789/0 conn(0x7f7490143520 legacy=0x7f74901a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.971 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6789/0 conn(0x7f74900900c0 legacy=0x7f7490184010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: _finish_hunting 0 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 monclient: found mon.noname-b 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7490057630 con 0x7f7490142a40 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: _finish_auth 0 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: _check_auth_tickets 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7480003110 con 0x7f7490142a40 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 5 monclient: authenticate success, global_id 94391 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:17.972 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: dump: 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.973 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74959a4640 1 --1- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6790/0 conn(0x7f7490142a40 0x7f7490142e10 :-1 s=OPENED pgs=571 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f74800048e0 mon_map magic: 0 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7480003320 con 0x7f7490142a40 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 4 set_mon_vals no callback set 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: shutdown 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- 10.20.193.102:0/2205630448 >> v1:10.20.193.102:6790/0 conn(0x7f7490142a40 legacy=0x7f7490142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 1 -- 10.20.193.102:0/2205630448 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74800048e0 con 0x7f7490142a40 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74949a2640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:33:17.974 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- 10.20.193.102:0/2205630448 shutdown_connections 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- 10.20.193.102:0/2205630448 >> 10.20.193.102:0/2205630448 conn(0x7f749005e4c0 msgr2=0x7f7490095e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- 10.20.193.102:0/2205630448 shutdown_connections 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- 10.20.193.102:0/2205630448 wait complete. 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: build_initial_monmap 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: monmap: 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:17.975 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 Processor -- start 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- start start 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: init 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-b con 0x7f74900900c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-a con 0x7f7490142a40 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient(hunting): picked mon.noname-c con 0x7f7490143520 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:17.976 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490130000 con 0x7f7490142a40 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490131ac0 con 0x7f74900900c0 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7490132d40 con 0x7f7490143520 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f7497c2f640 10 monclient(hunting): authenticate will time out at 2207.203848s 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74961a5640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f7490143520 0x7f7490195d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59120/0 (socket says 10.20.193.102:59120) 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74961a5640 1 -- 10.20.193.102:0/4230834097 learned_addr learned my addr 10.20.193.102:0/4230834097 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:17.977 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.963+0000 7f74951a3640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f7490142a40 0x7f7490194650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37844/0 (socket says 10.20.193.102:37844) 2026-02-21T22:33:17.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f74961a5640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.102:6790/0 conn(0x7f7490143520 legacy=0x7f7490195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f74951a3640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.102:6789/0 conn(0x7f7490142a40 legacy=0x7f7490194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f74961a5640 1 --1- 10.20.193.102:0/4230834097 >> v1:10.20.193.102:6790/0 conn(0x7f7490143520 0x7f7490195d70 :-1 s=OPENED pgs=572 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f7490132d40 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:17.981 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 790354699 0 0) 0x7f7490130000 con 0x7f7490142a40 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): my global_id is 34911 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f74959a4640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.178:6789/0 conn(0x7f74900900c0 legacy=0x7f7490153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7460003620 con 0x7f7490142a40 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 398529691 0 0) 0x7f7460003620 con 0x7f7490142a40 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7490130000 con 0x7f7490142a40 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f74840033d0 con 0x7f7490142a40 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:17.982 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): dump: 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.983 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621431979 0 0) 0x7f7490131ac0 con 0x7f74900900c0 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient(hunting): my global_id is 134128 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7460003620 con 0x7f74900900c0 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2215151685 0 0) 0x7f7490130000 con 0x7f7490142a40 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.102:6790/0 conn(0x7f7490143520 legacy=0x7f7490195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.984 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.178:6789/0 conn(0x7f74900900c0 legacy=0x7f7490153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 monclient: found mon.a 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7490130b60 con 0x7f7490142a40 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1765899566 0 0) 0x7f7490132d40 con 0x7f7490143520 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:17.985 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 5 monclient: authenticate success, global_id 34911 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _renew_subs 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f749012f550 con 0x7f7490142a40 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _renew_subs 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7490153c00 con 0x7f7490142a40 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7484004b90 con 0x7f7490142a40 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f7484004ef0 con 0x7f7490142a40 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:17.986 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 10 monclient: dump: 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747ffff640 4 set_mon_vals no callback set 2026-02-21T22:33:17.987 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7490059170 con 0x7f7490142a40 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f74840051c0 con 0x7f7490142a40 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f748404c5a0 con 0x7f7490142a40 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.967+0000 7f74959a4640 10 monclient: get_auth_request con 0x7f746003d500 auth_method 0 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.971+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f7484017700 con 0x7f7490142a40 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.971+0000 7f747e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:17.988 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:17.971+0000 7f747e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:18.058 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.055+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7490132f10 con 0x7f746003d500 2026-02-21T22:33:18.059 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.055+0000 7f747e7fc640 1 -- 10.20.193.102:0/4230834097 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34069 (unknown 2965378022 0 3191784877) 0x7f7490132f10 con 0x7f746003d500 2026-02-21T22:33:18.060 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.178:6800/2549949439 conn(0x7f746003d500 legacy=0x7f746005d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 10 monclient: shutdown 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 >> v1:10.20.193.102:6789/0 conn(0x7f7490142a40 legacy=0x7f7490194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.061 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 shutdown_connections 2026-02-21T22:33:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 >> 10.20.193.102:0/4230834097 conn(0x7f749005e4c0 msgr2=0x7f749008a140 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 shutdown_connections 2026-02-21T22:33:18.062 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.059+0000 7f7497c2f640 1 -- 10.20.193.102:0/4230834097 wait complete. 2026-02-21T22:33:18.070 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1368,"stamp":"2026-02-21T22:33:17.617173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":212704,"kb_used_data":14848,"kb_used_omap":8260,"kb_used_meta":189499,"kb_avail":566018336,"statfs":{"total":579820584960,"available":579602776064,"internally_reserved":0,"allocated":15204352,"data_stored":9432172,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":8458769,"internal_metadata":194047471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001018"},"pg_stats":[{"pgid":"1.0","version":"14'112","reported_seq":1759,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169478+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:32:46.169478+0000","last_peered":"2026-02-21T22:32:46.169478+0000","last_clean":"2026-02-21T22:32:46.169478+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:32:46.169478+0000","last_undegraded":"2026-02-21T22:32:46.169478+0000","last_fullsized":"2026-02-21T22:32:46.169478+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1637,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169991+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:32:46.169991+0000","last_peered":"2026-02-21T22:32:46.169991+0000","last_clean":"2026-02-21T22:32:46.169991+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:32:46.169991+0000","last_undegraded":"2026-02-21T22:32:46.169991+0000","last_fullsized":"2026-02-21T22:32:46.169991+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169377+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:32:46.169377+0000","last_peered":"2026-02-21T22:32:46.169377+0000","last_clean":"2026-02-21T22:32:46.169377+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:32:46.169377+0000","last_undegraded":"2026-02-21T22:32:46.169377+0000","last_fullsized":"2026-02-21T22:32:46.169377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1624,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.667699+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:32:45.667699+0000","last_peered":"2026-02-21T22:32:45.667699+0000","last_clean":"2026-02-21T22:32:45.667699+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:32:45.667699+0000","last_undegraded":"2026-02-21T22:32:45.667699+0000","last_fullsized":"2026-02-21T22:32:45.667699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1638,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.170146+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:32:46.170146+0000","last_peered":"2026-02-21T22:32:46.170146+0000","last_clean":"2026-02-21T22:32:46.170146+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:32:46.170146+0000","last_undegraded":"2026-02-21T22:32:46.170146+0000","last_fullsized":"2026-02-21T22:32:46.170146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293813+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293813+0000","last_peered":"2026-02-21T22:32:45.293813+0000","last_clean":"2026-02-21T22:32:45.293813+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:32:45.293813+0000","last_undegraded":"2026-02-21T22:32:45.293813+0000","last_fullsized":"2026-02-21T22:32:45.293813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169308+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:32:46.169308+0000","last_peered":"2026-02-21T22:32:46.169308+0000","last_clean":"2026-02-21T22:32:46.169308+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:32:46.169308+0000","last_undegraded":"2026-02-21T22:32:46.169308+0000","last_fullsized":"2026-02-21T22:32:46.169308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293852+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293852+0000","last_peered":"2026-02-21T22:32:45.293852+0000","last_clean":"2026-02-21T22:32:45.293852+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:32:45.293852+0000","last_undegraded":"2026-02-21T22:32:45.293852+0000","last_fullsized":"2026-02-21T22:32:45.293852+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169305+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:32:46.169305+0000","last_peered":"2026-02-21T22:32:46.169305+0000","last_clean":"2026-02-21T22:32:46.169305+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:32:46.169305+0000","last_undegraded":"2026-02-21T22:32:46.169305+0000","last_fullsized":"2026-02-21T22:32:46.169305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29412,"kb_used_data":2452,"kb_used_omap":1172,"kb_used_meta":25771,"kb_avail":94342428,"statfs":{"total":96636764160,"available":96606646272,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1200561,"internal_metadata":26390095},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2456,"kb_used_omap":1308,"kb_used_meta":25635,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1340101,"internal_metadata":26250555},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":854,"kb_used_meta":26089,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":874656,"internal_metadata":26716000},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640471,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":2452,"kb_used_omap":1802,"kb_used_meta":43189,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1845358,"internal_metadata":44226450},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47464,"kb_used_data":2456,"kb_used_omap":1841,"kb_used_meta":43150,"kb_avail":94324376,"statfs":{"total":96636764160,"available":96588161024,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1885730,"internal_metadata":44186078},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":1281,"kb_used_meta":25662,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":1312363,"internal_metadata":26278293},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:33:18.073 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:33:18.143 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: get_monmap_and_config 2026-02-21T22:33:18.143 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.143 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: monmap: 2026-02-21T22:33:18.143 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:18.142641+0000 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:18.142641+0000 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.144 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 1 Processor -- start 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 1 -- start start 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: init 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-c con 0x7f8c0008ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-b con 0x7f8c0008b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-a con 0x7f8c0008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c00086a10 con 0x7f8c0008d8e0 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0008b130 con 0x7f8c0008ce00 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0008bd60 con 0x7f8c0008b7f0 2026-02-21T22:33:18.145 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8c04d59640 10 monclient(hunting): authenticate will time out at 2207.378499s 2026-02-21T22:33:18.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8bfdd74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f8c0008b7f0 0x7f8c00181e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59128/0 (socket says 10.20.193.102:59128) 2026-02-21T22:33:18.146 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.139+0000 7f8bfdd74640 1 -- 10.20.193.102:0/398406669 learned_addr learned my addr 10.20.193.102:0/398406669 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfdd74640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.102:6790/0 conn(0x7f8c0008b7f0 legacy=0x7f8c00181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfed76640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.102:6789/0 conn(0x7f8c0008d8e0 legacy=0x7f8c001a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3080196425 0 0) 0x7f8c0008bd60 con 0x7f8c0008b7f0 2026-02-21T22:33:18.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.153 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): my global_id is 94397 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfe575640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.178:6789/0 conn(0x7f8c0008ce00 legacy=0x7f8c0008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bec003620 con 0x7f8c0008b7f0 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587030727 0 0) 0x7f8c00086a10 con 0x7f8c0008d8e0 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): my global_id is 34914 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c0008bd60 con 0x7f8c0008d8e0 2026-02-21T22:33:18.154 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301233775 0 0) 0x7f8c0008b130 con 0x7f8c0008ce00 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient(hunting): my global_id is 134131 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c00086a10 con 0x7f8c0008ce00 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4202893187 0 0) 0x7f8bec003620 con 0x7f8c0008b7f0 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.178:6789/0 conn(0x7f8c0008ce00 legacy=0x7f8c0008d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.102:6789/0 conn(0x7f8c0008d8e0 legacy=0x7f8c001a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 monclient: found mon.noname-b 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.155 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0008b370 con 0x7f8c0008b7f0 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8be8003190 con 0x7f8c0008b7f0 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: dump: 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.156 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 5 monclient: authenticate success, global_id 94397 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfdd74640 1 --1- 10.20.193.102:0/398406669 >> v1:10.20.193.102:6790/0 conn(0x7f8c0008b7f0 0x7f8c00181e40 :-1 s=OPENED pgs=573 gs=2 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f8be8004b50 mon_map magic: 0 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8be8003420 con 0x7f8c0008b7f0 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:18.157 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 4 set_mon_vals no callback set 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: shutdown 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- 10.20.193.102:0/398406669 >> v1:10.20.193.102:6790/0 conn(0x7f8c0008b7f0 legacy=0x7f8c00181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 1 -- 10.20.193.102:0/398406669 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8be8004b50 con 0x7f8c0008b7f0 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfd573640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- 10.20.193.102:0/398406669 shutdown_connections 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- 10.20.193.102:0/398406669 >> 10.20.193.102:0/398406669 conn(0x7f8c00091a50 msgr2=0x7f8c00091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- 10.20.193.102:0/398406669 shutdown_connections 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- 10.20.193.102:0/398406669 wait complete. 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: monmap: 2026-02-21T22:33:18.158 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 Processor -- start 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- start start 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: init 2026-02-21T22:33:18.159 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-b con 0x7f8c0008b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-c con 0x7f8c0008ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient(hunting): picked mon.noname-a con 0x7f8c0008d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c000852e0 con 0x7f8c0008d8e0 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c00083360 con 0x7f8c0008b7f0 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c00083640 con 0x7f8c0008ce00 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8c04d59640 10 monclient(hunting): authenticate will time out at 2207.382585s 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfdd74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f8c0008ce00 0x7f8c00089570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59130/0 (socket says 10.20.193.102:59130) 2026-02-21T22:33:18.160 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.143+0000 7f8bfdd74640 1 -- 10.20.193.102:0/3685872957 learned_addr learned my addr 10.20.193.102:0/3685872957 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8bfdd74640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.102:6790/0 conn(0x7f8c0008ce00 legacy=0x7f8c00089570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8bfed76640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.102:6789/0 conn(0x7f8c0008d8e0 legacy=0x7f8c001956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896256690 0 0) 0x7f8c00083640 con 0x7f8c0008ce00 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): my global_id is 94400 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bcc003620 con 0x7f8c0008ce00 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914705110 0 0) 0x7f8c000852e0 con 0x7f8c0008d8e0 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.161 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): my global_id is 34917 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c00083640 con 0x7f8c0008d8e0 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8bfe575640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.178:6789/0 conn(0x7f8c0008b7f0 legacy=0x7f8c00088e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2335583784 0 0) 0x7f8bcc003620 con 0x7f8c0008ce00 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8c000852e0 con 0x7f8c0008ce00 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8be8003190 con 0x7f8c0008ce00 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): dump: 2026-02-21T22:33:18.162 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.163 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316971861 0 0) 0x7f8c00083360 con 0x7f8c0008b7f0 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): my global_id is 134134 2026-02-21T22:33:18.164 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bcc003620 con 0x7f8c0008b7f0 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 942497469 0 0) 0x7f8c00083640 con 0x7f8c0008d8e0 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8c00083360 con 0x7f8c0008d8e0 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8bf0004930 con 0x7f8c0008d8e0 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient(hunting): dump: 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.165 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.166 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1605836603 0 0) 0x7f8c000852e0 con 0x7f8c0008ce00 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.178:6789/0 conn(0x7f8c0008b7f0 legacy=0x7f8c00088e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.102:6789/0 conn(0x7f8c0008d8e0 legacy=0x7f8c001956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 monclient: found mon.c 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c00089c80 con 0x7f8c0008ce00 2026-02-21T22:33:18.167 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 5 monclient: authenticate success, global_id 94400 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _renew_subs 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8c00058f00 con 0x7f8c0008ce00 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _renew_subs 2026-02-21T22:33:18.168 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f8c00083080 con 0x7f8c0008ce00 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8be8003420 con 0x7f8c0008ce00 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8be8005200 con 0x7f8c0008ce00 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:18.169 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 10 monclient: dump: 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.170 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8bfcd72640 4 set_mon_vals no callback set 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c000008e0 con 0x7f8c0008ce00 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f8be8011890 con 0x7f8c0008ce00 2026-02-21T22:33:18.171 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f8be804c580 con 0x7f8c0008ce00 2026-02-21T22:33:18.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.147+0000 7f8bfe575640 10 monclient: get_auth_request con 0x7f8bcc03d880 auth_method 0 2026-02-21T22:33:18.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.151+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f8be8017740 con 0x7f8c0008ce00 2026-02-21T22:33:18.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.151+0000 7f8be6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.172 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.151+0000 7f8be6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:18.235 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.231+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c00090810 con 0x7f8bcc03d880 2026-02-21T22:33:18.236 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.231+0000 7f8be6ffd640 1 -- 10.20.193.102:0/3685872957 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34069 (unknown 2965378022 0 3191784877) 0x7f8c00090810 con 0x7f8bcc03d880 2026-02-21T22:33:18.236 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:18.237 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:33:18.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.178:6800/2549949439 conn(0x7f8bcc03d880 legacy=0x7f8bcc05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.237 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 10 monclient: shutdown 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 >> v1:10.20.193.102:6790/0 conn(0x7f8c0008ce00 legacy=0x7f8c00089570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 shutdown_connections 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 >> 10.20.193.102:0/3685872957 conn(0x7f8c00091a50 msgr2=0x7f8c0005a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 shutdown_connections 2026-02-21T22:33:18.238 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.235+0000 7f8c04d59640 1 -- 10.20.193.102:0/3685872957 wait complete. 2026-02-21T22:33:18.245 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1368,"stamp":"2026-02-21T22:33:17.617173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":212704,"kb_used_data":14848,"kb_used_omap":8260,"kb_used_meta":189499,"kb_avail":566018336,"statfs":{"total":579820584960,"available":579602776064,"internally_reserved":0,"allocated":15204352,"data_stored":9432172,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":8458769,"internal_metadata":194047471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001018"},"pg_stats":[{"pgid":"1.0","version":"14'112","reported_seq":1759,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169478+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:32:46.169478+0000","last_peered":"2026-02-21T22:32:46.169478+0000","last_clean":"2026-02-21T22:32:46.169478+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:32:46.169478+0000","last_undegraded":"2026-02-21T22:32:46.169478+0000","last_fullsized":"2026-02-21T22:32:46.169478+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1637,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169991+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:32:46.169991+0000","last_peered":"2026-02-21T22:32:46.169991+0000","last_clean":"2026-02-21T22:32:46.169991+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:32:46.169991+0000","last_undegraded":"2026-02-21T22:32:46.169991+0000","last_fullsized":"2026-02-21T22:32:46.169991+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169377+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:32:46.169377+0000","last_peered":"2026-02-21T22:32:46.169377+0000","last_clean":"2026-02-21T22:32:46.169377+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:32:46.169377+0000","last_undegraded":"2026-02-21T22:32:46.169377+0000","last_fullsized":"2026-02-21T22:32:46.169377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1624,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.667699+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:32:45.667699+0000","last_peered":"2026-02-21T22:32:45.667699+0000","last_clean":"2026-02-21T22:32:45.667699+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:32:45.667699+0000","last_undegraded":"2026-02-21T22:32:45.667699+0000","last_fullsized":"2026-02-21T22:32:45.667699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1638,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.170146+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:32:46.170146+0000","last_peered":"2026-02-21T22:32:46.170146+0000","last_clean":"2026-02-21T22:32:46.170146+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:32:46.170146+0000","last_undegraded":"2026-02-21T22:32:46.170146+0000","last_fullsized":"2026-02-21T22:32:46.170146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293813+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293813+0000","last_peered":"2026-02-21T22:32:45.293813+0000","last_clean":"2026-02-21T22:32:45.293813+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:32:45.293813+0000","last_undegraded":"2026-02-21T22:32:45.293813+0000","last_fullsized":"2026-02-21T22:32:45.293813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169308+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:32:46.169308+0000","last_peered":"2026-02-21T22:32:46.169308+0000","last_clean":"2026-02-21T22:32:46.169308+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:32:46.169308+0000","last_undegraded":"2026-02-21T22:32:46.169308+0000","last_fullsized":"2026-02-21T22:32:46.169308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293852+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293852+0000","last_peered":"2026-02-21T22:32:45.293852+0000","last_clean":"2026-02-21T22:32:45.293852+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:32:45.293852+0000","last_undegraded":"2026-02-21T22:32:45.293852+0000","last_fullsized":"2026-02-21T22:32:45.293852+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169305+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:32:46.169305+0000","last_peered":"2026-02-21T22:32:46.169305+0000","last_clean":"2026-02-21T22:32:46.169305+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:32:46.169305+0000","last_undegraded":"2026-02-21T22:32:46.169305+0000","last_fullsized":"2026-02-21T22:32:46.169305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29412,"kb_used_data":2452,"kb_used_omap":1172,"kb_used_meta":25771,"kb_avail":94342428,"statfs":{"total":96636764160,"available":96606646272,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1200561,"internal_metadata":26390095},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2456,"kb_used_omap":1308,"kb_used_meta":25635,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1340101,"internal_metadata":26250555},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":854,"kb_used_meta":26089,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":874656,"internal_metadata":26716000},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640471,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":2452,"kb_used_omap":1802,"kb_used_meta":43189,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1845358,"internal_metadata":44226450},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47464,"kb_used_data":2456,"kb_used_omap":1841,"kb_used_meta":43150,"kb_avail":94324376,"statfs":{"total":96636764160,"available":96588161024,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1885730,"internal_metadata":44186078},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":1281,"kb_used_meta":25662,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":1312363,"internal_metadata":26278293},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:33:18.247 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T22:33:18.247 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: get_monmap_and_config 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: monmap: 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:18.307669+0000 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:18.307669+0000 2026-02-21T22:33:18.307 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.308 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.308 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.308 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:18.308 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:18.308 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.322 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 1 Processor -- start 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 1 -- start start 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: init 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-b con 0x7fa3dc08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-a con 0x7fa3dc08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-c con 0x7fa3dc08d8e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc086a10 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc08b130 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.323 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc08bd60 con 0x7fa3dc08ce00 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e3a71640 10 monclient(hunting): authenticate will time out at 2207.543953s 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.303+0000 7fa3e17e6640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08ce00 0x7fa3dc08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59138/0 (socket says 10.20.193.102:59138) 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e17e6640 1 -- 10.20.193.102:0/3247541995 learned_addr learned my addr 10.20.193.102:0/3247541995 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e17e6640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08ce00 legacy=0x7fa3dc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e0fe5640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042052295 0 0) 0x7fa3dc08bd60 con 0x7fa3dc08ce00 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.324 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): my global_id is 94403 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3c4003620 con 0x7fa3dc08ce00 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905642554 0 0) 0x7fa3dc086a10 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.325 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient(hunting): my global_id is 34920 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3dc08bd60 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e1fe7640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3dc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1096174421 0 0) 0x7fa3c4003620 con 0x7fa3dc08ce00 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3dc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 monclient: found mon.noname-b 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.326 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc08b370 con 0x7fa3dc08ce00 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3cc003110 con 0x7fa3dc08ce00 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 5 monclient: authenticate success, global_id 94403 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: dump: 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.327 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3cc0033e0 con 0x7fa3dc08ce00 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.328 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 1 -- 10.20.193.102:0/3247541995 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3cc004b50 con 0x7fa3dc08ce00 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3cbfff640 10 monclient: dump: 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.329 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 4 set_mon_vals no callback set 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: shutdown 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.330 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- 10.20.193.102:0/3247541995 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08ce00 legacy=0x7fa3dc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- 10.20.193.102:0/3247541995 shutdown_connections 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- 10.20.193.102:0/3247541995 >> 10.20.193.102:0/3247541995 conn(0x7fa3dc091a50 msgr2=0x7fa3dc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- 10.20.193.102:0/3247541995 shutdown_connections 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- 10.20.193.102:0/3247541995 wait complete. 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: monmap: 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:18.331 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 Processor -- start 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 1 -- start start 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: init 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-a con 0x7fa3dc08b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.332 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-b con 0x7fa3dc08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.307+0000 7fa3e3a71640 10 monclient(hunting): picked mon.noname-c con 0x7fa3dc08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc05e410 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc088a70 con 0x7fa3dc08ce00 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3dc153910 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient(hunting): authenticate will time out at 2207.548038s 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 0x7fa3dc084970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37880/0 (socket says 10.20.193.102:37880) 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 -- 10.20.193.102:0/2495842953 learned_addr learned my addr 10.20.193.102:0/2495842953 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e1fe7640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3dc195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642742590 0 0) 0x7fa3dc05e410 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.333 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): my global_id is 34923 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3ac003620 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098928642 0 0) 0x7fa3dc153910 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e0fe5640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08ce00 legacy=0x7fa3dc1945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): my global_id is 94406 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3dc05e410 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.334 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1808727310 0 0) 0x7fa3ac003620 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa3dc153910 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3cc002b70 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.335 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 682876671 0 0) 0x7fa3dc05e410 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa3ac003620 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3d8004890 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.336 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3772008726 0 0) 0x7fa3dc088a70 con 0x7fa3dc08ce00 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.337 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): my global_id is 134137 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3dc05e410 con 0x7fa3dc08ce00 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2864836059 0 0) 0x7fa3dc153910 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3dc195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08ce00 legacy=0x7fa3dc1945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 monclient: found mon.a 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.338 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc0810e0 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 5 monclient: authenticate success, global_id 34923 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _renew_subs 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc082ba0 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _renew_subs 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa3dc085080 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3cc0033c0 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.339 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3cc003570 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: dump: 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.340 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3cb7fe640 4 set_mon_vals no callback set 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa3cc011810 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc05c0a0 con 0x7fa3dc08b7f0 2026-02-21T22:33:18.341 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc084970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 --1- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 0x7fa3dc084970 :-1 s=OPENED pgs=682 gs=1 cs=1 l=1).handle_message read tag failed 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 1 --1- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 0x7fa3dc084970 :-1 s=OPENED pgs=682 gs=1 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6789/0 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3dc08b7f0 legacy=0x7fa3dc084970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): picked mon.a con 0x7fa3ac05f4d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.342 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): picked mon.b con 0x7fa3dc08ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): picked mon.c con 0x7fa3dc08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa3dc153910 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa3dc088a70 con 0x7fa3dc08ce00 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa3ac0c6990 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3c9ffb640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.311+0000 7fa3e17e6640 10 monclient(hunting): get_auth_request con 0x7fa3ac03d830 auth_method 0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3e0fe5640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3ac05f4d0 legacy=0x7fa3ac07f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3e17e6640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3ac0c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549149855 0 0) 0x7fa3dc153910 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3e1fe7640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08ce00 legacy=0x7fa3ac0a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:33:18.343 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fa3ac0c6b60 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638415673 0 0) 0x7fa3ac0c6990 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fa3dc153910 con 0x7fa3dc08d8e0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 2737290211 0 0) 0x7fa3ac0c6b60 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6790/0 conn(0x7fa3dc08d8e0 legacy=0x7fa3ac0c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.178:6789/0 conn(0x7fa3dc08ce00 legacy=0x7fa3ac0a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 monclient: found mon.a 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7fa3ac0c7b90 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.344 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc088a70 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fa3d00031d0 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: dump: 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.345 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa3d00033b0 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3caffd640 4 set_mon_vals no callback set 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fa3d0003710 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.315+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 6 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7fa3d004ba00 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.319+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fa3d0048b10 con 0x7fa3ac05f4d0 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.319+0000 7fa3c9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.346 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.319+0000 7fa3c9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:18.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3dc062ce0 con 0x7fa3ac03d830 2026-02-21T22:33:18.401 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3c9ffb640 1 -- 10.20.193.102:0/2495842953 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34069 (unknown 2965378022 0 3191784877) 0x7fa3dc062ce0 con 0x7fa3ac03d830 2026-02-21T22:33:18.402 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.178:6800/2549949439 conn(0x7fa3ac03d830 legacy=0x7fa3ac05dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 10 monclient: shutdown 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 >> v1:10.20.193.102:6789/0 conn(0x7fa3ac05f4d0 legacy=0x7fa3ac07f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 shutdown_connections 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 >> 10.20.193.102:0/2495842953 conn(0x7fa3dc091a50 msgr2=0x7fa3dc089570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 shutdown_connections 2026-02-21T22:33:18.403 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.399+0000 7fa3e3a71640 1 -- 10.20.193.102:0/2495842953 wait complete. 2026-02-21T22:33:18.412 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1368,"stamp":"2026-02-21T22:33:17.617173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":212704,"kb_used_data":14848,"kb_used_omap":8260,"kb_used_meta":189499,"kb_avail":566018336,"statfs":{"total":579820584960,"available":579602776064,"internally_reserved":0,"allocated":15204352,"data_stored":9432172,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":8458769,"internal_metadata":194047471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001018"},"pg_stats":[{"pgid":"1.0","version":"14'112","reported_seq":1759,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169478+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:32:46.169478+0000","last_peered":"2026-02-21T22:32:46.169478+0000","last_clean":"2026-02-21T22:32:46.169478+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:32:46.169478+0000","last_undegraded":"2026-02-21T22:32:46.169478+0000","last_fullsized":"2026-02-21T22:32:46.169478+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1637,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169991+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:32:46.169991+0000","last_peered":"2026-02-21T22:32:46.169991+0000","last_clean":"2026-02-21T22:32:46.169991+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:32:46.169991+0000","last_undegraded":"2026-02-21T22:32:46.169991+0000","last_fullsized":"2026-02-21T22:32:46.169991+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169377+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:32:46.169377+0000","last_peered":"2026-02-21T22:32:46.169377+0000","last_clean":"2026-02-21T22:32:46.169377+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:32:46.169377+0000","last_undegraded":"2026-02-21T22:32:46.169377+0000","last_fullsized":"2026-02-21T22:32:46.169377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1624,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.667699+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:32:45.667699+0000","last_peered":"2026-02-21T22:32:45.667699+0000","last_clean":"2026-02-21T22:32:45.667699+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:32:45.667699+0000","last_undegraded":"2026-02-21T22:32:45.667699+0000","last_fullsized":"2026-02-21T22:32:45.667699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1638,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.170146+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:32:46.170146+0000","last_peered":"2026-02-21T22:32:46.170146+0000","last_clean":"2026-02-21T22:32:46.170146+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:32:46.170146+0000","last_undegraded":"2026-02-21T22:32:46.170146+0000","last_fullsized":"2026-02-21T22:32:46.170146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293813+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293813+0000","last_peered":"2026-02-21T22:32:45.293813+0000","last_clean":"2026-02-21T22:32:45.293813+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:32:45.293813+0000","last_undegraded":"2026-02-21T22:32:45.293813+0000","last_fullsized":"2026-02-21T22:32:45.293813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169308+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:32:46.169308+0000","last_peered":"2026-02-21T22:32:46.169308+0000","last_clean":"2026-02-21T22:32:46.169308+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:32:46.169308+0000","last_undegraded":"2026-02-21T22:32:46.169308+0000","last_fullsized":"2026-02-21T22:32:46.169308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293852+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293852+0000","last_peered":"2026-02-21T22:32:45.293852+0000","last_clean":"2026-02-21T22:32:45.293852+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:32:45.293852+0000","last_undegraded":"2026-02-21T22:32:45.293852+0000","last_fullsized":"2026-02-21T22:32:45.293852+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169305+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:32:46.169305+0000","last_peered":"2026-02-21T22:32:46.169305+0000","last_clean":"2026-02-21T22:32:46.169305+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:32:46.169305+0000","last_undegraded":"2026-02-21T22:32:46.169305+0000","last_fullsized":"2026-02-21T22:32:46.169305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29412,"kb_used_data":2452,"kb_used_omap":1172,"kb_used_meta":25771,"kb_avail":94342428,"statfs":{"total":96636764160,"available":96606646272,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1200561,"internal_metadata":26390095},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2456,"kb_used_omap":1308,"kb_used_meta":25635,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1340101,"internal_metadata":26250555},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":854,"kb_used_meta":26089,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":874656,"internal_metadata":26716000},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640471,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":2452,"kb_used_omap":1802,"kb_used_meta":43189,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1845358,"internal_metadata":44226450},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47464,"kb_used_data":2456,"kb_used_omap":1841,"kb_used_meta":43150,"kb_avail":94324376,"statfs":{"total":96636764160,"available":96588161024,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1885730,"internal_metadata":44186078},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640474,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":1281,"kb_used_meta":25662,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":1312363,"internal_metadata":26278293},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:33:18.414 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T22:33:18.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: get_monmap_and_config 2026-02-21T22:33:18.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: monmap: 2026-02-21T22:33:18.476 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:18.475387+0000 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:18.475387+0000 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 1 Processor -- start 2026-02-21T22:33:18.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 1 -- start start 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: init 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-b con 0x7f91f408cdc0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-a con 0x7f91f408d8a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-c con 0x7f91f408b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f40890e0 con 0x7f91f408d8a0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f4088ea0 con 0x7f91f408b8d0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f408aea0 con 0x7f91f408cdc0 2026-02-21T22:33:18.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91fbbbf640 10 monclient(hunting): authenticate will time out at 2207.711445s 2026-02-21T22:33:18.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91f9934640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f91f408cdc0 0x7f91f408d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59172/0 (socket says 10.20.193.102:59172) 2026-02-21T22:33:18.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91f9934640 1 -- 10.20.193.102:0/155964445 learned_addr learned my addr 10.20.193.102:0/155964445 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.471+0000 7f91f9133640 1 --1- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 0x7f91f41840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37908/0 (socket says 10.20.193.102:37908) 2026-02-21T22:33:18.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f9934640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6790/0 conn(0x7f91f408cdc0 legacy=0x7f91f408d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.486 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f9133640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 legacy=0x7f91f41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59983811 0 0) 0x7f91f40890e0 con 0x7f91f408d8a0 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): my global_id is 34926 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91dc003620 con 0x7f91f408d8a0 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576700788 0 0) 0x7f91f408aea0 con 0x7f91f408cdc0 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient(hunting): my global_id is 94409 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f40890e0 con 0x7f91f408cdc0 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fa135640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.178:6789/0 conn(0x7f91f408b8d0 legacy=0x7f91f41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 258937661 0 0) 0x7f91dc003620 con 0x7f91f408d8a0 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6790/0 conn(0x7f91f408cdc0 legacy=0x7f91f408d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.178:6789/0 conn(0x7f91f408b8d0 legacy=0x7f91f41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 monclient: found mon.noname-a 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f408a600 con 0x7f91f408d8a0 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f91e8003040 con 0x7f91f408d8a0 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: dump: 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.489 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.490 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 5 monclient: authenticate success, global_id 34926 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f9133640 1 --1- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 0x7f91f41840b0 :-1 s=OPENED pgs=684 gs=2 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f91e80048e0 mon_map magic: 0 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f91e8003310 con 0x7f91f408d8a0 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 4 set_mon_vals no callback set 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: shutdown 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.491 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- 10.20.193.102:0/155964445 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 legacy=0x7f91f41840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 1 -- 10.20.193.102:0/155964445 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f91e80048e0 con 0x7f91f408d8a0 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91f8932640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- 10.20.193.102:0/155964445 shutdown_connections 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- 10.20.193.102:0/155964445 >> 10.20.193.102:0/155964445 conn(0x7f91f4091a50 msgr2=0x7f91f4091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- 10.20.193.102:0/155964445 shutdown_connections 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- 10.20.193.102:0/155964445 wait complete. 2026-02-21T22:33:18.492 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: build_initial_monmap 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: monmap: 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:18.493 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 Processor -- start 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 1 -- start start 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: init 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.475+0000 7f91fbbbf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-c con 0x7f91f408b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:18.494 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-b con 0x7f91f408cdc0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient(hunting): picked mon.noname-a con 0x7f91f408d8a0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f405d4b0 con 0x7f91f408d8a0 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f4088a70 con 0x7f91f408cdc0 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f4153910 con 0x7f91f408b8d0 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient(hunting): authenticate will time out at 2207.716087s 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91f9934640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f91f408b8d0 0x7f91f4083750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59184/0 (socket says 10.20.193.102:59184) 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91f9934640 1 -- 10.20.193.102:0/4063737756 learned_addr learned my addr 10.20.193.102:0/4063737756 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:18.495 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91f9934640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.102:6790/0 conn(0x7f91f408b8d0 legacy=0x7f91f4083750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fa135640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 legacy=0x7f91f4195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681612301 0 0) 0x7f91f4153910 con 0x7f91f408b8d0 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): my global_id is 94412 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91c4003620 con 0x7f91f408b8d0 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3255425138 0 0) 0x7f91f405d4b0 con 0x7f91f408d8a0 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.496 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): my global_id is 34929 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f4153910 con 0x7f91f408d8a0 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91f9133640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.178:6789/0 conn(0x7f91f408cdc0 legacy=0x7f91f41945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2401488700 0 0) 0x7f91c4003620 con 0x7f91f408b8d0 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91f405d4b0 con 0x7f91f408b8d0 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f91e40048e0 con 0x7f91f408b8d0 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): dump: 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.497 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:18.498 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3590969102 0 0) 0x7f91f4153910 con 0x7f91f408d8a0 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91c4003620 con 0x7f91f408d8a0 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f91f00047c0 con 0x7f91f408d8a0 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): dump: 2026-02-21T22:33:18.499 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.500 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790393209 0 0) 0x7f91f4088a70 con 0x7f91f408cdc0 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:18.501 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient(hunting): my global_id is 134140 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f4153910 con 0x7f91f408cdc0 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3794922594 0 0) 0x7f91f405d4b0 con 0x7f91f408b8d0 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.178:6789/0 conn(0x7f91f408cdc0 legacy=0x7f91f41945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.102:6789/0 conn(0x7f91f408d8a0 legacy=0x7f91f4195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: _finish_hunting 0 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 monclient: found mon.c 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.502 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f407fec0 con 0x7f91f408b8d0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: _finish_auth 0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: _check_auth_tickets 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 5 monclient: authenticate success, global_id 94412 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _renew_subs 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f91f4082220 con 0x7f91f408b8d0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _renew_subs 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f91f4083e60 con 0x7f91f408b8d0 2026-02-21T22:33:18.503 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f91e4004d30 con 0x7f91f408b8d0 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f91e40050c0 con 0x7f91f408b8d0 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 10 monclient: dump: 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:18.504 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e3fff640 4 set_mon_vals no callback set 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f4081a30 con 0x7f91f408b8d0 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f91e4005390 con 0x7f91f408b8d0 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.479+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f91e404c640 con 0x7f91f408b8d0 2026-02-21T22:33:18.505 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.483+0000 7f91f9133640 10 monclient: get_auth_request con 0x7f91c403d820 auth_method 0 2026-02-21T22:33:18.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.483+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f91e40176e0 con 0x7f91f408b8d0 2026-02-21T22:33:18.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.483+0000 7f91e27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:18.506 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.483+0000 7f91e27fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.571+0000 7f91fbbbf640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.571+0000 7f91fbbbf640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.571+0000 7f91fbbbf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.571+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91f4062ce0 con 0x7f91f408b8d0 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91e27fc640 1 -- 10.20.193.102:0/4063737756 <== mon.2 v1:10.20.193.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v831) ==== 74+0+11046 (unknown 1316510701 0 30461340) 0x7f91e401ca00 con 0x7f91f408b8d0 2026-02-21T22:33:18.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91e27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-21T22:33:18.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91e27fc640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:18.577 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:18.577 INFO:teuthology.orchestra.run.trial102.stdout:{"epoch":831,"fsid":"740d24fb-83fc-48ed-963d-a0ec1185b484","created":"2026-02-21T22:15:22.685299+0000","modified":"2026-02-21T22:32:45.286325+0000","last_up_change":"2026-02-21T22:15:29.415195+0000","last_in_change":"2026-02-21T22:15:24.786352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T22:15:29.531133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T22:15:33.157956+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"883e925c-449b-4715-a8e9-b2a55b4b2970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6804","nonce":604370691}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6805","nonce":604370691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6807","nonce":604370691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6806","nonce":604370691}]},"public_addr":"10.20.193.102:6804/604370691","cluster_addr":"10.20.193.102:6805/604370691","heartbeat_back_addr":"10.20.193.102:6807/604370691","heartbeat_front_addr":"10.20.193.102:6806/604370691","state":["exists","up"]},{"osd":1,"uuid":"00fe9fdd-1d81-4b68-8b83-dc19782e2b32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6800","nonce":4138446215}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6801","nonce":4138446215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6803","nonce":4138446215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6802","nonce":4138446215}]},"public_addr":"10.20.193.102:6800/4138446215","cluster_addr":"10.20.193.102:6801/4138446215","heartbeat_back_addr":"10.20.193.102:6803/4138446215","heartbeat_front_addr":"10.20.193.102:6802/4138446215","state":["exists","up"]},{"osd":2,"uuid":"1047409d-0635-4dc6-bf64-7d300e5858de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6808","nonce":579535315}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6809","nonce":579535315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6811","nonce":579535315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.102:6810","nonce":579535315}]},"public_addr":"10.20.193.102:6808/579535315","cluster_addr":"10.20.193.102:6809/579535315","heartbeat_back_addr":"10.20.193.102:6811/579535315","heartbeat_front_addr":"10.20.193.102:6810/579535315","state":["exists","up"]},{"osd":3,"uuid":"a091d7ab-39a1-4f2b-9ccd-97e4d9c75e09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6805","nonce":653951931}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6806","nonce":653951931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6808","nonce":653951931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6807","nonce":653951931}]},"public_addr":"10.20.193.178:6805/653951931","cluster_addr":"10.20.193.178:6806/653951931","heartbeat_back_addr":"10.20.193.178:6808/653951931","heartbeat_front_addr":"10.20.193.178:6807/653951931","state":["exists","up"]},{"osd":4,"uuid":"507e846b-017c-4655-839e-e62916a2128f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6809","nonce":3629312922}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6810","nonce":3629312922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6812","nonce":3629312922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6811","nonce":3629312922}]},"public_addr":"10.20.193.178:6809/3629312922","cluster_addr":"10.20.193.178:6810/3629312922","heartbeat_back_addr":"10.20.193.178:6812/3629312922","heartbeat_front_addr":"10.20.193.178:6811/3629312922","state":["exists","up"]},{"osd":5,"uuid":"c24d73e4-3129-4e17-bde0-940b2c5e2abf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6801","nonce":2402357529}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6802","nonce":2402357529}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6804","nonce":2402357529}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"10.20.193.178:6803","nonce":2402357529}]},"public_addr":"10.20.193.178:6801/2402357529","cluster_addr":"10.20.193.178:6802/2402357529","heartbeat_back_addr":"10.20.193.178:6804/2402357529","heartbeat_front_addr":"10.20.193.178:6803/2402357529","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.408868+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.421183+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.371888+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.363811+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.440244+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T22:15:27.362846+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":381,"length":1},{"begin":384,"length":1},{"begin":408,"length":1},{"begin":417,"length":3}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T22:33:18.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.178:6800/2549949439 conn(0x7f91c403d820 legacy=0x7f91c405dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 10 monclient: shutdown 2026-02-21T22:33:18.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:18.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 >> v1:10.20.193.102:6790/0 conn(0x7f91f408b8d0 legacy=0x7f91f4083750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:18.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 shutdown_connections 2026-02-21T22:33:18.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 >> 10.20.193.102:0/4063737756 conn(0x7f91f4091a50 msgr2=0x7f91f40882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:18.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 shutdown_connections 2026-02-21T22:33:18.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:18.575+0000 7f91fbbbf640 1 -- 10.20.193.102:0/4063737756 wait complete. 2026-02-21T22:33:18.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:18.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:19.588 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T22:33:19.589 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: get_monmap_and_config 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: monmap: 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:19.652826+0000 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:19.652826+0000 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.652 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 1 Processor -- start 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 1 -- start start 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: init 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-b con 0x7f889008c1a0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-a con 0x7f889008cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-c con 0x7f88901834f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889008d4a0 con 0x7f889008cc80 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889008d050 con 0x7f88901834f0 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8890088730 con 0x7f889008c1a0 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient(hunting): authenticate will time out at 2208.888886s 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888f7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f889008c1a0 0x7f889008c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59188/0 (socket says 10.20.193.102:59188) 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888effd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f889008cc80 0x7f8890181dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37926/0 (socket says 10.20.193.102:37926) 2026-02-21T22:33:19.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888f7fe640 1 -- 10.20.193.102:0/2959862710 learned_addr learned my addr 10.20.193.102:0/2959862710 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888f7fe640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.102:6790/0 conn(0x7f889008c1a0 legacy=0x7f889008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888effd640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.102:6789/0 conn(0x7f889008cc80 legacy=0x7f8890181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063796466 0 0) 0x7f8890088730 con 0x7f889008c1a0 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): my global_id is 94415 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888ffff640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.178:6789/0 conn(0x7f88901834f0 legacy=0x7f88901a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8874003620 con 0x7f889008c1a0 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27807788 0 0) 0x7f889008d4a0 con 0x7f889008cc80 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient(hunting): my global_id is 34932 2026-02-21T22:33:19.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8890088730 con 0x7f889008cc80 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476469976 0 0) 0x7f8874003620 con 0x7f889008c1a0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.178:6789/0 conn(0x7f88901834f0 legacy=0x7f88901a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.102:6789/0 conn(0x7f889008cc80 legacy=0x7f8890181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 monclient: found mon.noname-b 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88900857a0 con 0x7f889008c1a0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f887c003110 con 0x7f889008c1a0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: dump: 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 5 monclient: authenticate success, global_id 94415 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f887c003450 con 0x7f889008c1a0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 1 -- 10.20.193.102:0/2959862710 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f887c004b50 con 0x7f889008c1a0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f888e7fc640 10 monclient: dump: 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.656 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 4 set_mon_vals no callback set 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.651+0000 7f8895eb9640 10 monclient: shutdown 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- 10.20.193.102:0/2959862710 >> v1:10.20.193.102:6790/0 conn(0x7f889008c1a0 legacy=0x7f889008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- 10.20.193.102:0/2959862710 shutdown_connections 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- 10.20.193.102:0/2959862710 >> 10.20.193.102:0/2959862710 conn(0x7f88900917a0 msgr2=0x7f8890091ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- 10.20.193.102:0/2959862710 shutdown_connections 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- 10.20.193.102:0/2959862710 wait complete. 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient: monmap: 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:19.657 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 Processor -- start 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- start start 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient: init 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-b con 0x7f889008c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-a con 0x7f889008cc80 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient(hunting): picked mon.noname-c con 0x7f88901834f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889007d950 con 0x7f889008cc80 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889007e4b0 con 0x7f889008c1a0 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889007f410 con 0x7f88901834f0 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f8895eb9640 10 monclient(hunting): authenticate will time out at 2208.893432s 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f888ffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f88901834f0 0x7f889011f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59192/0 (socket says 10.20.193.102:59192) 2026-02-21T22:33:19.658 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f888ffff640 1 -- 10.20.193.102:0/185888282 learned_addr learned my addr 10.20.193.102:0/185888282 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f888ffff640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6790/0 conn(0x7f88901834f0 legacy=0x7f889011f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f888effd640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6789/0 conn(0x7f889008cc80 legacy=0x7f8890181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f888ffff640 1 --1- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6790/0 conn(0x7f88901834f0 0x7f889011f8f0 :-1 s=OPENED pgs=581 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f889007f410 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355277801 0 0) 0x7f889007d950 con 0x7f889008cc80 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 10 monclient(hunting): my global_id is 34935 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.655+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8860003620 con 0x7f889008cc80 2026-02-21T22:33:19.659 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f888f7fe640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.178:6789/0 conn(0x7f889008c1a0 legacy=0x7f88901411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2368083241 0 0) 0x7f8860003620 con 0x7f889008cc80 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f889007d950 con 0x7f889008cc80 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8880002e70 con 0x7f889008cc80 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): dump: 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.660 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209456649 0 0) 0x7f889007e4b0 con 0x7f889008c1a0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient(hunting): my global_id is 134146 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8860003620 con 0x7f889008c1a0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 592934819 0 0) 0x7f889007d950 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6790/0 conn(0x7f88901834f0 legacy=0x7f889011f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.178:6789/0 conn(0x7f889008c1a0 legacy=0x7f88901411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 monclient: found mon.a 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889007f6f0 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2961834028 0 0) 0x7f889007f410 con 0x7f88901834f0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 5 monclient: authenticate success, global_id 34935 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 10 monclient: _renew_subs 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f88901418b0 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 10 monclient: _renew_subs 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f88900853a0 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8880004b90 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f8880004f30 con 0x7f889008cc80 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f888dffb640 4 set_mon_vals no callback set 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: dump: 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.661 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f8890084ce0 con 0x7f889008cc80 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f88800051d0 con 0x7f889008cc80 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f888004c4c0 con 0x7f889008cc80 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6804/604370691 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8880004d80 con 0x7f8860062130 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f888004c800 con 0x7f889008cc80 2026-02-21T22:33:19.662 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f886bfff640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:19.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f888f7fe640 10 monclient: get_auth_request con 0x7f886003d520 auth_method 0 2026-02-21T22:33:19.663 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.659+0000 7f888ffff640 10 monclient: get_auth_request con 0x7f8860062130 auth_method 0 2026-02-21T22:33:19.666 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.663+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== osd.0 v1:10.20.193.102:6804/604370691 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f8880004d80 con 0x7f8860062130 2026-02-21T22:33:19.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 --> v1:10.20.193.102:6804/604370691 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8890059ae0 con 0x7f8860062130 2026-02-21T22:33:19.673 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f886bfff640 1 -- 10.20.193.102:0/185888282 <== osd.0 v1:10.20.193.102:6804/604370691 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f8890059ae0 con 0x7f8860062130 2026-02-21T22:33:19.673 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:19.673 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:19.673 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6804/604370691 conn(0x7f8860062130 legacy=0x7f8860082540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.178:6800/2549949439 conn(0x7f886003d520 legacy=0x7f886005d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 10 monclient: shutdown 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 >> v1:10.20.193.102:6789/0 conn(0x7f889008cc80 legacy=0x7f8890181dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f888ffff640 1 -- 10.20.193.102:0/185888282 reap_dead start 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 shutdown_connections 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 >> 10.20.193.102:0/185888282 conn(0x7f88900917a0 msgr2=0x7f889008e230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 shutdown_connections 2026-02-21T22:33:19.674 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.671+0000 7f8895eb9640 1 -- 10.20.193.102:0/185888282 wait complete. 2026-02-21T22:33:19.681 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T22:33:19.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: get_monmap_and_config 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: monmap: 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:19.751422+0000 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:19.751422+0000 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.752 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 1 Processor -- start 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 1 -- start start 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: init 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient(hunting): picked mon.noname-b con 0x7fc97c08cd00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient(hunting): picked mon.noname-a con 0x7fc97c08d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient(hunting): picked mon.noname-c con 0x7fc97c08b8d0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c0890e0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c088ea0 con 0x7fc97c08b8d0 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c08aea0 con 0x7fc97c08cd00 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc983904640 10 monclient(hunting): authenticate will time out at 2208.987305s 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc981679640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fc97c08cd00 0x7fc97c08d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59202/0 (socket says 10.20.193.102:59202) 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc980e78640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fc97c08d7e0 0x7fc97c1840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37958/0 (socket says 10.20.193.102:37958) 2026-02-21T22:33:19.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.747+0000 7fc981679640 1 -- 10.20.193.102:0/3628068539 learned_addr learned my addr 10.20.193.102:0/3628068539 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc980e78640 1 -- 10.20.193.102:0/3628068539 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08d7e0 legacy=0x7fc97c1840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc981679640 1 -- 10.20.193.102:0/3628068539 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08cd00 legacy=0x7fc97c08d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767675205 0 0) 0x7fc97c0890e0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): my global_id is 34938 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc964003620 con 0x7fc97c08d7e0 2026-02-21T22:33:19.763 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367491335 0 0) 0x7fc97c08aea0 con 0x7fc97c08cd00 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient(hunting): my global_id is 94421 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc97c0890e0 con 0x7fc97c08cd00 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3219072469 0 0) 0x7fc964003620 con 0x7fc97c08d7e0 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08cd00 legacy=0x7fc97c08d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 >> v1:10.20.193.178:6789/0 conn(0x7fc97c08b8d0 legacy=0x7fc97c1a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 monclient: found mon.noname-a 2026-02-21T22:33:19.764 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c08a600 con 0x7fc97c08d7e0 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 5 monclient: authenticate success, global_id 34938 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc970003110 con 0x7fc97c08d7e0 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.765 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: dump: 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.766 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc9700033e0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 1 -- 10.20.193.102:0/3628068539 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc970004a90 con 0x7fc97c08d7e0 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc96bfff640 10 monclient: dump: 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.767 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 4 set_mon_vals no callback set 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: shutdown 2026-02-21T22:33:19.768 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- 10.20.193.102:0/3628068539 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08d7e0 legacy=0x7fc97c1840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- 10.20.193.102:0/3628068539 shutdown_connections 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- 10.20.193.102:0/3628068539 >> 10.20.193.102:0/3628068539 conn(0x7fc97c091a50 msgr2=0x7fc97c091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- 10.20.193.102:0/3628068539 shutdown_connections 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- 10.20.193.102:0/3628068539 wait complete. 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: monmap: 2026-02-21T22:33:19.769 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 Processor -- start 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- start start 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: init 2026-02-21T22:33:19.770 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient(hunting): picked mon.noname-a con 0x7fc97c08b8d0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient(hunting): picked mon.noname-b con 0x7fc97c08cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient(hunting): picked mon.noname-c con 0x7fc97c08d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c07fc80 con 0x7fc97c08b8d0 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c088a70 con 0x7fc97c08cd00 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c05d560 con 0x7fc97c08d7e0 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc983904640 10 monclient(hunting): authenticate will time out at 2208.991898s 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc981679640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 0x7fc97c1537a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37974/0 (socket says 10.20.193.102:37974) 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.751+0000 7fc981679640 1 -- 10.20.193.102:0/621684944 learned_addr learned my addr 10.20.193.102:0/621684944 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981679640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 legacy=0x7fc97c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.771 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981e7a640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08d7e0 legacy=0x7fc97c195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009383982 0 0) 0x7fc97c07fc80 con 0x7fc97c08b8d0 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): my global_id is 34941 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc94c003620 con 0x7fc97c08b8d0 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074763404 0 0) 0x7fc97c05d560 con 0x7fc97c08d7e0 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): my global_id is 94424 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc97c07fc80 con 0x7fc97c08d7e0 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc980e78640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.178:6789/0 conn(0x7fc97c08cd00 legacy=0x7fc97c1943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.772 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 106027957 0 0) 0x7fc94c003620 con 0x7fc97c08b8d0 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc97c05d560 con 0x7fc97c08b8d0 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc96c003290 con 0x7fc97c08b8d0 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.773 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2924447303 0 0) 0x7fc97c088a70 con 0x7fc97c08cd00 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): my global_id is 134149 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc94c003620 con 0x7fc97c08cd00 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2384455073 0 0) 0x7fc97c07fc80 con 0x7fc97c08d7e0 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc97c088a70 con 0x7fc97c08d7e0 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc978005890 con 0x7fc97c08d7e0 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:19.774 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.775 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 135032207 0 0) 0x7fc97c05d560 con 0x7fc97c08b8d0 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08d7e0 legacy=0x7fc97c195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.178:6789/0 conn(0x7fc97c08cd00 legacy=0x7fc97c1943c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 monclient: found mon.a 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c081fe0 con 0x7fc97c08b8d0 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.776 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 5 monclient: authenticate success, global_id 34941 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 monclient: _renew_subs 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c081740 con 0x7fc97c08b8d0 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 monclient: _renew_subs 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc97c0807e0 con 0x7fc97c08b8d0 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981679640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 legacy=0x7fc97c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981679640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 legacy=0x7fc97c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981679640 1 --1- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 0x7fc97c1537a0 :-1 s=OPENED pgs=689 gs=1 cs=1 l=1).handle_message read tag failed 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc981679640 1 --1- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 0x7fc97c1537a0 :-1 s=OPENED pgs=689 gs=1 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:33:19.777 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6789/0 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc97c08b8d0 legacy=0x7fc97c1537a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): picked mon.c con 0x7fc97c08d7e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): picked mon.a con 0x7fc94c004b80 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): picked mon.b con 0x7fc97c08cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc97c05d560 con 0x7fc94c004b80 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc97c07fc80 con 0x7fc97c08cd00 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc94c06c4a0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc969ffb640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.755+0000 7fc983904640 10 monclient(hunting): _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc981679640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08d7e0 legacy=0x7fc94c0042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.778 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc980e78640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc94c004b80 legacy=0x7fc94c046ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792462165 0 0) 0x7fc94c06c4a0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fc94c06a3a0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655658057 0 0) 0x7fc97c05d560 con 0x7fc94c004b80 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7fc94c06c4a0 con 0x7fc94c004b80 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc981e7a640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.178:6789/0 conn(0x7fc97c08cd00 legacy=0x7fc94c068840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 799345567 0 0) 0x7fc94c06a3a0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.178:6789/0 conn(0x7fc97c08cd00 legacy=0x7fc94c068840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.779 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6789/0 conn(0x7fc94c004b80 legacy=0x7fc94c046ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 monclient: found mon.c 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+,osdmap=0}) -- 0x7fc94c06d8c0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0008e0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:19.780 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c05d560 con 0x7fc97c08d7e0 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fc96c005050 con 0x7fc97c08d7e0 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:19.781 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: dump: 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.782 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc96c005320 con 0x7fc97c08d7e0 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc96b7fe640 4 set_mon_vals no callback set 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fc96c0054d0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 6 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7fc96c04c7b0 con 0x7fc97c08d7e0 2026-02-21T22:33:19.783 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc980e78640 10 monclient: get_auth_request con 0x7fc97c08b8d0 auth_method 0 2026-02-21T22:33:19.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fc96c017a30 con 0x7fc97c08d7e0 2026-02-21T22:33:19.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:19.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.759+0000 7fc969ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:19.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.763+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mon.2 v1:10.20.193.102:6790/0 8 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fc96c01f150 con 0x7fc97c08d7e0 2026-02-21T22:33:19.784 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.763+0000 7fc969ffb640 10 monclient: handle_mon_command_ack 1 not found 2026-02-21T22:33:19.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc97c081bb0 con 0x7fc97c08b8d0 2026-02-21T22:33:19.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc969ffb640 1 -- 10.20.193.102:0/621684944 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (unknown 2057319476 0 0) 0x7fc970003380 con 0x7fc97c08b8d0 2026-02-21T22:33:19.849 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.178:6800/2549949439 conn(0x7fc97c08b8d0 legacy=0x7fc94c0c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 10 monclient: shutdown 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 >> v1:10.20.193.102:6790/0 conn(0x7fc97c08d7e0 legacy=0x7fc94c0042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 shutdown_connections 2026-02-21T22:33:19.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 >> 10.20.193.102:0/621684944 conn(0x7fc97c091a50 msgr2=0x7fc97c05c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 shutdown_connections 2026-02-21T22:33:19.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.847+0000 7fc983904640 1 -- 10.20.193.102:0/621684944 wait complete. 2026-02-21T22:33:19.859 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T22:33:19.860 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:19.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: get_monmap_and_config 2026-02-21T22:33:19.921 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: monmap: 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:19.922294+0000 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:19.922294+0000 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:19.922 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:19.923 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 1 Processor -- start 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 1 -- start start 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: init 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-a con 0x7f9fdc091310 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-c con 0x7f9fdc0916e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-b con 0x7f9fdc185480 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.928 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc056390 con 0x7f9fdc091310 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc057430 con 0x7f9fdc0916e0 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc092560 con 0x7f9fdc185480 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fe18bc640 10 monclient(hunting): authenticate will time out at 2209.158628s 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fdaffd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9fdc091310 0x7f9fdc0584b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:37994/0 (socket says 10.20.193.102:37994) 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.919+0000 7f9fdaffd640 1 -- 10.20.193.102:0/4291303721 learned_addr learned my addr 10.20.193.102:0/4291303721 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fdaffd640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.102:6789/0 conn(0x7f9fdc091310 legacy=0x7f9fdc0584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fdb7fe640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.102:6790/0 conn(0x7f9fdc185480 legacy=0x7f9fdc1a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081539762 0 0) 0x7f9fdc056390 con 0x7f9fdc091310 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.929 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): my global_id is 34944 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fc8003620 con 0x7f9fdc091310 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630749851 0 0) 0x7f9fdc092560 con 0x7f9fdc185480 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient(hunting): my global_id is 94427 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fdc056390 con 0x7f9fdc185480 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fda7fc640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.178:6789/0 conn(0x7f9fdc0916e0 legacy=0x7f9fdc183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.930 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209621153 0 0) 0x7f9fdc056390 con 0x7f9fdc185480 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.178:6789/0 conn(0x7f9fdc0916e0 legacy=0x7f9fdc183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.102:6789/0 conn(0x7f9fdc091310 legacy=0x7f9fdc0584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 monclient: found mon.noname-b 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc091b20 con 0x7f9fdc185480 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9fcc0045a0 con 0x7f9fdc185480 2026-02-21T22:33:19.931 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: dump: 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.932 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 5 monclient: authenticate success, global_id 94427 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9fcc004870 con 0x7f9fdc185480 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 1 -- 10.20.193.102:0/4291303721 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9fcc004c20 con 0x7f9fdc185480 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fd9ffb640 10 monclient: dump: 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.933 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 4 set_mon_vals no callback set 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: shutdown 2026-02-21T22:33:19.934 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- 10.20.193.102:0/4291303721 >> v1:10.20.193.102:6790/0 conn(0x7f9fdc185480 legacy=0x7f9fdc1a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- 10.20.193.102:0/4291303721 shutdown_connections 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- 10.20.193.102:0/4291303721 >> 10.20.193.102:0/4291303721 conn(0x7f9fdc05e520 msgr2=0x7f9fdc095ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- 10.20.193.102:0/4291303721 shutdown_connections 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- 10.20.193.102:0/4291303721 wait complete. 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: build_initial_monmap 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: monmap: 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:19.935 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 Processor -- start 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- start start 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: init 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:19.936 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-b con 0x7f9fdc091310 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-c con 0x7f9fdc0916e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient(hunting): picked mon.noname-a con 0x7f9fdc185480 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc12b960 con 0x7f9fdc185480 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc112bb0 con 0x7f9fdc091310 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc111d40 con 0x7f9fdc0916e0 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fe18bc640 10 monclient(hunting): authenticate will time out at 2209.163233s 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fda7fc640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9fdc0916e0 0x7f9fdc172610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59226/0 (socket says 10.20.193.102:59226) 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fda7fc640 1 -- 10.20.193.102:0/878041955 learned_addr learned my addr 10.20.193.102:0/878041955 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:19.937 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.923+0000 7f9fdb7fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9fdc185480 0x7f9fdc173d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38004/0 (socket says 10.20.193.102:38004) 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fda7fc640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.102:6790/0 conn(0x7f9fdc0916e0 legacy=0x7f9fdc172610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fdb7fe640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.102:6789/0 conn(0x7f9fdc185480 legacy=0x7f9fdc173d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811603746 0 0) 0x7f9fdc12b960 con 0x7f9fdc185480 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): my global_id is 34947 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fac003620 con 0x7f9fdc185480 2026-02-21T22:33:19.943 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985995755 0 0) 0x7f9fdc111d40 con 0x7f9fdc0916e0 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): my global_id is 94430 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fdc12b960 con 0x7f9fdc0916e0 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fdaffd640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.178:6789/0 conn(0x7f9fdc091310 legacy=0x7f9fdc12b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3615712657 0 0) 0x7f9fac003620 con 0x7f9fdc185480 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9fdc111d40 con 0x7f9fdc185480 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9fcc003fa0 con 0x7f9fdc185480 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:19.944 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): dump: 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.945 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1876744007 0 0) 0x7f9fdc12b960 con 0x7f9fdc0916e0 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9fac003620 con 0x7f9fdc0916e0 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9fc4003420 con 0x7f9fdc0916e0 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): dump: 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.946 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618579947 0 0) 0x7f9fdc112bb0 con 0x7f9fdc091310 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient(hunting): my global_id is 134155 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fdc12b960 con 0x7f9fdc091310 2026-02-21T22:33:19.947 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3063393770 0 0) 0x7f9fdc111d40 con 0x7f9fdc185480 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.102:6790/0 conn(0x7f9fdc0916e0 legacy=0x7f9fdc172610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.178:6789/0 conn(0x7f9fdc091310 legacy=0x7f9fdc12b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 monclient: found mon.a 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc110850 con 0x7f9fdc185480 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: _finish_auth 0 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 5 monclient: authenticate success, global_id 34947 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 10 monclient: _renew_subs 2026-02-21T22:33:19.948 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc112310 con 0x7f9fdc185480 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 10 monclient: _renew_subs 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9fdc111580 con 0x7f9fdc185480 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9fcc004870 con 0x7f9fdc185480 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9fcc005330 con 0x7f9fdc185480 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: dump: 2026-02-21T22:33:19.949 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fd97fa640 4 set_mon_vals no callback set 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:19.950 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f9fdc095010 con 0x7f9fdc185480 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9fcc0055d0 con 0x7f9fdc185480 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f9fcc04d780 con 0x7f9fdc185480 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6800/4138446215 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fcc005180 con 0x7f9fac0624b0 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f9fcc04dac0 con 0x7f9fdc185480 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fb3fff640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fda7fc640 10 monclient: get_auth_request con 0x7f9fac0624b0 auth_method 0 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.927+0000 7f9fdaffd640 10 monclient: get_auth_request con 0x7f9fac03d830 auth_method 0 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.931+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== osd.1 v1:10.20.193.102:6800/4138446215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f9fcc005180 con 0x7f9fac0624b0 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 --> v1:10.20.193.102:6800/4138446215 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9fdc1a71d0 con 0x7f9fac0624b0 2026-02-21T22:33:19.951 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fb3fff640 1 -- 10.20.193.102:0/878041955 <== osd.1 v1:10.20.193.102:6800/4138446215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f9fdc1a71d0 con 0x7f9fac0624b0 2026-02-21T22:33:19.952 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:19.952 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:19.952 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.102:6800/4138446215 conn(0x7f9fac0624b0 legacy=0x7f9fac0828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9fac03d830 legacy=0x7f9fac05dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 10 monclient: shutdown 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 >> v1:10.20.193.102:6789/0 conn(0x7f9fdc185480 legacy=0x7f9fdc173d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:19.954 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fdb7fe640 1 -- 10.20.193.102:0/878041955 reap_dead start 2026-02-21T22:33:19.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 shutdown_connections 2026-02-21T22:33:19.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 >> 10.20.193.102:0/878041955 conn(0x7f9fdc05e520 msgr2=0x7f9fdc08b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:19.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 shutdown_connections 2026-02-21T22:33:19.955 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:19.939+0000 7f9fe18bc640 1 -- 10.20.193.102:0/878041955 wait complete. 2026-02-21T22:33:19.955 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 10 monclient: monmap: 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.015895+0000 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.015895+0000 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.015 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.016 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.016 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 1 Processor -- start 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 1 -- start start 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.011+0000 7f9cd4238640 10 monclient: init 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-b con 0x7f9ccc08ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-c con 0x7f9ccc08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-a con 0x7f9ccc08d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc086a10 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc08b130 con 0x7f9ccc08b7f0 2026-02-21T22:33:20.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc08bd60 con 0x7f9ccc08ce00 2026-02-21T22:33:20.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient(hunting): authenticate will time out at 2209.252200s 2026-02-21T22:33:20.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd1fad640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08ce00 0x7f9ccc08d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59236/0 (socket says 10.20.193.102:59236) 2026-02-21T22:33:20.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd1fad640 1 -- 10.20.193.102:0/2457089593 learned_addr learned my addr 10.20.193.102:0/2457089593 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd1fad640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08ce00 legacy=0x7f9ccc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd27ae640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.102:6789/0 conn(0x7f9ccc08d8e0 legacy=0x7f9ccc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455285153 0 0) 0x7f9ccc08bd60 con 0x7f9ccc08ce00 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): my global_id is 94433 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb4003620 con 0x7f9ccc08ce00 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd17ac640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.178:6789/0 conn(0x7f9ccc08b7f0 legacy=0x7f9ccc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1605301119 0 0) 0x7f9ccc086a10 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient(hunting): my global_id is 34950 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ccc08bd60 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584339037 0 0) 0x7f9cb4003620 con 0x7f9ccc08ce00 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.178:6789/0 conn(0x7f9ccc08b7f0 legacy=0x7f9ccc181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.102:6789/0 conn(0x7f9ccc08d8e0 legacy=0x7f9ccc1a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc08b370 con 0x7f9ccc08ce00 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9cbc003110 con 0x7f9ccc08ce00 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: dump: 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.022 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 5 monclient: authenticate success, global_id 94433 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9cbc0033e0 con 0x7f9ccc08ce00 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 1 -- 10.20.193.102:0/2457089593 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9cbc004b50 con 0x7f9ccc08ce00 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd0fab640 10 monclient: dump: 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.023 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 4 set_mon_vals no callback set 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: shutdown 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- 10.20.193.102:0/2457089593 >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08ce00 legacy=0x7f9ccc08d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- 10.20.193.102:0/2457089593 shutdown_connections 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- 10.20.193.102:0/2457089593 >> 10.20.193.102:0/2457089593 conn(0x7f9ccc091a50 msgr2=0x7f9ccc091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- 10.20.193.102:0/2457089593 shutdown_connections 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- 10.20.193.102:0/2457089593 wait complete. 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: monmap: 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.024 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 Processor -- start 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 1 -- start start 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.015+0000 7f9cd4238640 10 monclient: init 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-b con 0x7f9ccc08b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-a con 0x7f9ccc08ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient(hunting): picked mon.noname-c con 0x7f9ccc08d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc08f040 con 0x7f9ccc08ce00 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc083280 con 0x7f9ccc08b7f0 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ccc083560 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd4238640 10 monclient(hunting): authenticate will time out at 2209.256164s 2026-02-21T22:33:20.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd17ac640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f9ccc08ce00 0x7f9ccc080290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38030/0 (socket says 10.20.193.102:38030) 2026-02-21T22:33:20.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd17ac640 1 -- 10.20.193.102:0/2773670145 learned_addr learned my addr 10.20.193.102:0/2773670145 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd27ae640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08d8e0 0x7f9ccc1a2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59252/0 (socket says 10.20.193.102:59252) 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd17ac640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.102:6789/0 conn(0x7f9ccc08ce00 legacy=0x7f9ccc080290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd27ae640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08d8e0 legacy=0x7f9ccc1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3229331096 0 0) 0x7f9ccc083560 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): my global_id is 94436 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c9c003620 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265279607 0 0) 0x7f9ccc08f040 con 0x7f9ccc08ce00 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 10 monclient(hunting): my global_id is 34953 2026-02-21T22:33:20.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ccc083560 con 0x7f9ccc08ce00 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.019+0000 7f9cd1fad640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.178:6789/0 conn(0x7f9ccc08b7f0 legacy=0x7f9ccc07fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3008566487 0 0) 0x7f9c9c003620 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ccc08f040 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230418837 0 0) 0x7f9ccc083280 con 0x7f9ccc08b7f0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): my global_id is 134161 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c9c003620 con 0x7f9ccc08b7f0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3533327584 0 0) 0x7f9ccc083560 con 0x7f9ccc08ce00 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ccc083280 con 0x7f9ccc08ce00 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9cc8004900 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): dump: 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.029 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9cc0003380 con 0x7f9ccc08ce00 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient(hunting): dump: 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.030 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4029002909 0 0) 0x7f9ccc08f040 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.178:6789/0 conn(0x7f9ccc08b7f0 legacy=0x7f9ccc07fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.102:6789/0 conn(0x7f9ccc08ce00 legacy=0x7f9ccc080290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 monclient: found mon.c 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc0809a0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 5 monclient: authenticate success, global_id 94436 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _renew_subs 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc05e3c0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _renew_subs 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9ccc0814e0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9cc8004ed0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.031 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f9cc8005230 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: dump: 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cbbfff640 4 set_mon_vals no callback set 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ccc0850a0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f9cc80054c0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f9cc804d9d0 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cd1fad640 10 monclient: get_auth_request con 0x7f9c9c03d8f0 auth_method 0 2026-02-21T22:33:20.032 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f9cc8018b30 con 0x7f9ccc08d8e0 2026-02-21T22:33:20.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.033 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.023+0000 7f9cba7fc640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:20.114 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f9ccc083960 con 0x7f9c9c03d8f0 2026-02-21T22:33:20.115 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cba7fc640 1 -- 10.20.193.102:0/2773670145 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (unknown 489605429 0 0) 0x7f9cbc003490 con 0x7f9c9c03d8f0 2026-02-21T22:33:20.115 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.178:6800/2549949439 conn(0x7f9c9c03d8f0 legacy=0x7f9c9c05dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 10 monclient: shutdown 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 >> v1:10.20.193.102:6790/0 conn(0x7f9ccc08d8e0 legacy=0x7f9ccc1a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 shutdown_connections 2026-02-21T22:33:20.116 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 >> 10.20.193.102:0/2773670145 conn(0x7f9ccc091a50 msgr2=0x7f9ccc05c530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 shutdown_connections 2026-02-21T22:33:20.117 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.111+0000 7f9cd4238640 1 -- 10.20.193.102:0/2773670145 wait complete. 2026-02-21T22:33:20.125 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T22:33:20.125 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:20.196 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: monmap: 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.197317+0000 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.197317+0000 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.197 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.198 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.198 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 1 Processor -- start 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 1 -- start start 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: init 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:20.208 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-b con 0x7f676013fb40 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-a con 0x7f6760163ec0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-c con 0x7f67601859c0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6760090280 con 0x7f6760163ec0 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6760054480 con 0x7f67601859c0 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f676008e280 con 0x7f676013fb40 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f67653a5640 10 monclient(hunting): authenticate will time out at 2209.433720s 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f675effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 0x7f676013ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59258/0 (socket says 10.20.193.102:59258) 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f675effd640 1 -- 10.20.193.102:0/3710869368 learned_addr learned my addr 10.20.193.102:0/3710869368 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f675effd640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 legacy=0x7f676013ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.209 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.195+0000 7f675e7fc640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.102:6789/0 conn(0x7f6760163ec0 legacy=0x7f67601842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820888345 0 0) 0x7f676008e280 con 0x7f676013fb40 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): my global_id is 94439 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f674c003620 con 0x7f676013fb40 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330396922 0 0) 0x7f6760090280 con 0x7f6760163ec0 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.210 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient(hunting): my global_id is 34956 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f676008e280 con 0x7f6760163ec0 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 329569753 0 0) 0x7f674c003620 con 0x7f676013fb40 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675f7fe640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.178:6789/0 conn(0x7f67601859c0 legacy=0x7f67601a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.178:6789/0 conn(0x7f67601859c0 legacy=0x7f67601a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.102:6789/0 conn(0x7f6760163ec0 legacy=0x7f67601842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.211 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6760058890 con 0x7f676013fb40 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 5 monclient: authenticate success, global_id 94439 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6754003100 con 0x7f676013fb40 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: dump: 2026-02-21T22:33:20.212 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.213 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67540033d0 con 0x7f676013fb40 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 4 set_mon_vals no callback set 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 1 -- 10.20.193.102:0/3710869368 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6754004b50 con 0x7f676013fb40 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675dffb640 10 monclient: dump: 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.214 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: shutdown 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- 10.20.193.102:0/3710869368 >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 legacy=0x7f676013ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.215 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- 10.20.193.102:0/3710869368 shutdown_connections 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- 10.20.193.102:0/3710869368 >> 10.20.193.102:0/3710869368 conn(0x7f6760056d10 msgr2=0x7f6760093cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- 10.20.193.102:0/3710869368 shutdown_connections 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- 10.20.193.102:0/3710869368 wait complete. 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: monmap: 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.216 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 Processor -- start 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- start start 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: init 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:33:20.217 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-c con 0x7f676013fb40 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-b con 0x7f6760163ec0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient(hunting): picked mon.noname-a con 0x7f67601859c0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6760145250 con 0x7f67601859c0 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6760143d60 con 0x7f6760163ec0 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67601460c0 con 0x7f676013fb40 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f67653a5640 10 monclient(hunting): authenticate will time out at 2209.438043s 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 0x7f6760161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59260/0 (socket says 10.20.193.102:59260) 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.199+0000 7f675effd640 1 -- 10.20.193.102:0/2404453224 learned_addr learned my addr 10.20.193.102:0/2404453224 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675effd640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 legacy=0x7f6760161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.218 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675f7fe640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.102:6789/0 conn(0x7f67601859c0 legacy=0x7f676012da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155023826 0 0) 0x7f67601460c0 con 0x7f676013fb40 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): my global_id is 94442 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f672c003620 con 0x7f676013fb40 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326912663 0 0) 0x7f6760145250 con 0x7f67601859c0 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.219 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): my global_id is 34959 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67601460c0 con 0x7f67601859c0 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675e7fc640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.178:6789/0 conn(0x7f6760163ec0 legacy=0x7f676012c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2818448487 0 0) 0x7f672c003620 con 0x7f676013fb40 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6760145250 con 0x7f676013fb40 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67540048e0 con 0x7f676013fb40 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): dump: 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.220 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3735000037 0 0) 0x7f67601460c0 con 0x7f67601859c0 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f672c003620 con 0x7f67601859c0 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6750004890 con 0x7f67601859c0 2026-02-21T22:33:20.221 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): dump: 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.222 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266189361 0 0) 0x7f6760143d60 con 0x7f6760163ec0 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient(hunting): my global_id is 134164 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67601460c0 con 0x7f6760163ec0 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1129517566 0 0) 0x7f6760145250 con 0x7f676013fb40 2026-02-21T22:33:20.223 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.178:6789/0 conn(0x7f6760163ec0 legacy=0x7f676012c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.102:6789/0 conn(0x7f67601859c0 legacy=0x7f676012da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 monclient: found mon.c 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6760145820 con 0x7f676013fb40 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 5 monclient: authenticate success, global_id 94442 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 10 monclient: _renew_subs 2026-02-21T22:33:20.224 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6760144b20 con 0x7f676013fb40 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 10 monclient: _renew_subs 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6760161e30 con 0x7f676013fb40 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f67540033b0 con 0x7f676013fb40 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f67540052a0 con 0x7f676013fb40 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.225 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: dump: 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.226 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675d7fa640 4 set_mon_vals no callback set 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f6760163d00 con 0x7f676013fb40 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f6754011930 con 0x7f676013fb40 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f675404c510 con 0x7f676013fb40 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6808/579535315 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67540050f0 con 0x7f672c0624b0 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f675404c850 con 0x7f676013fb40 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f673f7fe640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675e7fc640 10 monclient: get_auth_request con 0x7f672c03d880 auth_method 0 2026-02-21T22:33:20.227 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.203+0000 7f675f7fe640 10 monclient: get_auth_request con 0x7f672c0624b0 auth_method 0 2026-02-21T22:33:20.228 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.207+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== osd.2 v1:10.20.193.102:6808/579535315 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f67540050f0 con 0x7f672c0624b0 2026-02-21T22:33:20.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 --> v1:10.20.193.102:6808/579535315 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6760162170 con 0x7f672c0624b0 2026-02-21T22:33:20.229 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f673f7fe640 1 -- 10.20.193.102:0/2404453224 <== osd.2 v1:10.20.193.102:6808/579535315 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f6760162170 con 0x7f672c0624b0 2026-02-21T22:33:20.229 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:20.229 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:20.229 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:20.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.102:6808/579535315 conn(0x7f672c0624b0 legacy=0x7f672c0828c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.178:6800/2549949439 conn(0x7f672c03d880 legacy=0x7f672c05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 10 monclient: shutdown 2026-02-21T22:33:20.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.231 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 >> v1:10.20.193.102:6790/0 conn(0x7f676013fb40 legacy=0x7f6760161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f675f7fe640 1 -- 10.20.193.102:0/2404453224 reap_dead start 2026-02-21T22:33:20.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 shutdown_connections 2026-02-21T22:33:20.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 >> 10.20.193.102:0/2404453224 conn(0x7f6760056d10 msgr2=0x7f67601842a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 shutdown_connections 2026-02-21T22:33:20.232 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.215+0000 7f67653a5640 1 -- 10.20.193.102:0/2404453224 wait complete. 2026-02-21T22:33:20.232 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T22:33:20.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.287+0000 7f4e954bf640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.287+0000 7f4e954bf640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.287+0000 7f4e954bf640 10 monclient: monmap: 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.292305+0000 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.292305+0000 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.292 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.293 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.293 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.287+0000 7f4e954bf640 1 Processor -- start 2026-02-21T22:33:20.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- start start 2026-02-21T22:33:20.293 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient: init 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4e9008c4e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-a con 0x7f4e9008d410 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4e9008d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9008bd50 con 0x7f4e9008d410 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9008c8b0 con 0x7f4e9008d890 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e90083560 con 0x7f4e9008c4e0 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.298 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient(hunting): authenticate will time out at 2209.528493s 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4e9008c4e0 0x7f4e9008cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59262/0 (socket says 10.20.193.102:59262) 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8effd640 1 -- 10.20.193.102:0/883898452 learned_addr learned my addr 10.20.193.102:0/883898452 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8effd640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.102:6790/0 conn(0x7f4e9008c4e0 legacy=0x7f4e9008cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8e7fc640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.102:6789/0 conn(0x7f4e9008d410 legacy=0x7f4e90188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477354269 0 0) 0x7f4e84002c60 con 0x7f4e9008c4e0 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): my global_id is 94445 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e7c003620 con 0x7f4e9008c4e0 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728608939 0 0) 0x7f4e9008bd50 con 0x7f4e9008d410 2026-02-21T22:33:20.299 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient(hunting): my global_id is 34962 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e84002c60 con 0x7f4e9008d410 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8f7fe640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.178:6789/0 conn(0x7f4e9008d890 legacy=0x7f4e901aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3609689494 0 0) 0x7f4e7c003620 con 0x7f4e9008c4e0 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.178:6789/0 conn(0x7f4e9008d890 legacy=0x7f4e901aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.102:6789/0 conn(0x7f4e9008d410 legacy=0x7f4e90188a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.300 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90084bc0 con 0x7f4e9008c4e0 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e840032e0 con 0x7f4e9008c4e0 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: dump: 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.301 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 5 monclient: authenticate success, global_id 94445 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e840049a0 con 0x7f4e9008c4e0 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 1 -- 10.20.193.102:0/883898452 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e84004d30 con 0x7f4e9008c4e0 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.302 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e8dffb640 10 monclient: dump: 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.303 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 4 set_mon_vals no callback set 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 10 monclient: shutdown 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- 10.20.193.102:0/883898452 >> v1:10.20.193.102:6790/0 conn(0x7f4e9008c4e0 legacy=0x7f4e9008cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- 10.20.193.102:0/883898452 shutdown_connections 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- 10.20.193.102:0/883898452 >> 10.20.193.102:0/883898452 conn(0x7f4e90091a50 msgr2=0x7f4e90091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- 10.20.193.102:0/883898452 shutdown_connections 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.291+0000 7f4e954bf640 1 -- 10.20.193.102:0/883898452 wait complete. 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: monmap: 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.304 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 Processor -- start 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- start start 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: init 2026-02-21T22:33:20.305 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-a con 0x7f4e9008c4e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4e9008d410 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4e9008d890 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9005d780 con 0x7f4e9008c4e0 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9005da60 con 0x7f4e9008d890 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e90087c90 con 0x7f4e9008d410 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient(hunting): authenticate will time out at 2209.532624s 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8e7fc640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4e9008d410 0x7f4e9021f8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59266/0 (socket says 10.20.193.102:59266) 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8effd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f4e9008c4e0 0x7f4e901b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38072/0 (socket says 10.20.193.102:38072) 2026-02-21T22:33:20.306 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8e7fc640 1 -- 10.20.193.102:0/2074880531 learned_addr learned my addr 10.20.193.102:0/2074880531 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8e7fc640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.102:6790/0 conn(0x7f4e9008d410 legacy=0x7f4e9021f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.310 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8effd640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.102:6789/0 conn(0x7f4e9008c4e0 legacy=0x7f4e901b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549775559 0 0) 0x7f4e90087c90 con 0x7f4e9008d410 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): my global_id is 94448 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e5c003620 con 0x7f4e9008d410 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118396357 0 0) 0x7f4e9005d780 con 0x7f4e9008c4e0 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): my global_id is 34965 2026-02-21T22:33:20.311 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e90087c90 con 0x7f4e9008c4e0 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8f7fe640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.178:6789/0 conn(0x7f4e9008d890 legacy=0x7f4e90241000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 442023800 0 0) 0x7f4e5c003620 con 0x7f4e9008d410 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e9005d780 con 0x7f4e9008d410 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e78003490 con 0x7f4e9008d410 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): dump: 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.312 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3772363149 0 0) 0x7f4e90087c90 con 0x7f4e9008c4e0 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e5c003620 con 0x7f4e9008c4e0 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87470842 0 0) 0x7f4e9005da60 con 0x7f4e9008d890 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.313 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): my global_id is 134170 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e90087c90 con 0x7f4e9008d890 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e840030c0 con 0x7f4e9008c4e0 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient(hunting): dump: 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.314 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2486468044 0 0) 0x7f4e9005d780 con 0x7f4e9008d410 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.178:6789/0 conn(0x7f4e9008d890 legacy=0x7f4e90241000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.102:6789/0 conn(0x7f4e9008c4e0 legacy=0x7f4e901b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 monclient: found mon.c 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.315 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90086020 con 0x7f4e9008d410 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 5 monclient: authenticate success, global_id 94448 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _renew_subs 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9005c260 con 0x7f4e9008d410 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _renew_subs 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4e9005c530 con 0x7f4e9008d410 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e78003790 con 0x7f4e9008d410 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.316 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e78003de0 con 0x7f4e9008d410 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e8d7fa640 4 set_mon_vals no callback set 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e6f7fe640 10 monclient: dump: 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.317 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.295+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9005e2e0 con 0x7f4e9008d410 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4e780040a0 con 0x7f4e9008d410 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f4e7804c640 con 0x7f4e9008d410 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e8effd640 10 monclient: get_auth_request con 0x7f4e5c03d880 auth_method 0 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f4e78017740 con 0x7f4e9008d410 2026-02-21T22:33:20.318 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e6f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.319 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.299+0000 7f4e6f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:20.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f4e90062ce0 con 0x7f4e5c03d880 2026-02-21T22:33:20.385 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e6f7fe640 1 -- 10.20.193.102:0/2074880531 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (unknown 3049115190 0 0) 0x7f4e90062ce0 con 0x7f4e5c03d880 2026-02-21T22:33:20.385 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.178:6800/2549949439 conn(0x7f4e5c03d880 legacy=0x7f4e5c05dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 10 monclient: shutdown 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 >> v1:10.20.193.102:6790/0 conn(0x7f4e9008d410 legacy=0x7f4e9021f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 shutdown_connections 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 >> 10.20.193.102:0/2074880531 conn(0x7f4e90091a50 msgr2=0x7f4e900881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 shutdown_connections 2026-02-21T22:33:20.387 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.383+0000 7f4e954bf640 1 -- 10.20.193.102:0/2074880531 wait complete. 2026-02-21T22:33:20.396 INFO:tasks.ceph:Scrubbing osd.3 2026-02-21T22:33:20.396 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:20.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: monmap: 2026-02-21T22:33:20.465 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.465 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.466461+0000 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.466461+0000 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.466 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 1 Processor -- start 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 1 -- start start 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: init 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-a con 0x7f39e0091310 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-c con 0x7f39e00916e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-b con 0x7f39e0185480 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e0056390 con 0x7f39e0091310 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e0057430 con 0x7f39e00916e0 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e0092560 con 0x7f39e0185480 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39e6a05640 10 monclient(hunting): authenticate will time out at 2209.702806s 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39dffff640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f39e0091310 0x7f39e00584b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38082/0 (socket says 10.20.193.102:38082) 2026-02-21T22:33:20.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.463+0000 7f39dffff640 1 -- 10.20.193.102:0/2439226369 learned_addr learned my addr 10.20.193.102:0/2439226369 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e4f7b640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.102:6790/0 conn(0x7f39e0185480 legacy=0x7f39e01a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39dffff640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.102:6789/0 conn(0x7f39e0091310 legacy=0x7f39e00584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699420248 0 0) 0x7f39e0092560 con 0x7f39e0185480 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): my global_id is 94451 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39c4003620 con 0x7f39e0185480 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561270426 0 0) 0x7f39e0056390 con 0x7f39e0091310 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient(hunting): my global_id is 34968 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e0092560 con 0x7f39e0091310 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39df7fe640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.178:6789/0 conn(0x7f39e00916e0 legacy=0x7f39e0183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1108462063 0 0) 0x7f39c4003620 con 0x7f39e0185480 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.178:6789/0 conn(0x7f39e00916e0 legacy=0x7f39e0183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.102:6789/0 conn(0x7f39e0091310 legacy=0x7f39e00584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.470 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e0091b20 con 0x7f39e0185480 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 5 monclient: authenticate success, global_id 94451 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39d4004670 con 0x7f39e0185480 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: dump: 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f39d4004940 con 0x7f39e0185480 2026-02-21T22:33:20.471 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 1 -- 10.20.193.102:0/2439226369 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39d4004dd0 con 0x7f39e0185480 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39deffd640 10 monclient: dump: 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 4 set_mon_vals no callback set 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: shutdown 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- 10.20.193.102:0/2439226369 >> v1:10.20.193.102:6790/0 conn(0x7f39e0185480 legacy=0x7f39e01a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- 10.20.193.102:0/2439226369 shutdown_connections 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- 10.20.193.102:0/2439226369 >> 10.20.193.102:0/2439226369 conn(0x7f39e005e520 msgr2=0x7f39e0095ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- 10.20.193.102:0/2439226369 shutdown_connections 2026-02-21T22:33:20.472 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- 10.20.193.102:0/2439226369 wait complete. 2026-02-21T22:33:20.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.474 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: monmap: 2026-02-21T22:33:20.474 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.474 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.474 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 Processor -- start 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- start start 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: init 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-b con 0x7f39e0091310 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-a con 0x7f39e00916e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient(hunting): picked mon.noname-c con 0x7f39e0185480 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e008b730 con 0x7f39e00916e0 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e01891c0 con 0x7f39e0091310 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e01a2230 con 0x7f39e0185480 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e6a05640 10 monclient(hunting): authenticate will time out at 2209.707282s 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e4f7b640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f39e0185480 0x7f39e01740b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59292/0 (socket says 10.20.193.102:59292) 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.467+0000 7f39e4f7b640 1 -- 10.20.193.102:0/1237605921 learned_addr learned my addr 10.20.193.102:0/1237605921 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.475 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e4f7b640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.102:6790/0 conn(0x7f39e0185480 legacy=0x7f39e01740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.102:6789/0 conn(0x7f39e00916e0 legacy=0x7f39e0172990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403681436 0 0) 0x7f39e01a2230 con 0x7f39e0185480 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): my global_id is 94454 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b0003620 con 0x7f39e0185480 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035991461 0 0) 0x7f39e0057430 con 0x7f39e00916e0 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): my global_id is 34971 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e01a2230 con 0x7f39e00916e0 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3179440767 0 0) 0x7f39b0003620 con 0x7f39e0185480 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39e0057430 con 0x7f39e0185480 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39d40040d0 con 0x7f39e0185480 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): dump: 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.476 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4037726990 0 0) 0x7f39e01a2230 con 0x7f39e00916e0 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f39b0003620 con 0x7f39e00916e0 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39d0003380 con 0x7f39e00916e0 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient(hunting): dump: 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1262424121 0 0) 0x7f39e0057430 con 0x7f39e0185480 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dffff640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6789/0 conn(0x7f39e0091310 legacy=0x7f39e01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.477 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6789/0 conn(0x7f39e0091310 legacy=0x7f39e01a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.102:6789/0 conn(0x7f39e00916e0 legacy=0x7f39e0172990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 monclient: found mon.c 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01894a0 con 0x7f39e0185480 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 5 monclient: authenticate success, global_id 94454 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 10 monclient: _renew_subs 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f39e00591d0 con 0x7f39e0185480 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 10 monclient: _renew_subs 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f39e0187f70 con 0x7f39e0185480 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f39d4004e50 con 0x7f39e0185480 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f39d40053f0 con 0x7f39e0185480 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39de7fc640 4 set_mon_vals no callback set 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: dump: 2026-02-21T22:33:20.478 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f39e0095010 con 0x7f39e0185480 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f39d40056c0 con 0x7f39e0185480 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f39d404d8b0 con 0x7f39e0185480 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.178:6805/653951931 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39d4004920 con 0x7f39b0062280 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f39d404dca0 con 0x7f39e0185480 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:20.479 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dffff640 10 monclient: get_auth_request con 0x7f39b003d650 auth_method 0 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 10 monclient: get_auth_request con 0x7f39b0062280 auth_method 0 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0062280 legacy=0x7f39b0082690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0062280 legacy=0x7f39b0082690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 1 --1- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0062280 0x7f39b0082690 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 gs=5 cs=0 l=1).handle_connect_reply_1 read connect reply failed 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 1 --1- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0062280 0x7f39b0082690 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 gs=5 cs=0 l=1).fault on lossy channel, failing 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0062280 legacy=0x7f39b0082690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.178:6805/653951931 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39d00031d0 con 0x7f39b0083fb0 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _renew_subs 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=832}) -- 0x7f39e0057430 con 0x7f39e0185480 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.471+0000 7f39df7fe640 10 monclient: get_auth_request con 0x7f39b0083fb0 auth_method 0 2026-02-21T22:33:20.480 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.479+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== osd.3 v1:10.20.193.178:6805/653951931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f39d00031d0 con 0x7f39b0083fb0 2026-02-21T22:33:20.487 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 --> v1:10.20.193.178:6805/653951931 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f39e01a71d0 con 0x7f39b0083fb0 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39dcff9640 1 -- 10.20.193.102:0/1237605921 <== osd.3 v1:10.20.193.178:6805/653951931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f39e01a71d0 con 0x7f39b0083fb0 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6805/653951931 conn(0x7f39b0083fb0 legacy=0x7f39b00a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.488 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.178:6800/2549949439 conn(0x7f39b003d650 legacy=0x7f39b005db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 10 monclient: shutdown 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 >> v1:10.20.193.102:6790/0 conn(0x7f39e0185480 legacy=0x7f39e01740b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e4f7b640 1 -- 10.20.193.102:0/1237605921 reap_dead start 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 shutdown_connections 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 >> 10.20.193.102:0/1237605921 conn(0x7f39e005e520 msgr2=0x7f39e008cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 shutdown_connections 2026-02-21T22:33:20.489 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.487+0000 7f39e6a05640 1 -- 10.20.193.102:0/1237605921 wait complete. 2026-02-21T22:33:20.496 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: monmap: 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.558 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.557603+0000 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.557603+0000 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 1 Processor -- start 2026-02-21T22:33:20.559 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 1 -- start start 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: init 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-c con 0x7fdf1008c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-b con 0x7fdf1008cc80 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-a con 0x7fdf10183480 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1008d780 con 0x7fdf10183480 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf10085da0 con 0x7fdf1008c1a0 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf10085420 con 0x7fdf1008cc80 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.560 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf16128640 10 monclient(hunting): authenticate will time out at 2209.793543s 2026-02-21T22:33:20.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf14925640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 0x7fdf10181d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59308/0 (socket says 10.20.193.102:59308) 2026-02-21T22:33:20.561 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf14925640 1 -- 10.20.193.102:0/1821873647 learned_addr learned my addr 10.20.193.102:0/1821873647 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf14925640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 legacy=0x7fdf10181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdf15927640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.102:6789/0 conn(0x7fdf10183480 legacy=0x7fdf101a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739159773 0 0) 0x7fdf10085420 con 0x7fdf1008cc80 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdefffff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdefffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdefffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.555+0000 7fdefffff640 10 monclient(hunting): my global_id is 94457 2026-02-21T22:33:20.569 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdef8003620 con 0x7fdf1008cc80 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf15126640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.178:6789/0 conn(0x7fdf1008c1a0 legacy=0x7fdf1008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2848859330 0 0) 0x7fdf0c0041c0 con 0x7fdf10183480 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient(hunting): my global_id is 34974 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf10085420 con 0x7fdf10183480 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3444336879 0 0) 0x7fdef8003620 con 0x7fdf1008cc80 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.178:6789/0 conn(0x7fdf1008c1a0 legacy=0x7fdf1008c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.570 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.102:6789/0 conn(0x7fdf10183480 legacy=0x7fdf101a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf10086640 con 0x7fdf1008cc80 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf04003110 con 0x7fdf1008cc80 2026-02-21T22:33:20.571 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: dump: 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.572 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 5 monclient: authenticate success, global_id 94457 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf040033e0 con 0x7fdf1008cc80 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 1 -- 10.20.193.102:0/1821873647 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf04004b50 con 0x7fdf1008cc80 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdefffff640 10 monclient: dump: 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.573 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 4 set_mon_vals no callback set 2026-02-21T22:33:20.574 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: shutdown 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- 10.20.193.102:0/1821873647 >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 legacy=0x7fdf10181d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- 10.20.193.102:0/1821873647 shutdown_connections 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- 10.20.193.102:0/1821873647 >> 10.20.193.102:0/1821873647 conn(0x7fdf1008f6a0 msgr2=0x7fdf1008faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- 10.20.193.102:0/1821873647 shutdown_connections 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- 10.20.193.102:0/1821873647 wait complete. 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: monmap: 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.575 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 Processor -- start 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- start start 2026-02-21T22:33:20.576 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: init 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-b con 0x7fdf1008c1a0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-c con 0x7fdf1008cc80 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient(hunting): picked mon.noname-a con 0x7fdf10183480 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf100834e0 con 0x7fdf10183480 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf10088990 con 0x7fdf1008c1a0 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf10153880 con 0x7fdf1008cc80 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.577 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf16128640 10 monclient(hunting): authenticate will time out at 2209.798214s 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf14925640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 0x7fdf101744d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59320/0 (socket says 10.20.193.102:59320) 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.559+0000 7fdf14925640 1 -- 10.20.193.102:0/1516113575 learned_addr learned my addr 10.20.193.102:0/1516113575 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf14925640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 legacy=0x7fdf101744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf15927640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.102:6789/0 conn(0x7fdf10183480 legacy=0x7fdf10175bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346695182 0 0) 0x7fdf10153880 con 0x7fdf1008cc80 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): my global_id is 94460 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdee0003620 con 0x7fdf1008cc80 2026-02-21T22:33:20.578 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884007824 0 0) 0x7fdf100834e0 con 0x7fdf10183480 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): my global_id is 34977 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf10153880 con 0x7fdf10183480 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf15126640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.178:6789/0 conn(0x7fdf1008c1a0 legacy=0x7fdf100849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1462752225 0 0) 0x7fdee0003620 con 0x7fdf1008cc80 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf100834e0 con 0x7fdf1008cc80 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf040048e0 con 0x7fdf1008cc80 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.579 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): dump: 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.580 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 709578118 0 0) 0x7fdf10153880 con 0x7fdf10183480 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdee0003620 con 0x7fdf10183480 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf0c004070 con 0x7fdf10183480 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient(hunting): dump: 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.581 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2989559048 0 0) 0x7fdf100834e0 con 0x7fdf1008cc80 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.178:6789/0 conn(0x7fdf1008c1a0 legacy=0x7fdf100849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.102:6789/0 conn(0x7fdf10183480 legacy=0x7fdf10175bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.582 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 monclient: found mon.c 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1008a790 con 0x7fdf1008cc80 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 5 monclient: authenticate success, global_id 94460 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _renew_subs 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.583 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdf10083200 con 0x7fdf1008cc80 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _renew_subs 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fdf100852d0 con 0x7fdf1008cc80 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf04003390 con 0x7fdf1008cc80 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7fdf04005010 con 0x7fdf1008cc80 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 10 monclient: dump: 2026-02-21T22:33:20.584 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdeff7fe640 4 set_mon_vals no callback set 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.585 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf10082db0 con 0x7fdf1008cc80 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7fdf040052e0 con 0x7fdf1008cc80 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7fdf0404c5b0 con 0x7fdf1008cc80 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.563+0000 7fdf15126640 10 monclient: get_auth_request con 0x7fdee003d650 auth_method 0 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.567+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7fdf10082db0 con 0x7fdf1008cc80 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.567+0000 7fdefdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.586 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.567+0000 7fdefdffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:20.653 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fdf10062ce0 con 0x7fdee003d650 2026-02-21T22:33:20.654 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdefdffb640 1 -- 10.20.193.102:0/1516113575 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (unknown 3526570807 0 0) 0x7fdf10062ce0 con 0x7fdee003d650 2026-02-21T22:33:20.654 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 3 to deep-scrub 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.178:6800/2549949439 conn(0x7fdee003d650 legacy=0x7fdee005db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 10 monclient: shutdown 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 >> v1:10.20.193.102:6790/0 conn(0x7fdf1008cc80 legacy=0x7fdf101744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 shutdown_connections 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 >> 10.20.193.102:0/1516113575 conn(0x7fdf1008f6a0 msgr2=0x7fdf100574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.655 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 shutdown_connections 2026-02-21T22:33:20.656 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.651+0000 7fdf16128640 1 -- 10.20.193.102:0/1516113575 wait complete. 2026-02-21T22:33:20.664 INFO:tasks.ceph:Scrubbing osd.4 2026-02-21T22:33:20.664 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.733 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: monmap: 2026-02-21T22:33:20.733 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.733 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.733985+0000 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.733985+0000 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.734 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 1 Processor -- start 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 1 -- start start 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: init 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-c con 0x7f6ae4142a00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f6ae4092400 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.736 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-a con 0x7f6ae41434e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae40572b0 con 0x7f6ae41434e0 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae4057590 con 0x7f6ae4142a00 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae40582b0 con 0x7f6ae4092400 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae8a8f640 10 monclient(hunting): authenticate will time out at 2209.970326s 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae1d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f6ae4092400 0x7f6ae4142dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59336/0 (socket says 10.20.193.102:59336) 2026-02-21T22:33:20.737 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.731+0000 7f6ae1d74640 1 -- 10.20.193.102:0/2268116351 learned_addr learned my addr 10.20.193.102:0/2268116351 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1d74640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.102:6790/0 conn(0x7f6ae4092400 legacy=0x7f6ae4142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.744 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae2d76640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.102:6789/0 conn(0x7f6ae41434e0 legacy=0x7f6ae41a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae2575640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.178:6789/0 conn(0x7f6ae4142a00 legacy=0x7f6ae4090850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614977605 0 0) 0x7f6ae40582b0 con 0x7f6ae4092400 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): my global_id is 94463 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ad0003620 con 0x7f6ae4092400 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571384110 0 0) 0x7f6ae40572b0 con 0x7f6ae41434e0 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient(hunting): my global_id is 34980 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ae40582b0 con 0x7f6ae41434e0 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3920932481 0 0) 0x7f6ad0003620 con 0x7f6ae4092400 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.178:6789/0 conn(0x7f6ae4142a00 legacy=0x7f6ae4090850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.745 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.102:6789/0 conn(0x7f6ae41434e0 legacy=0x7f6ae41a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 monclient: found mon.noname-b 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae41a96a0 con 0x7f6ae4092400 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6acc003110 con 0x7f6ae4092400 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: dump: 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.746 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 5 monclient: authenticate success, global_id 94463 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6acc003450 con 0x7f6ae4092400 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 1 -- 10.20.193.102:0/2268116351 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6acc004b50 con 0x7f6ae4092400 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1573640 10 monclient: dump: 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.747 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 4 set_mon_vals no callback set 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: shutdown 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/2268116351 >> v1:10.20.193.102:6790/0 conn(0x7f6ae4092400 legacy=0x7f6ae4142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/2268116351 shutdown_connections 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/2268116351 >> 10.20.193.102:0/2268116351 conn(0x7f6ae405e430 msgr2=0x7f6ae4095e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/2268116351 shutdown_connections 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/2268116351 wait complete. 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: monmap: 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.748 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 Processor -- start 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- start start 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: init 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-a con 0x7f6ae4092400 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-c con 0x7f6ae4142a00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f6ae41434e0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae41a8b20 con 0x7f6ae4092400 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae408efb0 con 0x7f6ae41434e0 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae412f510 con 0x7f6ae4142a00 2026-02-21T22:33:20.749 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae8a8f640 10 monclient(hunting): authenticate will time out at 2209.974895s 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f6ae4142a00 0x7f6ae41914c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59340/0 (socket says 10.20.193.102:59340) 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.735+0000 7f6ae1d74640 1 -- 10.20.193.102:0/4279717195 learned_addr learned my addr 10.20.193.102:0/4279717195 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae1d74640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.102:6790/0 conn(0x7f6ae4142a00 legacy=0x7f6ae41914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae2575640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.102:6789/0 conn(0x7f6ae4092400 legacy=0x7f6ae4140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae2575640 1 --1- 10.20.193.102:0/4279717195 >> v1:10.20.193.102:6789/0 conn(0x7f6ae4092400 0x7f6ae4140b40 :-1 s=OPENED pgs=704 gs=1 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f6ae41a8b20 auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476818401 0 0) 0x7f6ae412f510 con 0x7f6ae4142a00 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): my global_id is 94466 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab0003620 con 0x7f6ae4142a00 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae2d76640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.178:6789/0 conn(0x7f6ae41434e0 legacy=0x7f6ae4192be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1055217745 0 0) 0x7f6ab0003620 con 0x7f6ae4142a00 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ae412f510 con 0x7f6ae4142a00 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6acc0048e0 con 0x7f6ae4142a00 2026-02-21T22:33:20.750 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): dump: 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 504595414 0 0) 0x7f6ae412f510 con 0x7f6ae4142a00 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.178:6789/0 conn(0x7f6ae41434e0 legacy=0x7f6ae4192be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.102:6789/0 conn(0x7f6ae4092400 legacy=0x7f6ae4140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 monclient: found mon.c 2026-02-21T22:33:20.751 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae412ec70 con 0x7f6ae4142a00 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719076917 0 0) 0x7f6ae41a8b20 con 0x7f6ae4092400 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 5 monclient: authenticate success, global_id 94466 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 10 monclient: _renew_subs 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae412dd10 con 0x7f6ae4142a00 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 10 monclient: _renew_subs 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6ae4141420 con 0x7f6ae4142a00 2026-02-21T22:33:20.752 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6acc003380 con 0x7f6ae4142a00 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f6acc0052e0 con 0x7f6ae4142a00 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: dump: 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae0d72640 4 set_mon_vals no callback set 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.753 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f6ae4141a90 con 0x7f6ae4142a00 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f6acc011970 con 0x7f6ae4142a00 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f6acc04c4f0 con 0x7f6ae4142a00 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.178:6809/3629312922 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6acc005130 con 0x7f6ab0061ff0 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f6acc04c830 con 0x7f6ae4142a00 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6acaffd640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae2575640 10 monclient: get_auth_request con 0x7f6ab003d3c0 auth_method 0 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.739+0000 7f6ae2d76640 10 monclient: get_auth_request con 0x7f6ab0061ff0 auth_method 0 2026-02-21T22:33:20.754 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.747+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== osd.4 v1:10.20.193.178:6809/3629312922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f6acc005130 con 0x7f6ab0061ff0 2026-02-21T22:33:20.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 --> v1:10.20.193.178:6809/3629312922 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6ae41a6880 con 0x7f6ab0061ff0 2026-02-21T22:33:20.755 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6acaffd640 1 -- 10.20.193.102:0/4279717195 <== osd.4 v1:10.20.193.178:6809/3629312922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f6ae41a6880 con 0x7f6ab0061ff0 2026-02-21T22:33:20.755 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:20.755 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:20.755 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.178:6809/3629312922 conn(0x7f6ab0061ff0 legacy=0x7f6ab0082400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.178:6800/2549949439 conn(0x7f6ab003d3c0 legacy=0x7f6ab005d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 10 monclient: shutdown 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 >> v1:10.20.193.102:6790/0 conn(0x7f6ae4142a00 legacy=0x7f6ae41914c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.756 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae2d76640 1 -- 10.20.193.102:0/4279717195 reap_dead start 2026-02-21T22:33:20.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 shutdown_connections 2026-02-21T22:33:20.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 >> 10.20.193.102:0/4279717195 conn(0x7f6ae405e430 msgr2=0x7f6ae41a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 shutdown_connections 2026-02-21T22:33:20.757 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.751+0000 7f6ae8a8f640 1 -- 10.20.193.102:0/4279717195 wait complete. 2026-02-21T22:33:20.764 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-21T22:33:20.825 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: monmap: 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.824949+0000 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.824949+0000 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.826 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 1 Processor -- start 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 1 -- start start 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: init 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient(hunting): picked mon.noname-c con 0x7f158008cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient(hunting): picked mon.noname-a con 0x7f158008d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient(hunting): picked mon.noname-b con 0x7f158008b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15800890e0 con 0x7f158008d7e0 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1580088ea0 con 0x7f158008cd00 2026-02-21T22:33:20.827 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158008aea0 con 0x7f158008b8d0 2026-02-21T22:33:20.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient(hunting): authenticate will time out at 2210.060866s 2026-02-21T22:33:20.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157e7fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f158008d7e0 0x7f15801840b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38128/0 (socket says 10.20.193.102:38128) 2026-02-21T22:33:20.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157e7fc640 1 -- 10.20.193.102:0/2408792366 learned_addr learned my addr 10.20.193.102:0/2408792366 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.828 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157f7fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 0x7f15801a57e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59356/0 (socket says 10.20.193.102:59356) 2026-02-21T22:33:20.834 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157e7fc640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.102:6789/0 conn(0x7f158008d7e0 legacy=0x7f15801840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157f7fe640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 legacy=0x7f15801a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157effd640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.178:6789/0 conn(0x7f158008cd00 legacy=0x7f158008d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318828641 0 0) 0x7f15800890e0 con 0x7f158008d7e0 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): my global_id is 34986 2026-02-21T22:33:20.835 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f156c003620 con 0x7f158008d7e0 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463085158 0 0) 0x7f158008aea0 con 0x7f158008b8d0 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient(hunting): my global_id is 94469 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15800890e0 con 0x7f158008b8d0 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095838290 0 0) 0x7f156c003620 con 0x7f158008d7e0 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 legacy=0x7f15801a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.178:6789/0 conn(0x7f158008cd00 legacy=0x7f158008d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.836 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 monclient: found mon.noname-a 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158008a600 con 0x7f158008d7e0 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 5 monclient: authenticate success, global_id 34986 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1568003190 con 0x7f158008d7e0 2026-02-21T22:33:20.837 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: dump: 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.838 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1568003420 con 0x7f158008d7e0 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 1 -- 10.20.193.102:0/2408792366 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1568004b50 con 0x7f158008d7e0 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f157dffb640 10 monclient: dump: 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.839 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.823+0000 7f158589d640 4 set_mon_vals no callback set 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: shutdown 2026-02-21T22:33:20.840 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- 10.20.193.102:0/2408792366 >> v1:10.20.193.102:6789/0 conn(0x7f158008d7e0 legacy=0x7f15801840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- 10.20.193.102:0/2408792366 shutdown_connections 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- 10.20.193.102:0/2408792366 >> 10.20.193.102:0/2408792366 conn(0x7f1580091a50 msgr2=0x7f1580091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- 10.20.193.102:0/2408792366 shutdown_connections 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- 10.20.193.102:0/2408792366 wait complete. 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: monmap: 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:20.841 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 Processor -- start 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- start start 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: init 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient(hunting): picked mon.noname-c con 0x7f158008b8d0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient(hunting): picked mon.noname-b con 0x7f158008cd00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:20.842 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient(hunting): picked mon.noname-a con 0x7f158008d7e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15800885f0 con 0x7f158008d7e0 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1580088a70 con 0x7f158008cd00 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1580080040 con 0x7f158008b8d0 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f158589d640 10 monclient(hunting): authenticate will time out at 2210.065540s 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f157effd640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 0x7f15801534b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59360/0 (socket says 10.20.193.102:59360) 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f157effd640 1 -- 10.20.193.102:0/1075261940 learned_addr learned my addr 10.20.193.102:0/1075261940 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f157effd640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 legacy=0x7f15801534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f157f7fe640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.102:6789/0 conn(0x7f158008d7e0 legacy=0x7f1580195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800556430 0 0) 0x7f1580080040 con 0x7f158008b8d0 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.843 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): my global_id is 94472 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1550003620 con 0x7f158008b8d0 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824792268 0 0) 0x7f15800885f0 con 0x7f158008d7e0 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 10 monclient(hunting): my global_id is 34989 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580080040 con 0x7f158008d7e0 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.827+0000 7f157e7fc640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.178:6789/0 conn(0x7f158008cd00 legacy=0x7f15800838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2480462284 0 0) 0x7f1550003620 con 0x7f158008b8d0 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f15800885f0 con 0x7f158008b8d0 2026-02-21T22:33:20.844 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1574003430 con 0x7f158008b8d0 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): dump: 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.845 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2896841792 0 0) 0x7f1580080040 con 0x7f158008d7e0 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1550003620 con 0x7f158008d7e0 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1570004830 con 0x7f158008d7e0 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient(hunting): dump: 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.846 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 376414233 0 0) 0x7f15800885f0 con 0x7f158008b8d0 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.178:6789/0 conn(0x7f158008cd00 legacy=0x7f15800838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.102:6789/0 conn(0x7f158008d7e0 legacy=0x7f1580195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 monclient: found mon.c 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.847 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15800823a0 con 0x7f158008b8d0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: _finish_auth 0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 5 monclient: authenticate success, global_id 94472 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _renew_subs 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1580080ba0 con 0x7f158008b8d0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _renew_subs 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f15800841b0 con 0x7f158008b8d0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1574003ba0 con 0x7f158008b8d0 2026-02-21T22:33:20.848 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f1574003f00 con 0x7f158008b8d0 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 10 monclient: dump: 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:20.849 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f157d7fa640 4 set_mon_vals no callback set 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1580081f50 con 0x7f158008b8d0 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f1574004190 con 0x7f158008b8d0 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f157404c5a0 con 0x7f158008b8d0 2026-02-21T22:33:20.850 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.831+0000 7f157e7fc640 10 monclient: get_auth_request con 0x7f155003d6e0 auth_method 0 2026-02-21T22:33:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.835+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f1580081f50 con 0x7f158008b8d0 2026-02-21T22:33:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.835+0000 7f1557fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:20.851 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.835+0000 7f1557fff640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:20.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1580153bc0 con 0x7f155003d6e0 2026-02-21T22:33:20.922 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f1557fff640 1 -- 10.20.193.102:0/1075261940 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (unknown 3782719169 0 0) 0x7f15680048e0 con 0x7f155003d6e0 2026-02-21T22:33:20.922 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 4 to deep-scrub 2026-02-21T22:33:20.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.178:6800/2549949439 conn(0x7f155003d6e0 legacy=0x7f155005db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.923 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 10 monclient: shutdown 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 >> v1:10.20.193.102:6790/0 conn(0x7f158008b8d0 legacy=0x7f15801534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 shutdown_connections 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 >> 10.20.193.102:0/1075261940 conn(0x7f1580091a50 msgr2=0x7f1580085760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 shutdown_connections 2026-02-21T22:33:20.924 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.919+0000 7f158589d640 1 -- 10.20.193.102:0/1075261940 wait complete. 2026-02-21T22:33:20.932 INFO:tasks.ceph:Scrubbing osd.5 2026-02-21T22:33:20.933 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-21T22:33:20.997 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: get_monmap_and_config 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: build_initial_monmap 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: monmap: 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:20.998384+0000 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:20.998384+0000 2026-02-21T22:33:20.998 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:20.999 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:20.999 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:20.999 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:20.999 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:20.999 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.009 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 1 Processor -- start 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 1 -- start start 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: init 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-c con 0x7f4e58091310 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-a con 0x7f4e580916e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e58185480 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58056390 con 0x7f4e580916e0 2026-02-21T22:33:21.010 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58057430 con 0x7f4e58091310 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58092560 con 0x7f4e58185480 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5f19c640 10 monclient(hunting): authenticate will time out at 2210.234802s 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5d712640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4e58185480 0x7f4e581a5860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59374/0 (socket says 10.20.193.102:59374) 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.995+0000 7f4e5d712640 1 -- 10.20.193.102:0/4270563816 learned_addr learned my addr 10.20.193.102:0/4270563816 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5d712640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.102:6790/0 conn(0x7f4e58185480 legacy=0x7f4e581a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4ffff640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.102:6789/0 conn(0x7f4e580916e0 legacy=0x7f4e58183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5cf11640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.178:6789/0 conn(0x7f4e58091310 legacy=0x7f4e580584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3283960005 0 0) 0x7f4e58092560 con 0x7f4e58185480 2026-02-21T22:33:21.011 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): my global_id is 94475 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e44003620 con 0x7f4e58185480 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136036913 0 0) 0x7f4e58056390 con 0x7f4e580916e0 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient(hunting): my global_id is 34992 2026-02-21T22:33:21.012 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e58092560 con 0x7f4e580916e0 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 361273035 0 0) 0x7f4e44003620 con 0x7f4e58185480 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.178:6789/0 conn(0x7f4e58091310 legacy=0x7f4e580584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.102:6789/0 conn(0x7f4e580916e0 legacy=0x7f4e58183d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 monclient: found mon.noname-b 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58091b20 con 0x7f4e58185480 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.013 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e54004620 con 0x7f4e58185480 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: dump: 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.014 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 5 monclient: authenticate success, global_id 94475 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e540048f0 con 0x7f4e58185480 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 1 -- 10.20.193.102:0/4270563816 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e54006060 con 0x7f4e58185480 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4f7fe640 10 monclient: dump: 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.015 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 4 set_mon_vals no callback set 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: shutdown 2026-02-21T22:33:21.016 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- 10.20.193.102:0/4270563816 >> v1:10.20.193.102:6790/0 conn(0x7f4e58185480 legacy=0x7f4e581a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- 10.20.193.102:0/4270563816 shutdown_connections 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- 10.20.193.102:0/4270563816 >> 10.20.193.102:0/4270563816 conn(0x7f4e5805e520 msgr2=0x7f4e58095ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- 10.20.193.102:0/4270563816 shutdown_connections 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- 10.20.193.102:0/4270563816 wait complete. 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: build_initial_monmap 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: monmap: 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:21.017 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 Processor -- start 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- start start 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: init 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-b con 0x7f4e58091310 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.018 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-c con 0x7f4e580916e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient(hunting): picked mon.noname-a con 0x7f4e58185480 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e5812bb40 con 0x7f4e58185480 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58112d90 con 0x7f4e58091310 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58111f20 con 0x7f4e580916e0 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e5f19c640 10 monclient(hunting): authenticate will time out at 2210.238688s 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4ffff640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f4e580916e0 0x7f4e581727f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59376/0 (socket says 10.20.193.102:59376) 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:20.999+0000 7f4e4ffff640 1 -- 10.20.193.102:0/3397859748 learned_addr learned my addr 10.20.193.102:0/3397859748 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4ffff640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.102:6790/0 conn(0x7f4e580916e0 legacy=0x7f4e581727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5d712640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.102:6789/0 conn(0x7f4e58185480 legacy=0x7f4e58173f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5cf11640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.178:6789/0 conn(0x7f4e58091310 legacy=0x7f4e5812b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236511782 0 0) 0x7f4e58111f20 con 0x7f4e580916e0 2026-02-21T22:33:21.019 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): my global_id is 94478 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e28003620 con 0x7f4e580916e0 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430570925 0 0) 0x7f4e5812bb40 con 0x7f4e58185480 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): my global_id is 34995 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e58111f20 con 0x7f4e58185480 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1582491918 0 0) 0x7f4e28003620 con 0x7f4e580916e0 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e5812bb40 con 0x7f4e580916e0 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e400033c0 con 0x7f4e580916e0 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.020 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): dump: 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2720289181 0 0) 0x7f4e58111f20 con 0x7f4e58185480 2026-02-21T22:33:21.021 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e28003620 con 0x7f4e58185480 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e54005df0 con 0x7f4e58185480 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient(hunting): dump: 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.022 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1852110007 0 0) 0x7f4e5812bb40 con 0x7f4e580916e0 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.178:6789/0 conn(0x7f4e58091310 legacy=0x7f4e5812b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.102:6789/0 conn(0x7f4e58185480 legacy=0x7f4e58173f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 monclient: found mon.c 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58110a30 con 0x7f4e580916e0 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.023 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 5 monclient: authenticate success, global_id 94478 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 10 monclient: _renew_subs 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4e58112550 con 0x7f4e580916e0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 10 monclient: _renew_subs 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4e58111760 con 0x7f4e580916e0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e400039c0 con 0x7f4e580916e0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f4e40003d50 con 0x7f4e580916e0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: dump: 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.024 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4effd640 4 set_mon_vals no callback set 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.102:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f4e58095010 con 0x7f4e580916e0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f4e40003fe0 con 0x7f4e580916e0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f4e4004c530 con 0x7f4e580916e0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.178:6801/2402357529 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e40003ba0 con 0x7f4e280622d0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=831) ==== 24+0+0 (unknown 2528087267 0 0) 0x7f4e4004c870 con 0x7f4e580916e0 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e4d7fa640 10 monclient: handle_get_version_reply finishing 1 version 831 2026-02-21T22:33:21.025 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5cf11640 10 monclient: get_auth_request con 0x7f4e2803d650 auth_method 0 2026-02-21T22:33:21.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.003+0000 7f4e5d712640 10 monclient: get_auth_request con 0x7f4e280622d0 auth_method 0 2026-02-21T22:33:21.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.007+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== osd.5 v1:10.20.193.178:6801/2402357529 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (unknown 0 0 1763635307) 0x7f4e40003ba0 con 0x7f4e280622d0 2026-02-21T22:33:21.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 --> v1:10.20.193.178:6801/2402357529 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4e581a71d0 con 0x7f4e280622d0 2026-02-21T22:33:21.026 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e4d7fa640 1 -- 10.20.193.102:0/3397859748 <== osd.5 v1:10.20.193.178:6801/2402357529 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f4e581a71d0 con 0x7f4e280622d0 2026-02-21T22:33:21.027 INFO:teuthology.orchestra.run.trial102.stdout:{ 2026-02-21T22:33:21.027 INFO:teuthology.orchestra.run.trial102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T22:33:21.027 INFO:teuthology.orchestra.run.trial102.stdout:} 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.178:6801/2402357529 conn(0x7f4e280622d0 legacy=0x7f4e280826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.178:6800/2549949439 conn(0x7f4e2803d650 legacy=0x7f4e2805db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 10 monclient: shutdown 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 >> v1:10.20.193.102:6790/0 conn(0x7f4e580916e0 legacy=0x7f4e581727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.028 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5d712640 1 -- 10.20.193.102:0/3397859748 reap_dead start 2026-02-21T22:33:21.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 shutdown_connections 2026-02-21T22:33:21.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 >> 10.20.193.102:0/3397859748 conn(0x7f4e5805e520 msgr2=0x7f4e5808bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 shutdown_connections 2026-02-21T22:33:21.029 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.015+0000 7f4e5f19c640 1 -- 10.20.193.102:0/3397859748 wait complete. 2026-02-21T22:33:21.029 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-21T22:33:21.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: get_monmap_and_config 2026-02-21T22:33:21.090 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: build_initial_monmap 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: monmap: 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:21.091473+0000 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:21.091473+0000 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:21.091 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 1 Processor -- start 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 1 -- start start 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: init 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-c con 0x7f26d808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.092 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-a con 0x7f26d808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-b con 0x7f26d808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d8086a10 con 0x7f26d808b7f0 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d808b130 con 0x7f26d808ce00 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d808bd60 con 0x7f26d808d8e0 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26de59c640 10 monclient(hunting): authenticate will time out at 2210.327425s 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26dcb12640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f26d808d8e0 0x7f26d81a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59384/0 (socket says 10.20.193.102:59384) 2026-02-21T22:33:21.093 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.087+0000 7f26dcb12640 1 -- 10.20.193.102:0/1764208331 learned_addr learned my addr 10.20.193.102:0/1764208331 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26dcb12640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.102:6790/0 conn(0x7f26d808d8e0 legacy=0x7f26d81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d77fe640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.102:6789/0 conn(0x7f26d808b7f0 legacy=0x7f26d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378470943 0 0) 0x7f26d808bd60 con 0x7f26d808d8e0 2026-02-21T22:33:21.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.095 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): my global_id is 94481 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26bc003620 con 0x7f26d808d8e0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526176103 0 0) 0x7f26d8086a10 con 0x7f26d808b7f0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient(hunting): my global_id is 34998 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d7fff640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.178:6789/0 conn(0x7f26d808ce00 legacy=0x7f26d808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d808bd60 con 0x7f26d808b7f0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1555620589 0 0) 0x7f26bc003620 con 0x7f26d808d8e0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.178:6789/0 conn(0x7f26d808ce00 legacy=0x7f26d808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.102:6789/0 conn(0x7f26d808b7f0 legacy=0x7f26d8181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 monclient: found mon.noname-b 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d808b370 con 0x7f26d808d8e0 2026-02-21T22:33:21.096 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26cc004620 con 0x7f26d808d8e0 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: dump: 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.097 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 5 monclient: authenticate success, global_id 94481 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f26cc0048f0 con 0x7f26d808d8e0 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 1 -- 10.20.193.102:0/1764208331 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26cc006060 con 0x7f26d808d8e0 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d6ffd640 10 monclient: dump: 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:21.098 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 4 set_mon_vals no callback set 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: shutdown 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- 10.20.193.102:0/1764208331 >> v1:10.20.193.102:6790/0 conn(0x7f26d808d8e0 legacy=0x7f26d81a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- 10.20.193.102:0/1764208331 shutdown_connections 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- 10.20.193.102:0/1764208331 >> 10.20.193.102:0/1764208331 conn(0x7f26d8091a50 msgr2=0x7f26d8091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- 10.20.193.102:0/1764208331 shutdown_connections 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- 10.20.193.102:0/1764208331 wait complete. 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: build_initial_monmap 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: monmap: 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:21.099 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 Processor -- start 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- start start 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: init 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-b con 0x7f26d808b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-c con 0x7f26d808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient(hunting): picked mon.noname-a con 0x7f26d808d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d8083300 con 0x7f26d808d8e0 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d8089bd0 con 0x7f26d808b7f0 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d80835e0 con 0x7f26d808ce00 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26de59c640 10 monclient(hunting): authenticate will time out at 2210.331279s 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d77fe640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f26d808ce00 0x7f26d80894c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:59392/0 (socket says 10.20.193.102:59392) 2026-02-21T22:33:21.100 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.091+0000 7f26d77fe640 1 -- 10.20.193.102:0/3358597698 learned_addr learned my addr 10.20.193.102:0/3358597698 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d77fe640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.102:6790/0 conn(0x7f26d808ce00 legacy=0x7f26d80894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.102 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26dcb12640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.102:6789/0 conn(0x7f26d808d8e0 legacy=0x7f26d8195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931682863 0 0) 0x7f26d8083300 con 0x7f26d808d8e0 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): my global_id is 35001 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a8003620 con 0x7f26d808d8e0 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423954847 0 0) 0x7f26d80835e0 con 0x7f26d808ce00 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): my global_id is 94484 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d8083300 con 0x7f26d808ce00 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d7fff640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.178:6789/0 conn(0x7f26d808b7f0 legacy=0x7f26d8088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 DispatchQueue::entry inject delay of 0.002000 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 45512919 0 0) 0x7f26a8003620 con 0x7f26d808d8e0 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26d80835e0 con 0x7f26d808d8e0 2026-02-21T22:33:21.103 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1429276090 0 0) 0x7f26d8083300 con 0x7f26d808ce00 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26a8003620 con 0x7f26d808ce00 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26cc004080 con 0x7f26d808d8e0 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): dump: 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26c80033f0 con 0x7f26d808ce00 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.104 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient(hunting): dump: 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4071419313 0 0) 0x7f26d80835e0 con 0x7f26d808d8e0 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.102:6790/0 conn(0x7f26d808ce00 legacy=0x7f26d80894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.178:6789/0 conn(0x7f26d808b7f0 legacy=0x7f26d8088db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 monclient: found mon.a 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d8088a70 con 0x7f26d808d8e0 2026-02-21T22:33:21.105 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26d4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 5 monclient: authenticate success, global_id 35001 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 10 monclient: _renew_subs 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f26d8083020 con 0x7f26d808d8e0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 10 monclient: _renew_subs 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.095+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f26d8196d80 con 0x7f26d808d8e0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f26cc0048d0 con 0x7f26d808d8e0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f26cc006680 con 0x7f26d808d8e0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: dump: 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.106 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d67fc640 4 set_mon_vals no callback set 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26de59c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26de59c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26de59c640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d80008e0 con 0x7f26d808d8e0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f26cc012d10 con 0x7f26d808d8e0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f26cc04d830 con 0x7f26d808d8e0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d7fff640 10 monclient: get_auth_request con 0x7f26a803d740 auth_method 0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f26cc0189f0 con 0x7f26d808d8e0 2026-02-21T22:33:21.107 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.108 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.099+0000 7f26d4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:21.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f26d80589b0 con 0x7f26a803d740 2026-02-21T22:33:21.188 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26d4ff9640 1 -- 10.20.193.102:0/3358597698 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (unknown 2264484800 0 0) 0x7f26d80589b0 con 0x7f26a803d740 2026-02-21T22:33:21.188 INFO:teuthology.orchestra.run.trial102.stderr:instructed osd(s) 5 to deep-scrub 2026-02-21T22:33:21.189 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.178:6800/2549949439 conn(0x7f26a803d740 legacy=0x7f26a805dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 10 monclient: shutdown 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 >> v1:10.20.193.102:6789/0 conn(0x7f26d808d8e0 legacy=0x7f26d8195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 shutdown_connections 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 >> 10.20.193.102:0/3358597698 conn(0x7f26d8091a50 msgr2=0x7f26d80575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 shutdown_connections 2026-02-21T22:33:21.190 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.187+0000 7f26de59c640 1 -- 10.20.193.102:0/3358597698 wait complete. 2026-02-21T22:33:21.199 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:33:21.259 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: get_monmap_and_config 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: build_initial_monmap 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: monmap: 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:21.258899+0000 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:21.258899+0000 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.260 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 1 Processor -- start 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 1 -- start start 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: init 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-c con 0x7f0e6808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.261 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-b con 0x7f0e6808b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-a con 0x7f0e6808d8e0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e68086a10 con 0x7f0e6808d8e0 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6808b130 con 0x7f0e6808ce00 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6808bd60 con 0x7f0e6808b7f0 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e6d5dc640 10 monclient(hunting): authenticate will time out at 2210.494910s 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e677fe640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 0x7f0e681a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38182/0 (socket says 10.20.193.102:38182) 2026-02-21T22:33:21.262 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.255+0000 7f0e677fe640 1 -- 10.20.193.102:0/504915458 learned_addr learned my addr 10.20.193.102:0/504915458 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 legacy=0x7f0e681a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e667fc640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808b7f0 legacy=0x7f0e68181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.272 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e66ffd640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.178:6789/0 conn(0x7f0e6808ce00 legacy=0x7f0e6808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601767262 0 0) 0x7f0e68086a10 con 0x7f0e6808d8e0 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): my global_id is 35004 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e54003620 con 0x7f0e6808d8e0 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218721471 0 0) 0x7f0e6808bd60 con 0x7f0e6808b7f0 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.273 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): my global_id is 94487 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e68086a10 con 0x7f0e6808b7f0 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4106752868 0 0) 0x7f0e54003620 con 0x7f0e6808d8e0 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808b7f0 legacy=0x7f0e68181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.178:6789/0 conn(0x7f0e6808ce00 legacy=0x7f0e6808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 monclient: found mon.noname-a 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6808b370 con 0x7f0e6808d8e0 2026-02-21T22:33:21.274 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e58004620 con 0x7f0e6808d8e0 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: dump: 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.275 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e6d5dc640 5 monclient: authenticate success, global_id 35004 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e6d5dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 legacy=0x7f0e681a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 legacy=0x7f0e681a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-21T22:33:21.276 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 --1- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 0x7f0e681a3570 :-1 s=OPENED pgs=712 gs=3 cs=1 l=1).handle_message read tag failed 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 --1- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 0x7f0e681a3570 :-1 s=OPENED pgs=712 gs=3 cs=1 l=1).fault on lossy channel, failing 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: ms_handle_reset current mon v1:10.20.193.102:6789/0 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808d8e0 legacy=0x7f0e681a3570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): picked mon.b con 0x7f0e6808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): picked mon.a con 0x7f0e54004460 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): picked mon.c con 0x7f0e6808b7f0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0e54003620 con 0x7f0e54004460 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0e6808bd60 con 0x7f0e6808ce00 2026-02-21T22:33:21.277 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0e5406af70 con 0x7f0e6808b7f0 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e65ffb640 10 monclient(hunting): ms_handle_reset hunted mon v1:10.20.193.102:6789/0 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e667fc640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e54004460 legacy=0x7f0e54045aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.259+0000 7f0e677fe640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808b7f0 legacy=0x7f0e54067580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e66ffd640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.178:6789/0 conn(0x7f0e6808ce00 legacy=0x7f0e54003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292520304 0 0) 0x7f0e54003620 con 0x7f0e54004460 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-21T22:33:21.278 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6789/0 -- auth(proto 2 132 bytes epoch 0) -- 0x7f0e5406b140 con 0x7f0e54004460 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (unknown 281411334 0 0) 0x7f0e5406b140 con 0x7f0e54004460 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808b7f0 legacy=0x7f0e54067580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.178:6789/0 conn(0x7f0e6808ce00 legacy=0x7f0e54003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 monclient: found mon.a 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=2+}) -- 0x7f0e5406c2d0 con 0x7f0e54004460 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.279 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e50003310 con 0x7f0e54004460 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: dump: 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.280 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 1 -- 10.20.193.102:0/504915458 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e50003710 con 0x7f0e54004460 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e65ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 4 set_mon_vals no callback set 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: shutdown 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/504915458 >> v1:10.20.193.102:6789/0 conn(0x7f0e54004460 legacy=0x7f0e54045aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/504915458 shutdown_connections 2026-02-21T22:33:21.281 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/504915458 >> 10.20.193.102:0/504915458 conn(0x7f0e68091a50 msgr2=0x7f0e68091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/504915458 shutdown_connections 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/504915458 wait complete. 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: build_initial_monmap 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: monmap: 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:21.282 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 Processor -- start 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- start start 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: init 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-b con 0x7f0e54004460 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-a con 0x7f0e6808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:21.283 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient(hunting): picked mon.noname-c con 0x7f0e6808ce00 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e680811b0 con 0x7f0e6808b7f0 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e68083510 con 0x7f0e54004460 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e68082c70 con 0x7f0e6808ce00 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e6d5dc640 10 monclient(hunting): authenticate will time out at 2210.501658s 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e667fc640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f0e6808b7f0 0x7f0e68084a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:38202/0 (socket says 10.20.193.102:38202) 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e667fc640 1 -- 10.20.193.102:0/3637518606 learned_addr learned my addr 10.20.193.102:0/3637518606 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e667fc640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808b7f0 legacy=0x7f0e68084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.263+0000 7f0e677fe640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808ce00 legacy=0x7f0e6823c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e66ffd640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.178:6789/0 conn(0x7f0e54004460 legacy=0x7f0e681d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3215496995 0 0) 0x7f0e680811b0 con 0x7f0e6808b7f0 2026-02-21T22:33:21.284 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): my global_id is 35007 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e34003620 con 0x7f0e6808b7f0 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195139134 0 0) 0x7f0e68082c70 con 0x7f0e6808ce00 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _negotiate 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): my global_id is 94490 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e680811b0 con 0x7f0e6808ce00 2026-02-21T22:33:21.285 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4162299405 0 0) 0x7f0e34003620 con 0x7f0e6808b7f0 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e68082c70 con 0x7f0e6808b7f0 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e50003510 con 0x7f0e6808b7f0 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): dump: 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.286 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 277403552 0 0) 0x7f0e680811b0 con 0x7f0e6808ce00 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e34003620 con 0x7f0e6808ce00 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e58004970 con 0x7f0e6808ce00 2026-02-21T22:33:21.287 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient(hunting): dump: 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.288 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3653505631 0 0) 0x7f0e68082c70 con 0x7f0e6808b7f0 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.102:6790/0 conn(0x7f0e6808ce00 legacy=0x7f0e6823c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.178:6789/0 conn(0x7f0e54004460 legacy=0x7f0e681d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: _finish_hunting 0 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 monclient: found mon.a 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e68081d10 con 0x7f0e6808b7f0 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: _finish_auth 0 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: _check_auth_tickets 2026-02-21T22:33:21.289 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 5 monclient: authenticate success, global_id 35007 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _renew_subs 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0e68085150 con 0x7f0e6808b7f0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _renew_subs 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0e6823daa0 con 0x7f0e6808b7f0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0e50003710 con 0x7f0e6808b7f0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f0e50003e80 con 0x7f0e6808b7f0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:21.290 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: dump: 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e657fa640 4 set_mon_vals no callback set 2026-02-21T22:33:21.291 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 10 monclient: _send_mon_message to mon.a at v1:10.20.193.102:6789/0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e680008e0 con 0x7f0e6808b7f0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f0e50004060 con 0x7f0e6808b7f0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f0e5004c3b0 con 0x7f0e6808b7f0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e66ffd640 10 monclient: get_auth_request con 0x7f0e3403d5f0 auth_method 0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mon.0 v1:10.20.193.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f0e50017630 con 0x7f0e6808b7f0 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:21.292 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.267+0000 7f0e477fe640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:21.356 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e68088a70 con 0x7f0e3403d5f0 2026-02-21T22:33:21.357 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e477fe640 1 -- 10.20.193.102:0/3637518606 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34069 (unknown 2965378022 0 2619499919) 0x7f0e68088a70 con 0x7f0e3403d5f0 2026-02-21T22:33:21.357 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:21.357 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.178:6800/2549949439 conn(0x7f0e3403d5f0 legacy=0x7f0e3405daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 10 monclient: shutdown 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 >> v1:10.20.193.102:6789/0 conn(0x7f0e6808b7f0 legacy=0x7f0e68084a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 shutdown_connections 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 >> 10.20.193.102:0/3637518606 conn(0x7f0e68091a50 msgr2=0x7f0e6805d740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 shutdown_connections 2026-02-21T22:33:21.358 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:21.355+0000 7f0e6d5dc640 1 -- 10.20.193.102:0/3637518606 wait complete. 2026-02-21T22:33:21.367 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1369,"stamp":"2026-02-21T22:33:19.617427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":212704,"kb_used_data":14848,"kb_used_omap":8260,"kb_used_meta":189499,"kb_avail":566018336,"statfs":{"total":579820584960,"available":579602776064,"internally_reserved":0,"allocated":15204352,"data_stored":9432172,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":8458769,"internal_metadata":194047471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001114"},"pg_stats":[{"pgid":"1.0","version":"14'112","reported_seq":1759,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169478+0000","last_change":"2026-02-21T22:15:31.430477+0000","last_active":"2026-02-21T22:32:46.169478+0000","last_peered":"2026-02-21T22:32:46.169478+0000","last_clean":"2026-02-21T22:32:46.169478+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:32:46.169478+0000","last_undegraded":"2026-02-21T22:32:46.169478+0000","last_fullsized":"2026-02-21T22:32:46.169478+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:30.418217+0000","last_clean_scrub_stamp":"2026-02-21T22:15:30.418217+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T22:38:13.236635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":1637,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169991+0000","last_change":"2026-02-21T22:15:36.542334+0000","last_active":"2026-02-21T22:32:46.169991+0000","last_peered":"2026-02-21T22:32:46.169991+0000","last_clean":"2026-02-21T22:32:46.169991+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:32:46.169991+0000","last_undegraded":"2026-02-21T22:32:46.169991+0000","last_fullsized":"2026-02-21T22:32:46.169991+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:30:23.554423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169377+0000","last_change":"2026-02-21T22:15:36.540688+0000","last_active":"2026-02-21T22:32:46.169377+0000","last_peered":"2026-02-21T22:32:46.169377+0000","last_clean":"2026-02-21T22:32:46.169377+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:32:46.169377+0000","last_undegraded":"2026-02-21T22:32:46.169377+0000","last_fullsized":"2026-02-21T22:32:46.169377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:44:10.422431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1624,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.667699+0000","last_change":"2026-02-21T22:15:36.538007+0000","last_active":"2026-02-21T22:32:45.667699+0000","last_peered":"2026-02-21T22:32:45.667699+0000","last_clean":"2026-02-21T22:32:45.667699+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:32:45.667699+0000","last_undegraded":"2026-02-21T22:32:45.667699+0000","last_fullsized":"2026-02-21T22:32:45.667699+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T04:45:39.571478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":1638,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.170146+0000","last_change":"2026-02-21T22:15:36.542874+0000","last_active":"2026-02-21T22:32:46.170146+0000","last_peered":"2026-02-21T22:32:46.170146+0000","last_clean":"2026-02-21T22:32:46.170146+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:32:46.170146+0000","last_undegraded":"2026-02-21T22:32:46.170146+0000","last_fullsized":"2026-02-21T22:32:46.170146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:45:03.025741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293813+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293813+0000","last_peered":"2026-02-21T22:32:45.293813+0000","last_clean":"2026-02-21T22:32:45.293813+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:32:45.293813+0000","last_undegraded":"2026-02-21T22:32:45.293813+0000","last_fullsized":"2026-02-21T22:32:45.293813+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:13:30.078154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169308+0000","last_change":"2026-02-21T22:15:36.540562+0000","last_active":"2026-02-21T22:32:46.169308+0000","last_peered":"2026-02-21T22:32:46.169308+0000","last_clean":"2026-02-21T22:32:46.169308+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:32:46.169308+0000","last_undegraded":"2026-02-21T22:32:46.169308+0000","last_fullsized":"2026-02-21T22:32:46.169308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:59:14.821033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1631,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:45.293852+0000","last_change":"2026-02-21T22:15:36.538116+0000","last_active":"2026-02-21T22:32:45.293852+0000","last_peered":"2026-02-21T22:32:45.293852+0000","last_clean":"2026-02-21T22:32:45.293852+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:32:45.293852+0000","last_undegraded":"2026-02-21T22:32:45.293852+0000","last_fullsized":"2026-02-21T22:32:45.293852+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:03:55.450854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1622,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:32:46.169305+0000","last_change":"2026-02-21T22:15:36.540434+0000","last_active":"2026-02-21T22:32:46.169305+0000","last_peered":"2026-02-21T22:32:46.169305+0000","last_clean":"2026-02-21T22:32:46.169305+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:32:46.169305+0000","last_undegraded":"2026-02-21T22:32:46.169305+0000","last_fullsized":"2026-02-21T22:32:46.169305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:15:33.523307+0000","last_clean_scrub_stamp":"2026-02-21T22:15:33.523307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:04:47.036803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607769,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29412,"kb_used_data":2452,"kb_used_omap":1172,"kb_used_meta":25771,"kb_avail":94342428,"statfs":{"total":96636764160,"available":96606646272,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1200561,"internal_metadata":26390095},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640473,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2456,"kb_used_omap":1308,"kb_used_meta":25635,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1340101,"internal_metadata":26250555},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640472,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":854,"kb_used_meta":26089,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":874656,"internal_metadata":26716000},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640472,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":2452,"kb_used_omap":1802,"kb_used_meta":43189,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1845358,"internal_metadata":44226450},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640473,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47464,"kb_used_data":2456,"kb_used_omap":1841,"kb_used_meta":43150,"kb_avail":94324376,"statfs":{"total":96636764160,"available":96588161024,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1885730,"internal_metadata":44186078},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640475,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":1281,"kb_used_meta":25662,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":1312363,"internal_metadata":26278293},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T22:15:30.418217+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=30, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-21T22:15:33.523307+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=15, tm_sec=33, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=22, tm_min=33, tm_sec=18, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T22:33:21.370 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T22:33:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:33:23.543 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:33:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Waking up [active, now 2026-02-21_22:33:23] 2026-02-21T22:33:23.544 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Running 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Optimize plan auto_2026-02-21_22:33:23, mode upmap, max misplaced 0.050000 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:unknown 0.000000 degraded 0.000000 inactive 0.000000 misplaced 0 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:do_upmap 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pools ['.mgr', 'rbd'] 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:prepared 0/10 upmap changes 2026-02-21T22:33:23.545 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Sleeping for 60 2026-02-21T22:33:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:33:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:scanning for idle connections.. 2026-02-21T22:33:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:33:23.557 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:mgr_util:cleaning up connections: [] 2026-02-21T22:33:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: refresh_images 2026-02-21T22:33:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:MirrorSnapshotScheduleHandler: load_schedules 2026-02-21T22:33:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:33:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: refresh_pools 2026-02-21T22:33:23.579 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:TrashPurgeScheduleHandler: load_schedules 2026-02-21T22:33:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:load_schedules: rbd, start_after= 2026-02-21T22:33:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:MirrorSnapshotScheduleHandler: no schedules 2026-02-21T22:33:23.580 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TrashPurgeScheduleHandler: no schedules 2026-02-21T22:33:23.584 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:_maybe_adjust 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:pool: { 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: ".mgr": { 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "mgr": {} 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:33:23.585 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:29.531133+0000", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 1, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "15", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:33:23.586 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": { 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_max": 32, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_min": 1 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 1, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": ".mgr", 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:33:23.587 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 0.33000001311302185, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 6.059999942779541, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 6.059999942779541, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": { 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "application_metadata": { 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "rbd": {} 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "auid": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_evict_age": 0, 2026-02-21T22:33:23.588 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_min_flush_age": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_mode": "none", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "cache_target_full_ratio_micro": 800000, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "create_time": "2026-02-21T22:15:33.157956+0000", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "crush_rule": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "erasure_code_profile": "", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "expected_num_objects": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "fast_read": false, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags": 8193, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "flags_names": "hashpspool,selfmanaged_snaps", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "grade_table": [], 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_count": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_grade_decay_rate": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_params": { 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": "none" 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_period": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "hit_set_search_last_n": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "is_stretch_pool": false, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_change": "19", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend": "0", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_preluminous": "0", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_force_op_resend_prenautilus": "0", 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_pg_merge_meta": { 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_clean": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "last_epoch_started": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "ready_epoch": 0, 2026-02-21T22:33:23.589 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_pgid": "0.0", 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "source_version": "0'0", 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_version": "0'0" 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_read_recency_for_promote": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_size": 1, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "min_write_recency_for_promote": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "nonprimary_shards": "{}", 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "object_hash": 2, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "options": {}, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_barrier": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_count": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "peering_crush_bucket_target": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_autoscale_mode": "off", 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num": 8, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_pending": 8, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_num_target": 8, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num": 8, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pg_placement_num_target": 8, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool": 2, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_name": "rbd", 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "pool_snaps": [], 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_bytes": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "quota_max_objects": 0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_balance": { 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity": 1.0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "average_primary_affinity_weighted": 1.0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "optimal_score": 1.0, 2026-02-21T22:33:23.590 INFO:tasks.ceph.mgr.x.trial178.stderr: "primary_affinity_weighted": 1.0, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_acting": 2.25, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "raw_score_stable": 2.25, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_acting": 2.25, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_stable": 2.25, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "score_type": "Fair distribution" 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: }, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "read_tier": -1, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "removed_snaps": "[]", 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "size": 2, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_epoch": 19, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_mode": "selfmanaged", 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "snap_seq": 2, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "stripe_width": 0, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_bytes": 0, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "target_max_objects": 0, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "tier_of": -1, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "tiers": [], 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "type": 1, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "use_gmt_hitset": true, 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: "write_tier": -1 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr: } 2026-02-21T22:33:23.591 INFO:tasks.ceph.mgr.x.trial178.stderr:} 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:root_ids [-1, -1] pools [1, 2] with 6 osds, pg_target 600 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool '.mgr' root_id -1 using 2.2605613425925927e-07 of space, bias 1.0, pg target 6.781684027777778e-05 quantized to 1 (current 1) 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:effective_target_ratio 0.0 0.0 0 579820584960 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:INFO:root:Pool 'rbd' root_id -1 using 1.4128508391203704e-08 of space, bias 1.0, pg target 4.238552517361111e-06 quantized to 32 (current 8) 2026-02-21T22:33:23.592 INFO:tasks.ceph.mgr.x.trial178.stderr:WARNING:root:pool rbd won't scale because recommended PG_NUM target value varies from current PG_NUM value by more than '3.000000' scaling threshold 2026-02-21T22:33:23.597 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:23.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:28.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:28.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:33.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:33.598 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:38.599 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:38.599 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:41.371 DEBUG:teuthology.orchestra.run.trial102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T22:33:41.440 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: get_monmap_and_config 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: build_initial_monmap 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: monmap: 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:41.439742+0000 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:41.439742+0000 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:41.441 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 1 Processor -- start 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 1 -- start start 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: init 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:41.442 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-a con 0x7f5f5808ce00 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-c con 0x7f5f5808b7f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-b con 0x7f5f5808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f58086a10 con 0x7f5f5808ce00 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5808b130 con 0x7f5f5808b7f0 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f5808bd60 con 0x7f5f5808d8e0 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5f768640 10 monclient(hunting): authenticate will time out at 2230.675631s 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5d4dd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5f5808ce00 0x7f5f5808d1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60492/0 (socket says 10.20.193.102:60492) 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5d4dd640 1 -- 10.20.193.102:0/3441025463 learned_addr learned my addr 10.20.193.102:0/3441025463 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:41.443 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.434+0000 7f5f5dcde640 1 --1- 10.20.193.102:0/3441025463 >> v1:10.20.193.102:6790/0 conn(0x7f5f5808d8e0 0x7f5f581a3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=3 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:56694/0 (socket says 10.20.193.102:56694) 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5d4dd640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.102:6789/0 conn(0x7f5f5808ce00 legacy=0x7f5f5808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5dcde640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.102:6790/0 conn(0x7f5f5808d8e0 legacy=0x7f5f581a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 507158279 0 0) 0x7f5f58086a10 con 0x7f5f5808ce00 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): my global_id is 35010 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f40003620 con 0x7f5f5808ce00 2026-02-21T22:33:41.450 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5ccdc640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.178:6789/0 conn(0x7f5f5808b7f0 legacy=0x7f5f58181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560824173 0 0) 0x7f5f5808bd60 con 0x7f5f5808d8e0 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient(hunting): my global_id is 94493 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f58086a10 con 0x7f5f5808d8e0 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2305361929 0 0) 0x7f5f58086a10 con 0x7f5f5808d8e0 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.178:6789/0 conn(0x7f5f5808b7f0 legacy=0x7f5f58181e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.102:6789/0 conn(0x7f5f5808ce00 legacy=0x7f5f5808d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: _finish_hunting 0 2026-02-21T22:33:41.451 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 monclient: found mon.noname-b 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: _send_mon_message to mon.noname-b at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5808b370 con 0x7f5f5808d8e0 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: _finish_auth 0 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: _check_auth_tickets 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5f54004620 con 0x7f5f5808d8e0 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:41.452 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: dump: 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.453 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 5 monclient: authenticate success, global_id 94493 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.2 v1:10.20.193.102:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5f540048f0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 1 -- 10.20.193.102:0/3441025463 <== mon.2 v1:10.20.193.102:6790/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5f54005fa0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f47fff640 10 monclient: dump: 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.454 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 4 set_mon_vals no callback set 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: shutdown 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:41.455 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- 10.20.193.102:0/3441025463 >> v1:10.20.193.102:6790/0 conn(0x7f5f5808d8e0 legacy=0x7f5f581a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- 10.20.193.102:0/3441025463 shutdown_connections 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- 10.20.193.102:0/3441025463 >> 10.20.193.102:0/3441025463 conn(0x7f5f58091a50 msgr2=0x7f5f58091e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- 10.20.193.102:0/3441025463 shutdown_connections 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- 10.20.193.102:0/3441025463 wait complete. 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: build_initial_monmap 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: monmap: 2026-02-21T22:33:41.456 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 Processor -- start 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- start start 2026-02-21T22:33:41.457 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: init 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-a con 0x7f5f5808b7f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-b con 0x7f5f5808ce00 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient(hunting): picked mon.noname-c con 0x7f5f5808d8e0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f58080dc0 con 0x7f5f5808b7f0 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f58082880 con 0x7f5f5808ce00 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f58082b60 con 0x7f5f5808d8e0 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5f768640 10 monclient(hunting): authenticate will time out at 2230.679835s 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5d4dd640 1 --1- >> v1:10.20.193.102:6789/0 conn(0x7f5f5808b7f0 0x7f5f58153190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=1 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6789/0 says I am v1:10.20.193.102:60498/0 (socket says 10.20.193.102:60498) 2026-02-21T22:33:41.458 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.438+0000 7f5f5d4dd640 1 -- 10.20.193.102:0/204816617 learned_addr learned my addr 10.20.193.102:0/204816617 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5d4dd640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.102:6789/0 conn(0x7f5f5808b7f0 legacy=0x7f5f58153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5dcde640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.102:6790/0 conn(0x7f5f5808d8e0 legacy=0x7f5f58195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444462179 0 0) 0x7f5f58082b60 con 0x7f5f5808d8e0 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): my global_id is 94496 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f28003620 con 0x7f5f5808d8e0 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355012805 0 0) 0x7f5f58080dc0 con 0x7f5f5808b7f0 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.459 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): my global_id is 35013 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f58082b60 con 0x7f5f5808b7f0 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5ccdc640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.178:6789/0 conn(0x7f5f5808ce00 legacy=0x7f5f581944c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4196942161 0 0) 0x7f5f28003620 con 0x7f5f5808d8e0 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5f58080dc0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5f54005df0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.460 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230496106 0 0) 0x7f5f58082880 con 0x7f5f5808ce00 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.461 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): my global_id is 134176 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f28003620 con 0x7f5f5808ce00 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 74504311 0 0) 0x7f5f58082b60 con 0x7f5f5808b7f0 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5f58082880 con 0x7f5f5808b7f0 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5f480048e0 con 0x7f5f5808b7f0 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient(hunting): dump: 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.462 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1546059792 0 0) 0x7f5f58080dc0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.178:6789/0 conn(0x7f5f5808ce00 legacy=0x7f5f581944c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.102:6789/0 conn(0x7f5f5808b7f0 legacy=0x7f5f58153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: _finish_hunting 0 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 monclient: found mon.c 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.463 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f58081920 con 0x7f5f5808d8e0 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: _finish_auth 0 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: _check_auth_tickets 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 5 monclient: authenticate success, global_id 94496 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _renew_subs 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5f58083b00 con 0x7f5f5808d8e0 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _renew_subs 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.464 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5f58080620 con 0x7f5f5808d8e0 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5f540048c0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f5f54006770 con 0x7f5f5808d8e0 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 10 monclient: dump: 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.465 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f477fe640 4 set_mon_vals no callback set 2026-02-21T22:33:41.466 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f580008e0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f5f54012e00 con 0x7f5f5808d8e0 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f5f5404d900 con 0x7f5f5808d8e0 2026-02-21T22:33:41.467 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.442+0000 7f5f5d4dd640 10 monclient: get_auth_request con 0x7f5f2803d8c0 auth_method 0 2026-02-21T22:33:41.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.446+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f5f54018ac0 con 0x7f5f5808d8e0 2026-02-21T22:33:41.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.446+0000 7f5f45ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.468 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.446+0000 7f5f45ffb640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:41.531 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.526+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 --> v1:10.20.193.178:6800/2549949439 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f58088a70 con 0x7f5f2803d8c0 2026-02-21T22:33:41.532 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f45ffb640 1 -- 10.20.193.102:0/204816617 <== mgr.4105 v1:10.20.193.178:6800/2549949439 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+34088 (unknown 2965378022 0 2885844549) 0x7f5f58088a70 con 0x7f5f2803d8c0 2026-02-21T22:33:41.533 INFO:teuthology.orchestra.run.trial102.stdout: 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:dumped all 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.178:6800/2549949439 conn(0x7f5f2803d8c0 legacy=0x7f5f2805dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 10 monclient: shutdown 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 >> v1:10.20.193.102:6790/0 conn(0x7f5f5808d8e0 legacy=0x7f5f58195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 shutdown_connections 2026-02-21T22:33:41.534 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 >> 10.20.193.102:0/204816617 conn(0x7f5f58091a50 msgr2=0x7f5f5805d590 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:41.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 shutdown_connections 2026-02-21T22:33:41.535 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.530+0000 7f5f5f768640 1 -- 10.20.193.102:0/204816617 wait complete. 2026-02-21T22:33:41.542 INFO:teuthology.orchestra.run.trial102.stdout:{"pg_ready":true,"pg_map":{"version":1379,"stamp":"2026-02-21T22:33:39.619242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":212704,"kb_used_data":14848,"kb_used_omap":8260,"kb_used_meta":189499,"kb_avail":566018336,"statfs":{"total":579820584960,"available":579602776064,"internally_reserved":0,"allocated":15204352,"data_stored":9432172,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":8458769,"internal_metadata":194047471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001016"},"pg_stats":[{"pgid":"1.0","version":"831'114","reported_seq":1770,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:21.085998+0000","last_change":"2026-02-21T22:33:21.085954+0000","last_active":"2026-02-21T22:33:21.085998+0000","last_peered":"2026-02-21T22:33:21.085998+0000","last_clean":"2026-02-21T22:33:21.085998+0000","last_became_active":"2026-02-21T22:15:31.430260+0000","last_became_peered":"2026-02-21T22:15:31.430260+0000","last_unstale":"2026-02-21T22:33:21.085998+0000","last_undegraded":"2026-02-21T22:33:21.085998+0000","last_fullsized":"2026-02-21T22:33:21.085998+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"831'114","last_scrub_stamp":"2026-02-21T22:33:21.085899+0000","last_deep_scrub":"831'114","last_deep_scrub_stamp":"2026-02-21T22:33:21.085899+0000","last_clean_scrub_stamp":"2026-02-21T22:33:21.085899+0000","objects_scrubbed":2,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:13:35.582497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"831'2","reported_seq":1646,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:22.365519+0000","last_change":"2026-02-21T22:33:22.365519+0000","last_active":"2026-02-21T22:33:22.365519+0000","last_peered":"2026-02-21T22:33:22.365519+0000","last_clean":"2026-02-21T22:33:22.365519+0000","last_became_active":"2026-02-21T22:15:34.533992+0000","last_became_peered":"2026-02-21T22:15:34.533992+0000","last_unstale":"2026-02-21T22:33:22.365519+0000","last_undegraded":"2026-02-21T22:33:22.365519+0000","last_fullsized":"2026-02-21T22:33:22.365519+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"831'2","last_scrub_stamp":"2026-02-21T22:33:22.365475+0000","last_deep_scrub":"831'2","last_deep_scrub_stamp":"2026-02-21T22:33:22.365475+0000","last_clean_scrub_stamp":"2026-02-21T22:33:22.365475+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:20:46.361394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036143200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1630,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:23.047237+0000","last_change":"2026-02-21T22:33:23.047237+0000","last_active":"2026-02-21T22:33:23.047237+0000","last_peered":"2026-02-21T22:33:23.047237+0000","last_clean":"2026-02-21T22:33:23.047237+0000","last_became_active":"2026-02-21T22:15:34.533971+0000","last_became_peered":"2026-02-21T22:15:34.533971+0000","last_unstale":"2026-02-21T22:33:23.047237+0000","last_undegraded":"2026-02-21T22:33:23.047237+0000","last_fullsized":"2026-02-21T22:33:23.047237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:23.047171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:23.047171+0000","last_clean_scrub_stamp":"2026-02-21T22:33:23.047171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:20:25.642441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473902,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1632,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:20.586085+0000","last_change":"2026-02-21T22:33:20.586085+0000","last_active":"2026-02-21T22:33:20.586085+0000","last_peered":"2026-02-21T22:33:20.586085+0000","last_clean":"2026-02-21T22:33:20.586085+0000","last_became_active":"2026-02-21T22:15:34.531002+0000","last_became_peered":"2026-02-21T22:15:34.531002+0000","last_unstale":"2026-02-21T22:33:20.586085+0000","last_undegraded":"2026-02-21T22:33:20.586085+0000","last_fullsized":"2026-02-21T22:33:20.586085+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:20.586025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:20.586025+0000","last_clean_scrub_stamp":"2026-02-21T22:33:20.586025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T23:51:02.563999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036968599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"831'3","reported_seq":1647,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:21.347520+0000","last_change":"2026-02-21T22:33:21.347520+0000","last_active":"2026-02-21T22:33:21.347520+0000","last_peered":"2026-02-21T22:33:21.347520+0000","last_clean":"2026-02-21T22:33:21.347520+0000","last_became_active":"2026-02-21T22:15:34.533274+0000","last_became_peered":"2026-02-21T22:15:34.533274+0000","last_unstale":"2026-02-21T22:33:21.347520+0000","last_undegraded":"2026-02-21T22:33:21.347520+0000","last_fullsized":"2026-02-21T22:33:21.347520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"831'3","last_scrub_stamp":"2026-02-21T22:33:21.347473+0000","last_deep_scrub":"831'3","last_deep_scrub_stamp":"2026-02-21T22:33:21.347473+0000","last_clean_scrub_stamp":"2026-02-21T22:33:21.347473+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:07:58.579848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087316399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1639,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:20.241762+0000","last_change":"2026-02-21T22:33:20.241762+0000","last_active":"2026-02-21T22:33:20.241762+0000","last_peered":"2026-02-21T22:33:20.241762+0000","last_clean":"2026-02-21T22:33:20.241762+0000","last_became_active":"2026-02-21T22:15:34.537194+0000","last_became_peered":"2026-02-21T22:15:34.537194+0000","last_unstale":"2026-02-21T22:33:20.241762+0000","last_undegraded":"2026-02-21T22:33:20.241762+0000","last_fullsized":"2026-02-21T22:33:20.241762+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:20.241701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:20.241701+0000","last_clean_scrub_stamp":"2026-02-21T22:33:20.241701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:48:34.560947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027795599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1630,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:22.091176+0000","last_change":"2026-02-21T22:33:22.091176+0000","last_active":"2026-02-21T22:33:22.091176+0000","last_peered":"2026-02-21T22:33:22.091176+0000","last_clean":"2026-02-21T22:33:22.091176+0000","last_became_active":"2026-02-21T22:15:34.539501+0000","last_became_peered":"2026-02-21T22:15:34.539501+0000","last_unstale":"2026-02-21T22:33:22.091176+0000","last_undegraded":"2026-02-21T22:33:22.091176+0000","last_fullsized":"2026-02-21T22:33:22.091176+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:22.091100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:22.091100+0000","last_clean_scrub_stamp":"2026-02-21T22:33:22.091100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:43:22.277483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1639,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:21.281953+0000","last_change":"2026-02-21T22:33:21.281953+0000","last_active":"2026-02-21T22:33:21.281953+0000","last_peered":"2026-02-21T22:33:21.281953+0000","last_clean":"2026-02-21T22:33:21.281953+0000","last_became_active":"2026-02-21T22:15:34.531900+0000","last_became_peered":"2026-02-21T22:15:34.531900+0000","last_unstale":"2026-02-21T22:33:21.281953+0000","last_undegraded":"2026-02-21T22:33:21.281953+0000","last_fullsized":"2026-02-21T22:33:21.281953+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:21.281894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:21.281894+0000","last_clean_scrub_stamp":"2026-02-21T22:33:21.281894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:06:18.861338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031443399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1630,"reported_epoch":831,"state":"active+clean","last_fresh":"2026-02-21T22:33:24.075129+0000","last_change":"2026-02-21T22:33:24.075129+0000","last_active":"2026-02-21T22:33:24.075129+0000","last_peered":"2026-02-21T22:33:24.075129+0000","last_clean":"2026-02-21T22:33:24.075129+0000","last_became_active":"2026-02-21T22:15:34.539165+0000","last_became_peered":"2026-02-21T22:15:34.539165+0000","last_unstale":"2026-02-21T22:33:24.075129+0000","last_undegraded":"2026-02-21T22:33:24.075129+0000","last_fullsized":"2026-02-21T22:33:24.075129+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T22:33:24.075067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T22:33:24.075067+0000","last_clean_scrub_stamp":"2026-02-21T22:33:24.075067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:59:02.130723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043782400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":16334,"data_compressed_allocated":106496,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":12,"seq":51539607773,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29412,"kb_used_data":2452,"kb_used_omap":1172,"kb_used_meta":25771,"kb_avail":94342428,"statfs":{"total":96636764160,"available":96606646272,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1200561,"internal_metadata":26390095},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640477,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2456,"kb_used_omap":1308,"kb_used_meta":25635,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1340101,"internal_metadata":26250555},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":854,"kb_used_meta":26089,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":874656,"internal_metadata":26716000},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640476,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":2452,"kb_used_omap":1802,"kb_used_meta":43189,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":2510848,"data_stored":1375233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1845358,"internal_metadata":44226450},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640477,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47464,"kb_used_data":2456,"kb_used_omap":1841,"kb_used_meta":43150,"kb_avail":94324376,"statfs":{"total":96636764160,"available":96588161024,"internally_reserved":0,"allocated":2514944,"data_stored":1375252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1885730,"internal_metadata":44186078},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29476,"kb_used_data":2516,"kb_used_omap":1281,"kb_used_meta":25662,"kb_avail":94342364,"statfs":{"total":96636764160,"available":96606580736,"internally_reserved":0,"allocated":2576384,"data_stored":1965601,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":1312363,"internal_metadata":26278293},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":8167,"data_compressed_allocated":53248,"data_compressed_original":581632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T22:33:41.545 DEBUG:teuthology.orchestra.run.trial102:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: get_monmap_and_config 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: build_initial_monmap 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: monmap: 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.606 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:33:41.607521+0000 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:33:41.607521+0000 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-c 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-b 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 1 Processor -- start 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 1 -- start start 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: init 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:41.607 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-c con 0x7f65c4084760 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-b con 0x7f65c408c3c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-a con 0x7f65c41855f0 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c4089860 con 0x7f65c41855f0 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c408a440 con 0x7f65c4084760 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c408a720 con 0x7f65c408c3c0 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c8f6d640 10 monclient(hunting): authenticate will time out at 2230.843612s 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c1d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 0x7f65c4183ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:56718/0 (socket says 10.20.193.102:56718) 2026-02-21T22:33:41.608 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.602+0000 7f65c1d74640 1 -- 10.20.193.102:0/3217681461 learned_addr learned my addr 10.20.193.102:0/3217681461 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:41.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1d74640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 legacy=0x7f65c4183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.609 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c2d76640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.102:6789/0 conn(0x7f65c41855f0 legacy=0x7f65c41a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898461366 0 0) 0x7f65c4089860 con 0x7f65c41855f0 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): my global_id is 35016 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65b0003620 con 0x7f65c41855f0 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250653968 0 0) 0x7f65c408a720 con 0x7f65c408c3c0 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.613 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient(hunting): my global_id is 94499 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c4089860 con 0x7f65c408c3c0 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c2575640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.178:6789/0 conn(0x7f65c4084760 legacy=0x7f65c4084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3891146512 0 0) 0x7f65b0003620 con 0x7f65c41855f0 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 legacy=0x7f65c4183ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.178:6789/0 conn(0x7f65c4084760 legacy=0x7f65c4084b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: _finish_hunting 0 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 monclient: found mon.noname-a 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: _send_mon_message to mon.noname-a at v1:10.20.193.102:6789/0 2026-02-21T22:33:41.614 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 --> v1:10.20.193.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4085240 con 0x7f65c41855f0 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: _finish_auth 0 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: _check_auth_tickets 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f65b40045a0 con 0x7f65c41855f0 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: dump: 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.615 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 5 monclient: authenticate success, global_id 35016 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c2d76640 1 --1- 10.20.193.102:0/3217681461 >> v1:10.20.193.102:6789/0 conn(0x7f65c41855f0 0x7f65c41a59d0 :-1 s=OPENED pgs=719 gs=3 cs=1 l=1).queue_received will delay after 1.77171e+09 on 0x7f65b4004c20 mon_map magic: 0 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.0 v1:10.20.193.102:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f65b4004870 con 0x7f65c41855f0 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 10 monclient: get_monmap_and_config success 2026-02-21T22:33:41.616 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 4 set_mon_vals no callback set 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 10 monclient: shutdown 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3217681461 >> v1:10.20.193.102:6789/0 conn(0x7f65c41855f0 legacy=0x7f65c41a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 1 -- 10.20.193.102:0/3217681461 <== mon.0 v1:10.20.193.102:6789/0 5 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f65b4004c20 con 0x7f65c41855f0 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c1573640 10 monclient: discarding stray monitor message mon_map magic: 0 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3217681461 shutdown_connections 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3217681461 >> 10.20.193.102:0/3217681461 conn(0x7f65c408e470 msgr2=0x7f65c408e870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3217681461 shutdown_connections 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3217681461 wait complete. 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 10 monclient: build_initial_monmap 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 10 monclient: monmap: 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:epoch 0 2026-02-21T22:33:41.617 INFO:teuthology.orchestra.run.trial102.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 0.000000 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:created 0.000000 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 0 (unknown) 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 1 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.noname-a 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.noname-b 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.noname-c 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 Processor -- start 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 1 -- start start 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.606+0000 7f65c8f6d640 10 monclient: init 2026-02-21T22:33:41.618 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _reopen_session rank -1 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-a con 0x7f65c4084760 addr v1:10.20.193.102:6789/0 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-c con 0x7f65c408c3c0 addr v1:10.20.193.102:6790/0 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient(hunting): picked mon.noname-b con 0x7f65c41855f0 addr v1:10.20.193.178:6789/0 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- --> v1:10.20.193.102:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c40840e0 con 0x7f65c4084760 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- --> v1:10.20.193.178:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c4088990 con 0x7f65c41855f0 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- --> v1:10.20.193.102:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c407e880 con 0x7f65c408c3c0 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient(hunting): _renew_subs 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient(hunting): authenticate will time out at 2230.848131s 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c1d74640 1 --1- >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 0x7f65c4172e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 gs=2 cs=0 l=1).handle_server_banner_and_identify peer v1:10.20.193.102:6790/0 says I am v1:10.20.193.102:56732/0 (socket says 10.20.193.102:56732) 2026-02-21T22:33:41.619 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c1d74640 1 -- 10.20.193.102:0/3802448335 learned_addr learned my addr 10.20.193.102:0/3802448335 (peer_addr_for_me v1:10.20.193.102:0/0) 2026-02-21T22:33:41.623 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c1d74640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 legacy=0x7f65c4172e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c2575640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.102:6789/0 conn(0x7f65c4084760 legacy=0x7f65c4131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1441179240 0 0) 0x7f65c407e880 con 0x7f65c408c3c0 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): my global_id is 94502 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6590003620 con 0x7f65c408c3c0 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.0 v1:10.20.193.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386770734 0 0) 0x7f65c40840e0 con 0x7f65c4084760 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.624 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): my global_id is 35019 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c407e880 con 0x7f65c4084760 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c2d76640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.178:6789/0 conn(0x7f65c41855f0 legacy=0x7f65c41745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).operator() setting up a delay queue 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2239512863 0 0) 0x7f6590003620 con 0x7f65c408c3c0 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f65c40840e0 con 0x7f65c408c3c0 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f65ac003090 con 0x7f65c408c3c0 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): dump: 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.625 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _finish_auth 0 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.0 v1:10.20.193.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3392271098 0 0) 0x7f65c407e880 con 0x7f65c4084760 2026-02-21T22:33:41.626 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6590003620 con 0x7f65c4084760 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.0 v1:10.20.193.102:6789/0 3 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f65b8003380 con 0x7f65c4084760 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): dump: 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.627 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.1 v1:10.20.193.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829979833 0 0) 0x7f65c4088990 con 0x7f65c41855f0 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _negotiate 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth method 2 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient(hunting): my global_id is 134182 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.178:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c407e880 con 0x7f65c41855f0 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3211565479 0 0) 0x7f65c40840e0 con 0x7f65c408c3c0 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.178:6789/0 conn(0x7f65c41855f0 legacy=0x7f65c41745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.102:6789/0 conn(0x7f65c4084760 legacy=0x7f65c4131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.628 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: _finish_hunting 0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 monclient: found mon.c 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4080be0 con 0x7f65c408c3c0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: _finish_auth 0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: _check_auth_tickets 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 5 monclient: authenticate success, global_id 94502 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _renew_subs 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f65c4080340 con 0x7f65c408c3c0 2026-02-21T22:33:41.629 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _renew_subs 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f65c40815e0 con 0x7f65c408c3c0 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f65ac004b90 con 0x7f65c408c3c0 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: handle_config config(0 keys) 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 6 ==== mon_map magic: 0 ==== 332+0+0 (unknown 3142309550 0 0) 0x7f65ac004ef0 con 0x7f65c408c3c0 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c0d72640 4 set_mon_vals no callback set 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 10 monclient: dump: 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:epoch 1 2026-02-21T22:33:41.630 INFO:teuthology.orchestra.run.trial102.stderr:fsid 740d24fb-83fc-48ed-963d-a0ec1185b484 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:last_changed 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:created 2026-02-21T22:15:04.891122+0000 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:min_mon_release 20 (tentacle) 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:election_strategy: 3 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:0: v1:10.20.193.102:6789/0 mon.a 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:1: v1:10.20.193.178:6789/0 mon.b 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:2: v1:10.20.193.102:6790/0 mon.c 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.631 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c40008e0 con 0x7f65c408c3c0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.610+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 7 ==== mgrmap(e 5) ==== 50408+0+0 (unknown 3065122959 0 0) 0x7f65ac005180 con 0x7f65c408c3c0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.614+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 8 ==== osd_map(831..831 src has 681..831) ==== 3579+0+0 (unknown 3173632890 0 0) 0x7f65ac04c630 con 0x7f65c408c3c0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.614+0000 7f65c2575640 10 monclient: get_auth_request con 0x7f659003d890 auth_method 0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.614+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+204907 (unknown 1092875540 0 3942553110) 0x7f65ac0176d0 con 0x7f65c408c3c0 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.614+0000 7f65aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-21T22:33:41.632 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.614+0000 7f65aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-21T22:33:41.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.698+0000 7f65c8f6d640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-21T22:33:41.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.698+0000 7f65c8f6d640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-21T22:33:41.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.698+0000 7f65c8f6d640 10 monclient: _send_mon_message to mon.c at v1:10.20.193.102:6790/0 2026-02-21T22:33:41.702 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.698+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 --> v1:10.20.193.102:6790/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f65c4090810 con 0x7f65c408c3c0 2026-02-21T22:33:41.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 10 ==== config(1 keys) ==== 35+0+0 (unknown 2555425225 0 0) 0x7f65ac006820 con 0x7f65c408c3c0 2026-02-21T22:33:41.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65aaffd640 10 monclient: handle_config config(1 keys) 2026-02-21T22:33:41.705 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c0d72640 4 set_mon_vals no callback set 2026-02-21T22:33:41.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c0d72640 10 set_mon_vals mon_health_to_clog = false 2026-02-21T22:33:41.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65aaffd640 1 -- 10.20.193.102:0/3802448335 <== mon.2 v1:10.20.193.102:6790/0 11 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v3)=0 v3) ==== 123+0+0 (unknown 3022875181 0 0) 0x7f65ac01c9f0 con 0x7f65c408c3c0 2026-02-21T22:33:41.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-21T22:33:41.706 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65aaffd640 10 monclient: _finish_command 2 = system:0 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.178:6800/2549949439 conn(0x7f659003d890 legacy=0x7f659005dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 10 monclient: shutdown 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 >> v1:10.20.193.102:6790/0 conn(0x7f65c408c3c0 legacy=0x7f65c4172e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 shutdown_connections 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 >> 10.20.193.102:0/3802448335 conn(0x7f65c408e470 msgr2=0x7f65c405bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 shutdown_connections 2026-02-21T22:33:41.707 INFO:teuthology.orchestra.run.trial102.stderr:2026-02-21T22:33:41.702+0000 7f65c8f6d640 1 -- 10.20.193.102:0/3802448335 wait complete. 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: max_completed_events = 50 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 5 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = True 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: allow_pg_recovery_event = False 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_target_pg_per_osd = 100 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: native option mon_max_pg_per_osd = 10000 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sleep_interval = 60 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option threshold = 3.0 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:33:41.710 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: url = https://telemetry.ceph.com/report 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enable_monitoring = True 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: device_url = https://telemetry.ceph.com/device 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: scrape_frequency = 86400 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option warn_recent_interval = 1209600 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option max_concurrent_clones = 4 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: enabled = False 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: pool_name = device_health_metrics 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option retain_interval = 31536000 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_delay = 0 2026-02-21T22:33:41.711 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: last_opt_revision = 1 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: retention_period = 15552000 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option periodic_async_work = False 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard = False 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mark_out_threshold = 2419200 2026-02-21T22:33:41.713 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: leaderboard_description = None 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option snapshot_clone_no_wait = True 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: warn_threshold = 7257600 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: description = None 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_purging = False 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: self_heal = True 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: contact = None 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:processing 0 volume entries 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sleep_interval = 600 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: organization = None 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option pause_cloning = False 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: proxy = None 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:volumes.fs.async_job:resume() no need to resume, run_event is already set. 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: interval = 24 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_level = 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_basic = True 2026-02-21T22:33:41.714 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_file = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_ident = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option sqlite3_killpoint = 0 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_crash = True 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_device = True 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: mgr option log_to_cluster_level = info 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: channel_perf = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_level = 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_file = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: sqlite3_killpoint = 0 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster = False 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root: log_to_cluster_level = info 2026-02-21T22:33:41.715 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:Not sending report until user re-opts-in 2026-02-21T22:33:41.717 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T22:33:41.717 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T22:33:41.717 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T22:33:41.717 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:41.784 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T22:33:41.784 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T22:33:41.784 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:41.871 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T22:33:41.872 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T22:33:41.872 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:43.599 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:PerfHandler: tick 2026-02-21T22:33:43.599 INFO:tasks.ceph.mgr.x.trial178.stderr:DEBUG:root:TaskHandler: tick 2026-02-21T22:33:46.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-21T22:33:47.974 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T22:33:47.974 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-21T22:33:47.974 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.047 INFO:tasks.ceph.osd.3:Stopped 2026-02-21T22:33:48.048 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-21T22:33:48.048 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.134 INFO:tasks.ceph.osd.4:Stopped 2026-02-21T22:33:48.134 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-21T22:33:48.134 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.219 INFO:tasks.ceph.osd.5:Stopped 2026-02-21T22:33:48.219 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T22:33:48.219 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T22:33:48.219 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.234 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T22:33:48.234 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T22:33:48.234 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T22:33:48.234 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.256 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T22:33:48.256 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-21T22:33:48.256 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.314 INFO:tasks.ceph.mon.c:Stopped 2026-02-21T22:33:48.314 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-21T22:33:48.314 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T22:33:48.325 INFO:tasks.ceph.mon.b:Stopped 2026-02-21T22:33:48.326 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T22:33:48.326 DEBUG:teuthology.orchestra.run.trial102:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v ScrubResult | grep -E -v 'scrub mismatch' | grep -E -v 'overall HEALTH_' | grep -E -v '\(MGR_DOWN\)' | grep -E -v '\(PG_AVAILABILITY\)' | grep -E -v '\(SLOW_OPS\)' | grep -E -v 'slow request' | grep -E -v '\(OSD_SLOW_PING_TIME' | grep -E -v '\(MON_NETSPLIT)' | grep -E -v 'overall HEALTH_' | grep -E -v '\(MON_DOWN\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-21T22:33:48.366 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:33:48.366 DEBUG:teuthology.orchestra.run.trial102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T22:33:48.419 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:33:48.420 DEBUG:teuthology.orchestra.run.trial102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T22:33:48.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:33:48.477 DEBUG:teuthology.orchestra.run.trial102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T22:33:48.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:33:48.533 DEBUG:teuthology.orchestra.run.trial178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-21T22:33:48.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:33:48.548 DEBUG:teuthology.orchestra.run.trial178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-21T22:33:48.603 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:33:48.603 DEBUG:teuthology.orchestra.run.trial178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-21T22:33:48.659 INFO:tasks.ceph:Archiving mon data... 2026-02-21T22:33:48.660 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/data/mon.a.tgz 2026-02-21T22:33:48.661 DEBUG:teuthology.orchestra.run.trial102:> mktemp 2026-02-21T22:33:48.666 INFO:teuthology.orchestra.run.trial102.stdout:/tmp/tmp.7WNMwKvr9x 2026-02-21T22:33:48.666 DEBUG:teuthology.orchestra.run.trial102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.7WNMwKvr9x 2026-02-21T22:33:48.740 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod 0666 /tmp/tmp.7WNMwKvr9x 2026-02-21T22:33:48.836 DEBUG:teuthology.orchestra.remote:trial102:/tmp/tmp.7WNMwKvr9x is 276KB 2026-02-21T22:33:48.893 DEBUG:teuthology.orchestra.run.trial102:> rm -fr /tmp/tmp.7WNMwKvr9x 2026-02-21T22:33:48.897 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/data/mon.c.tgz 2026-02-21T22:33:48.898 DEBUG:teuthology.orchestra.run.trial102:> mktemp 2026-02-21T22:33:48.943 INFO:teuthology.orchestra.run.trial102.stdout:/tmp/tmp.GuMXc1lS1u 2026-02-21T22:33:48.944 DEBUG:teuthology.orchestra.run.trial102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.GuMXc1lS1u 2026-02-21T22:33:49.020 DEBUG:teuthology.orchestra.run.trial102:> sudo chmod 0666 /tmp/tmp.GuMXc1lS1u 2026-02-21T22:33:49.121 DEBUG:teuthology.orchestra.remote:trial102:/tmp/tmp.GuMXc1lS1u is 264KB 2026-02-21T22:33:49.177 DEBUG:teuthology.orchestra.run.trial102:> rm -fr /tmp/tmp.GuMXc1lS1u 2026-02-21T22:33:49.182 DEBUG:teuthology.misc:Transferring archived files from trial178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/data/mon.b.tgz 2026-02-21T22:33:49.182 DEBUG:teuthology.orchestra.run.trial178:> mktemp 2026-02-21T22:33:49.187 INFO:teuthology.orchestra.run.trial178.stdout:/tmp/tmp.js8ZNjmAQl 2026-02-21T22:33:49.187 DEBUG:teuthology.orchestra.run.trial178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.js8ZNjmAQl 2026-02-21T22:33:49.270 DEBUG:teuthology.orchestra.run.trial178:> sudo chmod 0666 /tmp/tmp.js8ZNjmAQl 2026-02-21T22:33:49.363 DEBUG:teuthology.orchestra.remote:trial178:/tmp/tmp.js8ZNjmAQl is 254KB 2026-02-21T22:33:49.412 DEBUG:teuthology.orchestra.run.trial178:> rm -fr /tmp/tmp.js8ZNjmAQl 2026-02-21T22:33:49.416 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T22:33:49.416 DEBUG:teuthology.orchestra.run.trial102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T22:33:49.417 DEBUG:teuthology.orchestra.run.trial178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T22:33:49.471 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T22:33:49.510 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T22:33:49.510 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T22:33:49.512 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial102/crash 2026-02-21T22:33:49.512 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T22:33:49.524 DEBUG:teuthology.misc:Transferring archived files from trial178:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial178/crash 2026-02-21T22:33:49.525 DEBUG:teuthology.orchestra.run.trial178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T22:33:49.558 INFO:tasks.ceph:Compressing logs... 2026-02-21T22:33:49.558 DEBUG:teuthology.orchestra.run.trial102:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:33:49.568 DEBUG:teuthology.orchestra.run.trial178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:33:49.575 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18832.log 2026-02-21T22:33:49.575 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21857.log 2026-02-21T22:33:49.576 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20522.log 2026-02-21T22:33:49.576 INFO:teuthology.orchestra.run.trial102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.18832.log.gz 2026-02-21T22:33:49.576 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19467.log 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22144.log 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20522.log: /var/log/ceph/ceph-client.admin.19467.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21857.log.gz 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21683.log 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.19467.log.gz 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.20522.log.gz 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23009.log 2026-02-21T22:33:49.581 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22118.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.22144.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21735.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21683.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23009.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21291.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22317.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21735.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.22118.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22343.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.21291.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.21735.log.gz 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23165.log 2026-02-21T22:33:49.582 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18992.log 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.22317.log.gz 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23113.log 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23165.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22343.log.gz 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20293.log 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.23165.log.gz 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20652.log 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.18992.log.gz 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21422.log 2026-02-21T22:33:49.583 INFO:teuthology.orchestra.run.trial102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23113.log.gz 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21961.log 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.20293.log.gz 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18836.log 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21422.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.20652.log.gz 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22656.log 2026-02-21T22:33:49.598 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.21422.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21961.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19902.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.18836.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16060.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21987.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19902.log: /var/log/ceph/ceph-client.admin.16060.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.22656.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22065.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.16060.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.19902.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19718.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18989.log 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22065.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.21987.log.gz 2026-02-21T22:33:49.599 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19336.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.22065.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18834.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.19718.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19336.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.18989.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22603.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20756.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18834.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.19336.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23373.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22603.log: /var/log/ceph/ceph-client.admin.20756.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.18834.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20704.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.22603.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.20756.log.gz 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21090.log 2026-02-21T22:33:49.600 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19692.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23373.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.20704.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.21090.log: -- /var/log/ceph/ceph-client.admin.21343.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23347.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.21090.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22708.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.19692.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23347.log: gzip 87.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.21343.log.gz --verbose 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: -- /var/log/ceph/ceph-client.admin.23503.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22708.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23347.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19366.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.22708.log.gz 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22957.log 2026-02-21T22:33:49.601 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23529.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.23503.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19366.log: /var/log/ceph/ceph-client.admin.22957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20626.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.22957.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.19366.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18835.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22577.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.23529.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18835.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.20626.log.gz /var/log/ceph/ceph-client.admin.22187.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22577.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.18835.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23451.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18833.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22187.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.22577.log.gz 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22930.log 2026-02-21T22:33:49.602 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.22187.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23451.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18988.log 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.18833.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.22930.log: -- /var/log/ceph/ceph-client.admin.19441.log 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18988.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.22930.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20189.log 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.18988.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20959.log/var/log/ceph/ceph-client.admin.19441.log: 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23087.log 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.20189.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.19441.log.gz 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19415.log 2026-02-21T22:33:49.603 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20959.log: /var/log/ceph/ceph-client.admin.23087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23269.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.20959.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.23087.log.gz -- 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: /var/log/ceph/ceph-client.admin.20241.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19415.log: /var/log/ceph/ceph-client.admin.23269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22760.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23269.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.19415.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22091.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.20241.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22395.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19180.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.22760.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22395.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22091.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22395.log.gz 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20267.log 2026-02-21T22:33:49.604 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21552.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19180.log: /var/log/ceph/ceph-client.admin.20267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23191.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.19180.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21552.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.20267.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20600.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23191.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21552.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19874.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23191.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18790.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.20600.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22473.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.19874.log.gz 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21883.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21474.log 2026-02-21T22:33:49.605 INFO:teuthology.orchestra.run.trial102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.18790.log.gz 2026-02-21T22:33:49.606 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21883.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.22473.log.gz 2026-02-21T22:33:49.606 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23477.log 2026-02-21T22:33:49.623 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-21T22:33:49.623 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.18531.log 2026-02-21T22:33:49.623 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-client.0.18531.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15953.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-client.admin.15953.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph.audit.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.15953.log.gz 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15977.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16001.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:gzip 89.4% -5 --verbose -- -- replaced with /var/log/ceph/ceph.audit.log.gz /var/log/ceph/ceph-client.admin.15804.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-client.admin.15977.log: /var/log/ceph/ceph-client.admin.16001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15851.log 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.15977.log.gz 2026-02-21T22:33:49.624 INFO:teuthology.orchestra.run.trial178.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.16001.log.gz 2026-02-21T22:33:49.625 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15780.log 2026-02-21T22:33:49.625 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-client.admin.15804.log: /var/log/ceph/ceph-client.admin.15851.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.15804.log.gz 2026-02-21T22:33:49.625 INFO:teuthology.orchestra.run.trial178.stderr:/var/log/ceph/ceph-client.admin.15780.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.15851.log.gz 2026-02-21T22:33:49.625 INFO:teuthology.orchestra.run.trial178.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.15780.log.gz 2026-02-21T22:33:49.625 INFO:teuthology.orchestra.run.trial178.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21474.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.21883.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23477.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12304.log 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21474.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23581.log 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23477.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21526.log 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph.tmp-client.admin.12304.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12304.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23035.log 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23581.log.gz 2026-02-21T22:33:49.632 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21500.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.21526.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22838.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.23035.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19981.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21500.log: /var/log/ceph/ceph-client.admin.22838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21369.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.21500.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19981.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.22838.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17209.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21369.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.19981.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19770.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18831.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.21369.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.17209.log: /var/log/ceph/ceph-client.admin.19770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21168.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.19770.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.17209.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19545.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.18831.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23557.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19545.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.21168.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23061.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23557.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.19545.log.gz 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21935.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21194.log 2026-02-21T22:33:49.633 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23061.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23557.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22904.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21194.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.23061.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.21935.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.20397.log -- replaced with /var/log/ceph/ceph-client.admin.21194.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18986.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20397.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.22904.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19258.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.20397.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19744.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18986.log: /var/log/ceph/ceph-client.admin.19258.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.19258.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22447.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.18986.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19232.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21909.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.19744.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.19232.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22213.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22447.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19128.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22213.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.21909.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22551.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22213.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18667.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.19128.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19284.log 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22551.log: /var/log/ceph/ceph-client.admin.18667.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.18667.log.gz 2026-02-21T22:33:49.634 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19955.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22551.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20111.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19284.log: /var/log/ceph/ceph-client.admin.19955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22682.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.19284.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.19955.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22786.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16015.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.20111.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22786.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22682.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23295.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.16015.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22786.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19206.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23295.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.16015.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20475.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23295.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19519.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20475.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.19206.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20894.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19519.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.20475.log.gz -5 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.20137.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20894.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.19519.log.gz -5 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.21604.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20137.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.20894.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19666.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21604.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.20137.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21011.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.21604.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22983.log 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.19666.log.gz 2026-02-21T22:33:49.635 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18692.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.21011.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18991.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18990.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18692.log: /var/log/ceph/ceph-client.admin.18991.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.22983.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19848.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.18692.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.18991.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23139.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20985.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19848.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.18990.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23139.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.19848.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23139.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21317.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20985.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20861.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.20985.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21317.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.21317.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18764.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-osd.2.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.20861.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21831.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-mon.c.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.18764.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22239.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21831.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.21831.log.gz 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20808.log 2026-02-21T22:33:49.636 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.22239.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21063.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23217.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21063.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.20808.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19928.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23217.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.21063.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23321.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23217.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22291.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20007.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.19928.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22291.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23321.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21630.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.22291.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20059.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21630.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.20007.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21116.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip/var/log/ceph/ceph-client.admin.20059.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.20678.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.21630.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20085.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21116.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.20059.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23243.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18738.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.20678.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23243.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.20085.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21396.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18738.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.23243.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22525.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18712.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.18738.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21766.log 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.21396.log.gz 2026-02-21T22:33:49.637 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18712.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.22525.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20215.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.18712.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21578.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19572.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20215.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.21766.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21037.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19572.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.20215.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23399.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.19572.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.21578.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18642.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.21037.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20730.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.18642.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23399.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.18642.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22265.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.20730.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22878.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19796.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22878.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22265.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22499.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22878.log.gz 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19154.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22369.log 2026-02-21T22:33:49.638 INFO:teuthology.orchestra.run.trial102.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.19796.log.gz 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.22499.log.gz 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20371.log 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22369.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.19154.log.gz 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22421.log 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20371.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22369.log.gz -5 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr: --verbose -- /var/log/ceph/ceph-client.admin.22812.log 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.20371.log.gz 2026-02-21T22:33:49.641 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19365.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22629.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.22421.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21142.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.22812.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22629.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.19365.log.gz --verbose 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: -- /var/log/ceph/ceph-client.admin.20345.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23425.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.22629.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20345.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.21142.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22013.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.20345.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.23425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20548.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22013.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-osd.1.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23425.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20548.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.22013.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21220.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.20548.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21657.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22734.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20782.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.21220.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19493.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.21657.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20782.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.20782.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.22734.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20163.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20449.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19493.log: /var/log/ceph/ceph-client.admin.20163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22039.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.19493.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20449.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.20163.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19310.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.20449.log.gz 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.22039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19822.log 2026-02-21T22:33:49.642 INFO:teuthology.orchestra.run.trial102.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.22039.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20835.log 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.19822.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.19310.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21709.log 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20319.log 89.2% 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.19822.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.21709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20423.log 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.20835.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20319.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.21709.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20033.log 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20423.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.20319.log.gz 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16102.log 2026-02-21T22:33:49.643 INFO:teuthology.orchestra.run.trial102.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.20423.log.gz 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21448.log 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.16102.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.20033.log.gz 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20574.log 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr: 88.4%/var/log/ceph/ceph-client.admin.21448.log: -- replaced with /var/log/ceph/ceph-client.admin.16102.log.gz 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/ceph-client.admin.20574.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.21448.log.gz 2026-02-21T22:33:49.644 INFO:teuthology.orchestra.run.trial102.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.20574.log.gz 2026-02-21T22:33:49.663 INFO:teuthology.orchestra.run.trial178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.0.18531.log.gz 2026-02-21T22:33:49.722 INFO:teuthology.orchestra.run.trial178.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T22:33:50.253 INFO:teuthology.orchestra.run.trial178.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-21T22:33:50.461 INFO:teuthology.orchestra.run.trial102.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-21T22:33:50.792 INFO:teuthology.orchestra.run.trial178.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-21T22:33:50.881 INFO:teuthology.orchestra.run.trial102.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T22:33:51.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-21T22:33:51.307 INFO:teuthology.orchestra.run.trial178.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-21T22:33:51.696 INFO:teuthology.orchestra.run.trial102.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T22:33:51.827 INFO:teuthology.orchestra.run.trial178.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-21T22:33:51.829 INFO:teuthology.orchestra.run.trial178.stderr: 2026-02-21T22:33:51.829 INFO:teuthology.orchestra.run.trial178.stderr:real 0m2.219s 2026-02-21T22:33:51.829 INFO:teuthology.orchestra.run.trial178.stderr:user 0m5.666s 2026-02-21T22:33:51.829 INFO:teuthology.orchestra.run.trial178.stderr:sys 0m0.230s 2026-02-21T22:33:52.266 INFO:teuthology.orchestra.run.trial102.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T22:33:52.840 INFO:teuthology.orchestra.run.trial102.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T22:33:52.841 INFO:teuthology.orchestra.run.trial102.stderr: 2026-02-21T22:33:52.841 INFO:teuthology.orchestra.run.trial102.stderr:real 0m3.272s 2026-02-21T22:33:52.841 INFO:teuthology.orchestra.run.trial102.stderr:user 0m9.766s 2026-02-21T22:33:52.841 INFO:teuthology.orchestra.run.trial102.stderr:sys 0m0.510s 2026-02-21T22:33:52.843 INFO:tasks.ceph:Archiving logs... 2026-02-21T22:33:52.843 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/log/ceph to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial102/log 2026-02-21T22:33:52.844 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T22:33:53.877 DEBUG:teuthology.misc:Transferring archived files from trial178:/var/log/ceph to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial178/log 2026-02-21T22:33:53.877 DEBUG:teuthology.orchestra.run.trial178:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T22:33:54.363 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T22:33:54.375 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T22:33:54.375 DEBUG:teuthology.orchestra.run.trial102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T22:33:54.376 DEBUG:teuthology.orchestra.run.trial178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T22:33:54.523 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T22:33:54.534 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T22:33:54.534 DEBUG:teuthology.orchestra.run.trial102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:33:54.536 DEBUG:teuthology.orchestra.run.trial178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T22:33:54.610 INFO:teuthology.orchestra.run.trial102.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T22:33:54.610 INFO:teuthology.orchestra.run.trial102.stdout:============================================================================== 2026-02-21T22:33:54.611 INFO:teuthology.orchestra.run.trial102.stdout:-198.23.133.146 45.79.214.107 3 u 106 128 377 43.582 -2.980 0.593 2026-02-21T22:33:54.611 INFO:teuthology.orchestra.run.trial102.stdout:+23.186.168.130 132.163.97.4 2 u 54 128 377 69.638 -2.121 1.535 2026-02-21T22:33:54.611 INFO:teuthology.orchestra.run.trial102.stdout:*155.248.196.28 132.163.97.4 2 u 54 128 377 67.774 -2.391 0.743 2026-02-21T22:33:54.611 INFO:teuthology.orchestra.run.trial102.stdout:+50.218.103.254 50.205.57.38 2 u 121 128 377 9.375 -4.837 0.591 2026-02-21T22:33:54.630 INFO:teuthology.orchestra.run.trial178.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T22:33:54.631 INFO:teuthology.orchestra.run.trial178.stdout:============================================================================== 2026-02-21T22:33:54.631 INFO:teuthology.orchestra.run.trial178.stdout:-172-233-155-39. 40.119.6.228 4 u 48 64 377 72.098 -3.283 1.645 2026-02-21T22:33:54.631 INFO:teuthology.orchestra.run.trial178.stdout:+time1.lshiy.com 216.239.35.4 2 u 63 64 377 186.334 -5.936 2.121 2026-02-21T22:33:54.631 INFO:teuthology.orchestra.run.trial178.stdout:+LAX.CALTICK.NET 17.253.83.251 2 u 46 64 377 61.720 -6.790 2.206 2026-02-21T22:33:54.631 INFO:teuthology.orchestra.run.trial178.stdout:*69.172.133.130 68.234.48.70 2 u 62 64 377 4.717 -5.927 1.293 2026-02-21T22:33:54.631 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T22:33:54.642 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T22:33:54.642 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T22:33:54.656 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T22:33:54.670 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T22:33:54.684 INFO:teuthology.task.internal:Duration was 1897.106495 seconds 2026-02-21T22:33:54.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T22:33:54.699 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T22:33:54.699 DEBUG:teuthology.orchestra.run.trial102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T22:33:54.701 DEBUG:teuthology.orchestra.run.trial178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T22:33:54.727 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T22:33:54.727 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial102.front.sepia.ceph.com 2026-02-21T22:33:54.727 DEBUG:teuthology.orchestra.run.trial102:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T22:33:54.773 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial178.front.sepia.ceph.com 2026-02-21T22:33:54.773 DEBUG:teuthology.orchestra.run.trial178:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T22:33:54.778 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T22:33:54.778 DEBUG:teuthology.orchestra.run.trial102:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:33:54.816 DEBUG:teuthology.orchestra.run.trial178:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:33:55.045 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T22:33:55.045 DEBUG:teuthology.orchestra.run.trial102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:33:55.046 DEBUG:teuthology.orchestra.run.trial178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T22:33:55.054 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:33:55.054 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T22:33:55.054 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:33:55.054 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T22:33:55.054 INFO:teuthology.orchestra.run.trial102.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T22:33:55.055 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:33:55.055 INFO:teuthology.orchestra.run.trial178.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T22:33:55.055 INFO:teuthology.orchestra.run.trial178.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T22:33:55.055 INFO:teuthology.orchestra.run.trial102.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T22:33:55.055 INFO:teuthology.orchestra.run.trial178.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T22:33:55.122 INFO:teuthology.orchestra.run.trial102.stderr: 81.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T22:33:55.123 INFO:teuthology.orchestra.run.trial178.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T22:33:55.125 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T22:33:55.139 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T22:33:55.139 DEBUG:teuthology.orchestra.run.trial102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T22:33:55.175 DEBUG:teuthology.orchestra.run.trial178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T22:33:55.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T22:33:55.194 DEBUG:teuthology.orchestra.run.trial102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:33:55.216 DEBUG:teuthology.orchestra.run.trial178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:33:55.223 INFO:teuthology.orchestra.run.trial102.stdout:kernel.core_pattern = core 2026-02-21T22:33:55.238 INFO:teuthology.orchestra.run.trial178.stdout:kernel.core_pattern = core 2026-02-21T22:33:55.248 DEBUG:teuthology.orchestra.run.trial102:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:33:55.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:33:55.279 DEBUG:teuthology.orchestra.run.trial178:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T22:33:55.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T22:33:55.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T22:33:55.312 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T22:33:55.313 DEBUG:teuthology.misc:Transferring archived files from trial102:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial102 2026-02-21T22:33:55.313 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T22:33:55.362 DEBUG:teuthology.misc:Transferring archived files from trial178:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63058/remote/trial178 2026-02-21T22:33:55.362 DEBUG:teuthology.orchestra.run.trial178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T22:33:55.401 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T22:33:55.401 DEBUG:teuthology.orchestra.run.trial102:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T22:33:55.403 DEBUG:teuthology.orchestra.run.trial178:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T22:33:55.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T22:33:55.446 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T22:33:55.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T22:33:55.466 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T22:33:55.466 DEBUG:teuthology.orchestra.run.trial102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T22:33:55.468 DEBUG:teuthology.orchestra.run.trial178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T22:33:55.471 INFO:teuthology.orchestra.run.trial102.stdout: 1179659 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 22:33 /home/ubuntu/cephtest 2026-02-21T22:33:55.474 INFO:teuthology.orchestra.run.trial178.stdout: 1179659 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 22:33 /home/ubuntu/cephtest 2026-02-21T22:33:55.476 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T22:33:55.489 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T22:33:55.528 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/3-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{bitmap} base mem$/{low} onode-segment$/{512K-onoff} write$/{random/{compr$/{yes$/{zstd}} random}}}} rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/snaps-few-objects} duration: 1897.1064953804016 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-21T22:33:55.528 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T22:33:55.641 INFO:teuthology.run:pass